Challenging Data for Stereo and Optical Flow

Abstract

Selected scenes for stereo disparity and optical flow estimation containing yet unsolved challenges. Dataset containing 11 challenging sequences for stereo and optical flow estimation. 

Introduction

Currently only few test sequences for optical flow and stereo are available. Most of these show highly controlled indoor scenes and do not contain the complexity that is commonly encountered in outdoor environments. Our aim is to provide new, challenging outdoor data to stimulate research in computer vision.We acquired several million frames with a carefully devised stereo camera system. The recorded scenes provide a huge variety of different weather conditions, different motion and depth layers; they contain city and countryside situations and were acquired at night and at day. From this large quantity of data we selected eleven scenes, each containing a different challenge, highlighting problems that occur regularly.We estimated optical flow and stereo on 10.000 manually selected frames and found that state-of-the-art algorithms frequently fail to estimate reliable correspondences in situations that are summarized in the selected scenes. We observed that these situations fundamentally violate common model assumptions such as brightness constancy and single motion per pixel. With access to this highly challenging data, we also like to encourage alternative approaches that open up new ways to deal with the occuring problems.On this webpage, the challenges in these frames are described and links for the download of the sequences are provided. To keep the data managable, each scene contains about 30 frames and keyframes are named for which the described phenomenom is most explicit. Furthermore we provide a minute description of the recoding and calibration procedures and additionally we supply code to simplify the dealing with the data and the visualization of the results. 

This dataset is part of the robust vision challenge: click here for more information!