This paper proposes a new set of benchmarks and evaluation methods for optical flow algorithms, addressing the challenges posed by complex natural scenes, including nonrigid motion, real sensor noise, and motion discontinuities. The authors contribute four types of data to test different aspects of optical flow algorithms: sequences with nonrigid motion, realistic synthetic sequences, high frame-rate video for interpolation error study, and modified stereo sequences of static scenes. They extend the evaluation metrics to include average angular error, flow endpoint error, frame interpolation error, robustness measures, and results at motion discontinuities and in textureless regions. The data is freely available online, and the performance of several well-known methods on preliminary versions of the data has been published, leading to significant improvements. The paper analyzes the results obtained so far and draws conclusions from them. The introduction highlights the importance of datasets for evaluating optical flow algorithms, and the related work section provides a taxonomy of optical flow algorithms, focusing on recent developments and their context. The database design section explains the collection of four different types of data to address various challenges in optical flow estimation.This paper proposes a new set of benchmarks and evaluation methods for optical flow algorithms, addressing the challenges posed by complex natural scenes, including nonrigid motion, real sensor noise, and motion discontinuities. The authors contribute four types of data to test different aspects of optical flow algorithms: sequences with nonrigid motion, realistic synthetic sequences, high frame-rate video for interpolation error study, and modified stereo sequences of static scenes. They extend the evaluation metrics to include average angular error, flow endpoint error, frame interpolation error, robustness measures, and results at motion discontinuities and in textureless regions. The data is freely available online, and the performance of several well-known methods on preliminary versions of the data has been published, leading to significant improvements. The paper analyzes the results obtained so far and draws conclusions from them. The introduction highlights the importance of datasets for evaluating optical flow algorithms, and the related work section provides a taxonomy of optical flow algorithms, focusing on recent developments and their context. The database design section explains the collection of four different types of data to address various challenges in optical flow estimation.