23 May 2024 | Yihan Wang, Lahav Lipson, and Jia Deng
SEA-RAFT is a simplified, efficient, and accurate version of the RAFT model for optical flow estimation. It improves upon RAFT by using a new loss function, a mixture of Laplace distributions, and direct regression of initial flow. SEA-RAFT achieves state-of-the-art accuracy on the Spring benchmark with a 3.69-point-error (EPE) and a 0.36-1-pixel-outlier rate (1px), representing 22.9% and 17.8% error reduction from the best published results. It also performs best in cross-dataset generalization on KITTI and Spring. SEA-RAFT is significantly faster than existing methods, operating at least 2.3× faster while maintaining competitive performance. The model is publicly available at https://github.com/princeton-vl/SEA-RAFT.
The paper introduces SEA-RAFT, a new variant of RAFT that is more efficient and accurate. It achieves the best accuracy-efficiency Pareto frontier on the Spring benchmark. SEA-RAFT outperforms other methods in accuracy and efficiency, with a 18% error reduction on 1px-outlier rate and 24% error reduction on endpoint-error. It runs at least 2.3× faster than existing methods on each benchmark tested. The model is based on improvements including a mixture of Laplace loss, direct regression of initial flow, and rigid-flow pre-training. These improvements are novel in the context of RAFT-style methods for optical flow.
The paper also discusses the use of mixture of Laplace loss, which reduces overfitting to ambiguous cases and improves generalization. Direct regression of initial flow is used to reduce the number of iterations and improve efficiency. Rigid-flow pre-training is used to improve generalization. These improvements are orthogonal to existing RAFT-style methods, which focus on replacing certain blocks with newer designs.
The paper evaluates SEA-RAFT on standard benchmarks including Spring, Sintel, and KITTI. It also validates the effectiveness of the improvements through ablation studies. The results show that SEA-RAFT achieves state-of-the-art accuracy and efficiency, making it useful for real-world high-resolution optical flow. The model is publicly available and can be used for further research and development.SEA-RAFT is a simplified, efficient, and accurate version of the RAFT model for optical flow estimation. It improves upon RAFT by using a new loss function, a mixture of Laplace distributions, and direct regression of initial flow. SEA-RAFT achieves state-of-the-art accuracy on the Spring benchmark with a 3.69-point-error (EPE) and a 0.36-1-pixel-outlier rate (1px), representing 22.9% and 17.8% error reduction from the best published results. It also performs best in cross-dataset generalization on KITTI and Spring. SEA-RAFT is significantly faster than existing methods, operating at least 2.3× faster while maintaining competitive performance. The model is publicly available at https://github.com/princeton-vl/SEA-RAFT.
The paper introduces SEA-RAFT, a new variant of RAFT that is more efficient and accurate. It achieves the best accuracy-efficiency Pareto frontier on the Spring benchmark. SEA-RAFT outperforms other methods in accuracy and efficiency, with a 18% error reduction on 1px-outlier rate and 24% error reduction on endpoint-error. It runs at least 2.3× faster than existing methods on each benchmark tested. The model is based on improvements including a mixture of Laplace loss, direct regression of initial flow, and rigid-flow pre-training. These improvements are novel in the context of RAFT-style methods for optical flow.
The paper also discusses the use of mixture of Laplace loss, which reduces overfitting to ambiguous cases and improves generalization. Direct regression of initial flow is used to reduce the number of iterations and improve efficiency. Rigid-flow pre-training is used to improve generalization. These improvements are orthogonal to existing RAFT-style methods, which focus on replacing certain blocks with newer designs.
The paper evaluates SEA-RAFT on standard benchmarks including Spring, Sintel, and KITTI. It also validates the effectiveness of the improvements through ablation studies. The results show that SEA-RAFT achieves state-of-the-art accuracy and efficiency, making it useful for real-world high-resolution optical flow. The model is publicly available and can be used for further research and development.