PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume

PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume

25 Jun 2018 | Deqing Sun, Xiaodong Yang, Ming-Yu Liu, and Jan Kautz
PWC-Net is a compact and effective convolutional neural network (CNN) for optical flow estimation, designed using simple and well-established principles: pyramidal processing, warping, and cost volume. It uses a learnable feature pyramid to warp the second image's features based on the current optical flow estimate, then constructs a cost volume from the warped and first image features, which is processed by a CNN to estimate the optical flow. PWC-Net is 17 times smaller and easier to train than FlowNet2, and outperforms all published optical flow methods on the MPI Sintel final pass and KITTI 2015 benchmarks, running at about 35 fps on Sintel resolution images. It achieves a better balance between accuracy and size compared to existing end-to-end CNN models for flow. PWC-Net uses a feature pyramid extractor, a warping layer, a cost volume layer, and a context network to estimate optical flow. The model is trained using a multi-scale loss function and is capable of handling large displacements with a small search range. It outperforms FlowNet2 in accuracy and speed, and is more suitable for memory-limited applications. PWC-Net is also more accurate than traditional methods on the final pass of the Sintel benchmark, which is more realistic and challenging. The model has been tested on various datasets, including Sintel and KITTI, and has shown superior performance in both accuracy and efficiency. PWC-Net is a promising approach for optical flow estimation, combining domain knowledge with deep learning to achieve high accuracy and efficiency.PWC-Net is a compact and effective convolutional neural network (CNN) for optical flow estimation, designed using simple and well-established principles: pyramidal processing, warping, and cost volume. It uses a learnable feature pyramid to warp the second image's features based on the current optical flow estimate, then constructs a cost volume from the warped and first image features, which is processed by a CNN to estimate the optical flow. PWC-Net is 17 times smaller and easier to train than FlowNet2, and outperforms all published optical flow methods on the MPI Sintel final pass and KITTI 2015 benchmarks, running at about 35 fps on Sintel resolution images. It achieves a better balance between accuracy and size compared to existing end-to-end CNN models for flow. PWC-Net uses a feature pyramid extractor, a warping layer, a cost volume layer, and a context network to estimate optical flow. The model is trained using a multi-scale loss function and is capable of handling large displacements with a small search range. It outperforms FlowNet2 in accuracy and speed, and is more suitable for memory-limited applications. PWC-Net is also more accurate than traditional methods on the final pass of the Sintel benchmark, which is more realistic and challenging. The model has been tested on various datasets, including Sintel and KITTI, and has shown superior performance in both accuracy and efficiency. PWC-Net is a promising approach for optical flow estimation, combining domain knowledge with deep learning to achieve high accuracy and efficiency.
Reach us at info@study.space