19 Mar 2024 | Lingzhe Zhao, Peng Wang, and Peidong Liu
BAD-Gaussians is a novel method for deblurring motion-blurred images using 3D Gaussian Splatting. The method addresses the challenges of reconstructing high-quality 3D scenes from motion-blurred images with inaccurate camera poses. It models the physical image formation process of motion-blurred images and jointly learns the parameters of Gaussians while recovering camera motion trajectories during exposure time. BAD-Gaussians incorporates the physical process of motion blur into the training of 3D-GS, using a spline function to characterize the trajectory within the camera's exposure time. The camera trajectory within exposure time is optimized using gradients derived from the Gaussians of the scene, while jointly optimizing the Gaussians themselves. The method generates a sequence of virtual sharp images by projecting the scene's Gaussians onto the image plane and averages them to synthesize the blurred images following the physical blur process. The Gaussians along the trajectory are optimized by minimizing the photometric error between the synthesized blurred images and the input blurred images through differentiable Gaussian rasterization. BAD-Gaussians outperforms prior state-of-the-art implicit neural rendering methods on both synthetic and real datasets, achieving better rendering performance in terms of real-time rendering speed and superior rendering quality. The method achieves real-time rendering capabilities and successfully deblurs severe motion-blurred images, synthesizing higher-quality novel view images. The contributions of BAD-Gaussians include introducing a photometric bundle adjustment formulation specifically designed for motion-blurred images, demonstrating how this formulation enables the acquisition of high-quality 3D scene representation from motion-blurred images, and showing that the method successfully deblurs severe motion-blurred images, synthesizes higher-quality novel view images, and achieves real-time rendering, surpassing previous state-of-the-art implicit deblurring rendering methods.BAD-Gaussians is a novel method for deblurring motion-blurred images using 3D Gaussian Splatting. The method addresses the challenges of reconstructing high-quality 3D scenes from motion-blurred images with inaccurate camera poses. It models the physical image formation process of motion-blurred images and jointly learns the parameters of Gaussians while recovering camera motion trajectories during exposure time. BAD-Gaussians incorporates the physical process of motion blur into the training of 3D-GS, using a spline function to characterize the trajectory within the camera's exposure time. The camera trajectory within exposure time is optimized using gradients derived from the Gaussians of the scene, while jointly optimizing the Gaussians themselves. The method generates a sequence of virtual sharp images by projecting the scene's Gaussians onto the image plane and averages them to synthesize the blurred images following the physical blur process. The Gaussians along the trajectory are optimized by minimizing the photometric error between the synthesized blurred images and the input blurred images through differentiable Gaussian rasterization. BAD-Gaussians outperforms prior state-of-the-art implicit neural rendering methods on both synthetic and real datasets, achieving better rendering performance in terms of real-time rendering speed and superior rendering quality. The method achieves real-time rendering capabilities and successfully deblurs severe motion-blurred images, synthesizing higher-quality novel view images. The contributions of BAD-Gaussians include introducing a photometric bundle adjustment formulation specifically designed for motion-blurred images, demonstrating how this formulation enables the acquisition of high-quality 3D scene representation from motion-blurred images, and showing that the method successfully deblurs severe motion-blurred images, synthesizes higher-quality novel view images, and achieves real-time rendering, surpassing previous state-of-the-art implicit deblurring rendering methods.