This paper addresses the challenge of improving diffusion models for solving noisy linear inverse problems without retraining. Recent methods have been interpreted as using Gaussian approximations with hand-crafted isotropic covariance to approximate the intractable denoising posterior. The authors propose a principled approach to enhance these methods by optimizing the posterior covariance using maximum likelihood estimation (MLE). They provide plug-and-play solutions for optimizing posterior covariance without retraining, leveraging pre-trained unconditional diffusion models with or without reverse covariance. Additionally, they introduce a scalable method for learning posterior covariance prediction using orthonormal basis representations. Experimental results demonstrate that the proposed methods significantly improve reconstruction performance across various tasks, including inpainting, deblurring, and super-resolution, without requiring hyperparameter tuning.This paper addresses the challenge of improving diffusion models for solving noisy linear inverse problems without retraining. Recent methods have been interpreted as using Gaussian approximations with hand-crafted isotropic covariance to approximate the intractable denoising posterior. The authors propose a principled approach to enhance these methods by optimizing the posterior covariance using maximum likelihood estimation (MLE). They provide plug-and-play solutions for optimizing posterior covariance without retraining, leveraging pre-trained unconditional diffusion models with or without reverse covariance. Additionally, they introduce a scalable method for learning posterior covariance prediction using orthonormal basis representations. Experimental results demonstrate that the proposed methods significantly improve reconstruction performance across various tasks, including inpainting, deblurring, and super-resolution, without requiring hyperparameter tuning.