This paper proposes a new method called Decoupled Annealing Posterior Sampling (DAPS) for solving Bayesian inverse problems, particularly those with complex nonlinear measurement processes such as phase retrieval. DAPS introduces a novel noise annealing process that decouples consecutive steps in a diffusion sampling trajectory, allowing them to vary considerably while ensuring their time-marginals anneal to the true posterior as noise levels decrease. This approach enables the exploration of a larger solution space, improving the success rate for accurate reconstructions. The method is demonstrated to significantly improve sample quality and stability across multiple image restoration tasks, particularly in complex nonlinear inverse problems. For example, DAPS achieves a PSNR of 30.72dB on the FFHQ 256 dataset for phase retrieval, which is an improvement of 9.12dB compared to existing methods. The method can be combined with diffusion models in both raw pixel space and learned latent space, referred to as DAPS and LatentDAPS, respectively. The method is shown to provide superior reconstructions with improved visual perceptual quality across a wide range of nonlinear inverse problems. DAPS exhibits remarkable stability and sampling quality, particularly for challenging nonlinear inverse problems. On the FFHQ 256 dataset, DAPS achieves a 30.72dB PSNR for noisy phase retrieval, which is 9.12dB higher than all existing methods, while on the ImageNet dataset, it achieves a 25.78dB PSNR for phase retrieval, surpassing others by 5.24dB. Additionally, DAPS performs well even with a very small number of neural network evaluations, striking a better balance between efficiency and sample quality. The method is shown to be effective in solving various inverse problems, including linear and nonlinear tasks such as super-resolution, deblurring, inpainting, and phase retrieval. The method is compared with existing approaches and shown to outperform them in terms of sample quality, stability, and performance on complex inverse problems. The method is also shown to be effective in latent diffusion models, where it can be naturally extended to sampling with latent diffusion models. The method is evaluated on multiple datasets and tasks, demonstrating its effectiveness in solving a wide range of inverse problems.This paper proposes a new method called Decoupled Annealing Posterior Sampling (DAPS) for solving Bayesian inverse problems, particularly those with complex nonlinear measurement processes such as phase retrieval. DAPS introduces a novel noise annealing process that decouples consecutive steps in a diffusion sampling trajectory, allowing them to vary considerably while ensuring their time-marginals anneal to the true posterior as noise levels decrease. This approach enables the exploration of a larger solution space, improving the success rate for accurate reconstructions. The method is demonstrated to significantly improve sample quality and stability across multiple image restoration tasks, particularly in complex nonlinear inverse problems. For example, DAPS achieves a PSNR of 30.72dB on the FFHQ 256 dataset for phase retrieval, which is an improvement of 9.12dB compared to existing methods. The method can be combined with diffusion models in both raw pixel space and learned latent space, referred to as DAPS and LatentDAPS, respectively. The method is shown to provide superior reconstructions with improved visual perceptual quality across a wide range of nonlinear inverse problems. DAPS exhibits remarkable stability and sampling quality, particularly for challenging nonlinear inverse problems. On the FFHQ 256 dataset, DAPS achieves a 30.72dB PSNR for noisy phase retrieval, which is 9.12dB higher than all existing methods, while on the ImageNet dataset, it achieves a 25.78dB PSNR for phase retrieval, surpassing others by 5.24dB. Additionally, DAPS performs well even with a very small number of neural network evaluations, striking a better balance between efficiency and sample quality. The method is shown to be effective in solving various inverse problems, including linear and nonlinear tasks such as super-resolution, deblurring, inpainting, and phase retrieval. The method is compared with existing approaches and shown to outperform them in terms of sample quality, stability, and performance on complex inverse problems. The method is also shown to be effective in latent diffusion models, where it can be naturally extended to sampling with latent diffusion models. The method is evaluated on multiple datasets and tasks, demonstrating its effectiveness in solving a wide range of inverse problems.