2024 | Louis Grenioux, Maxence Noble, Marylou Gabrié, Alain Durmus
This paper introduces a novel stochastic localization (SL) framework for sampling from unnormalized target densities, with a focus on the Stochastic Localization via Iterative Posterior Sampling (SLIPS) algorithm. SLIPS leverages Monte Carlo estimation of the denoiser to generate approximate samples from the target distribution, without requiring explicit knowledge of the target density. The framework allows for flexible denoising schedules, enabling efficient sampling from complex, multi-modal distributions.
The key contribution of SLIPS is its ability to handle a wide range of target distributions, including those with non-log-concave densities, by exploiting the "duality of log-concavity." This duality refers to the balance between sampling from the observation process and the SL posterior, which is crucial for ensuring efficient and accurate sampling. The algorithm is designed to adapt to different denoising schedules, such as geometric and asymptotic schedules, which control the noise level and sampling efficiency.
SLIPS is evaluated on several benchmark tasks, including multi-modal Gaussian mixtures, Bayesian logistic regression, and a high-dimensional field system from statistical mechanics. The results show that SLIPS is competitive or superior to existing methods in terms of sampling accuracy and efficiency, particularly in high-dimensional settings. The algorithm is also shown to be robust to variations in the target distribution and performs well in capturing local properties of the distribution.
The paper also discusses related work, including score-based sampling with VI, Monte Carlo score estimation, and other sampling methods such as RDMC and OAT. It highlights the advantages of SLIPS in terms of flexibility, efficiency, and applicability to a wide range of target distributions. The algorithm is implemented with practical guidelines for tuning hyper-parameters and is shown to be effective in both low and high-dimensional settings.
Overall, SLIPS provides a versatile and efficient approach to sampling from unnormalized target densities, with theoretical guarantees and practical applicability across various domains. The method's ability to adapt to different denoising schedules and its robustness to complex distributions make it a promising tool for future research in sampling and generative modeling.This paper introduces a novel stochastic localization (SL) framework for sampling from unnormalized target densities, with a focus on the Stochastic Localization via Iterative Posterior Sampling (SLIPS) algorithm. SLIPS leverages Monte Carlo estimation of the denoiser to generate approximate samples from the target distribution, without requiring explicit knowledge of the target density. The framework allows for flexible denoising schedules, enabling efficient sampling from complex, multi-modal distributions.
The key contribution of SLIPS is its ability to handle a wide range of target distributions, including those with non-log-concave densities, by exploiting the "duality of log-concavity." This duality refers to the balance between sampling from the observation process and the SL posterior, which is crucial for ensuring efficient and accurate sampling. The algorithm is designed to adapt to different denoising schedules, such as geometric and asymptotic schedules, which control the noise level and sampling efficiency.
SLIPS is evaluated on several benchmark tasks, including multi-modal Gaussian mixtures, Bayesian logistic regression, and a high-dimensional field system from statistical mechanics. The results show that SLIPS is competitive or superior to existing methods in terms of sampling accuracy and efficiency, particularly in high-dimensional settings. The algorithm is also shown to be robust to variations in the target distribution and performs well in capturing local properties of the distribution.
The paper also discusses related work, including score-based sampling with VI, Monte Carlo score estimation, and other sampling methods such as RDMC and OAT. It highlights the advantages of SLIPS in terms of flexibility, efficiency, and applicability to a wide range of target distributions. The algorithm is implemented with practical guidelines for tuning hyper-parameters and is shown to be effective in both low and high-dimensional settings.
Overall, SLIPS provides a versatile and efficient approach to sampling from unnormalized target densities, with theoretical guarantees and practical applicability across various domains. The method's ability to adapt to different denoising schedules and its robustness to complex distributions make it a promising tool for future research in sampling and generative modeling.