Sampling in Unit Time with Kernel Fisher–Rao Flow

Sampling in Unit Time with Kernel Fisher–Rao Flow

5 Jun 2024 | Aimee Maurais, Youssef Marzouk
This paper introduces a novel mean-field ordinary differential equation (ODE) and corresponding interacting particle systems (IPS) for sampling from an unnormalized target density. The IPS are gradient-free and require only the ability to sample from a reference density and compute the ratio of the target-to-reference density. The mean-field ODE is derived by solving a Poisson equation for a velocity field that transports samples along the geometric mixture of the two densities, which corresponds to the path of a Fisher–Rao gradient flow. The velocity field is approximated using a reproducing kernel Hilbert space (RKHS) ansatz, making the Poisson equation tractable and enabling discretization over finite samples. The mean-field ODE can also be derived from a discrete-time perspective as the limit of successive linearizations of the Monge–Ampère equations within sample-driven optimal transport. The paper introduces a stochastic variant of the approach and demonstrates empirically that the IPS can produce high-quality samples from various target distributions, outperforming comparable gradient-free particle systems and competing with gradient-based alternatives. The methodology is applicable to a wide range of sampling problems, including Bayesian inference and data assimilation, and offers a flexible and efficient approach to sampling from complex distributions.This paper introduces a novel mean-field ordinary differential equation (ODE) and corresponding interacting particle systems (IPS) for sampling from an unnormalized target density. The IPS are gradient-free and require only the ability to sample from a reference density and compute the ratio of the target-to-reference density. The mean-field ODE is derived by solving a Poisson equation for a velocity field that transports samples along the geometric mixture of the two densities, which corresponds to the path of a Fisher–Rao gradient flow. The velocity field is approximated using a reproducing kernel Hilbert space (RKHS) ansatz, making the Poisson equation tractable and enabling discretization over finite samples. The mean-field ODE can also be derived from a discrete-time perspective as the limit of successive linearizations of the Monge–Ampère equations within sample-driven optimal transport. The paper introduces a stochastic variant of the approach and demonstrates empirically that the IPS can produce high-quality samples from various target distributions, outperforming comparable gradient-free particle systems and competing with gradient-based alternatives. The methodology is applicable to a wide range of sampling problems, including Bayesian inference and data assimilation, and offers a flexible and efficient approach to sampling from complex distributions.
Reach us at info@study.space
[slides] Sampling in Unit Time with Kernel Fisher-Rao Flow | StudySpace