FISHER-FLOW is a novel generative model for discrete data that leverages the Fisher-Rao metric on the probability simplex. The model reparameterizes discrete data as points on the positive orthant of a $d$-dimensional hypersphere, allowing for principled flow-matching between source and target distributions. Key contributions include: 1. **Geometric Perspective**: FISHER-FLOW considers categorical distributions over discrete data as points on a statistical manifold equipped with the Fisher-Rao metric. 2. **Continuous Reparameterization**: Discrete data is continuously reparameterized to the positive orthant of the hypersphere, enabling the definition of flows that map any source distribution to a target distribution. 3. **Riemannian Optimal Transport**: The learned flows can be further optimized using Riemannian optimal transport, leading to improved training dynamics and better performance. 4. **Theoretical Justification**: The gradient flow induced by FISHER-FLOW is shown to be optimal in reducing the forward KL divergence. 5. **Empirical Performance**: FISHER-FLOW outperforms prior diffusion and flow-matching models on synthetic and real-world benchmarks, including DNA promoter and enhancer sequence design. The paper also discusses the background on information geometry, flow matching on Riemannian manifolds, and the sphere map, which is a diffeomorphism between the interior of the probability simplex and the positive orthant of the hypersphere. The theoretical and empirical results demonstrate the effectiveness of FISHER-FLOW in various generative tasks.FISHER-FLOW is a novel generative model for discrete data that leverages the Fisher-Rao metric on the probability simplex. The model reparameterizes discrete data as points on the positive orthant of a $d$-dimensional hypersphere, allowing for principled flow-matching between source and target distributions. Key contributions include: 1. **Geometric Perspective**: FISHER-FLOW considers categorical distributions over discrete data as points on a statistical manifold equipped with the Fisher-Rao metric. 2. **Continuous Reparameterization**: Discrete data is continuously reparameterized to the positive orthant of the hypersphere, enabling the definition of flows that map any source distribution to a target distribution. 3. **Riemannian Optimal Transport**: The learned flows can be further optimized using Riemannian optimal transport, leading to improved training dynamics and better performance. 4. **Theoretical Justification**: The gradient flow induced by FISHER-FLOW is shown to be optimal in reducing the forward KL divergence. 5. **Empirical Performance**: FISHER-FLOW outperforms prior diffusion and flow-matching models on synthetic and real-world benchmarks, including DNA promoter and enhancer sequence design. The paper also discusses the background on information geometry, flow matching on Riemannian manifolds, and the sphere map, which is a diffeomorphism between the interior of the probability simplex and the positive orthant of the hypersphere. The theoretical and empirical results demonstrate the effectiveness of FISHER-FLOW in various generative tasks.