2025 | Hunter Nisonoff, Junhao Xiong, Stephan Allenspach, Jennifer Listgarten
This paper introduces Discrete Guidance, a principled method for applying guidance to diffusion and flow models on discrete state-spaces. The method leverages continuous-time Markov processes on discrete state-spaces to enable computational tractability for sampling from desired guided distributions. The approach is demonstrated on applications including guided generation of small-molecules, DNA sequences, and protein sequences.
Generative models based on diffusion and flow matching have shown great potential in image applications and increasingly in the sciences. However, these methods are not readily applicable to discrete state-spaces. Discrete Guidance addresses this by using continuous-time Markov chains (CTMCs) to model discrete state-spaces, enabling the application of guidance techniques that are typically used in continuous state-spaces.
The key insight of Discrete Guidance is that in continuous time, only a single dimension of the discrete state-space Markov chain can change at any point in time, making guidance exact and tractable. The method is applied to a broad set of discrete state-space conditional generation tasks, including small-molecules, DNA sequences, and protein sequences. The approach is shown to be effective in generating samples that meet desired properties, such as binding affinity or enzymatic activity.
The paper also presents efficient approximations for predictor guidance, such as Taylor-approximated guidance (TAG), which allows for more efficient sampling while maintaining sample quality. The method is compared to existing approaches like DiGress and Dirichlet FM, showing that Discrete Guidance provides better results in terms of sample quality and efficiency.
Empirical investigations demonstrate that Discrete Guidance outperforms other methods in generating samples with desired properties. The method is applied to tasks such as small-molecule generation, DNA enhancer design, and protein inverse-folding, showing its effectiveness in these domains. The results indicate that Discrete Guidance is a promising approach for improving conditional generative modeling in discrete state-spaces, with potential applications in the natural sciences.This paper introduces Discrete Guidance, a principled method for applying guidance to diffusion and flow models on discrete state-spaces. The method leverages continuous-time Markov processes on discrete state-spaces to enable computational tractability for sampling from desired guided distributions. The approach is demonstrated on applications including guided generation of small-molecules, DNA sequences, and protein sequences.
Generative models based on diffusion and flow matching have shown great potential in image applications and increasingly in the sciences. However, these methods are not readily applicable to discrete state-spaces. Discrete Guidance addresses this by using continuous-time Markov chains (CTMCs) to model discrete state-spaces, enabling the application of guidance techniques that are typically used in continuous state-spaces.
The key insight of Discrete Guidance is that in continuous time, only a single dimension of the discrete state-space Markov chain can change at any point in time, making guidance exact and tractable. The method is applied to a broad set of discrete state-space conditional generation tasks, including small-molecules, DNA sequences, and protein sequences. The approach is shown to be effective in generating samples that meet desired properties, such as binding affinity or enzymatic activity.
The paper also presents efficient approximations for predictor guidance, such as Taylor-approximated guidance (TAG), which allows for more efficient sampling while maintaining sample quality. The method is compared to existing approaches like DiGress and Dirichlet FM, showing that Discrete Guidance provides better results in terms of sample quality and efficiency.
Empirical investigations demonstrate that Discrete Guidance outperforms other methods in generating samples with desired properties. The method is applied to tasks such as small-molecule generation, DNA enhancer design, and protein inverse-folding, showing its effectiveness in these domains. The results indicate that Discrete Guidance is a promising approach for improving conditional generative modeling in discrete state-spaces, with potential applications in the natural sciences.