This paper presents a rigorous treatment of entropy-regularized fine-tuning in continuous-time diffusion models, building on recent work by Uehara et al. (2024). The approach uses stochastic control to generate samples, incorporating an entropy regularizer to prevent reward collapse. The analysis is extended to fine-tuning with general f-divergence regularizers. The key idea is to use entropy regularization to improve diversity and avoid overfitting to limited reward signals. The paper introduces a stochastic control framework to emulate the entropy-regularized distribution, where both the control and initial distribution are decision variables. The resulting problem involves decoupling these variables by first solving a standard control problem and then finding the optimal initial distribution. The paper also extends the analysis to f-divergence regularization, showing how the results carry over. Theoretical results are provided, including bounds on the total variation distance between the fine-tuned and original distributions. The paper concludes with a discussion of potential extensions and applications to real-world data generation tasks.This paper presents a rigorous treatment of entropy-regularized fine-tuning in continuous-time diffusion models, building on recent work by Uehara et al. (2024). The approach uses stochastic control to generate samples, incorporating an entropy regularizer to prevent reward collapse. The analysis is extended to fine-tuning with general f-divergence regularizers. The key idea is to use entropy regularization to improve diversity and avoid overfitting to limited reward signals. The paper introduces a stochastic control framework to emulate the entropy-regularized distribution, where both the control and initial distribution are decision variables. The resulting problem involves decoupling these variables by first solving a standard control problem and then finding the optimal initial distribution. The paper also extends the analysis to f-divergence regularization, showing how the results carry over. Theoretical results are provided, including bounds on the total variation distance between the fine-tuned and original distributions. The paper concludes with a discussion of potential extensions and applications to real-world data generation tasks.