The paper introduces a local reparameterization technique to reduce the variance of stochastic gradients in variational Bayesian inference (SGVB) for model parameters, while maintaining parallelizability. This technique translates global parameter uncertainty into local noise that is independent across data points in the minibatch, leading to faster convergence. The method is demonstrated through experiments and is shown to be a generalization of Gaussian dropout, where the dropout rates are learned, often resulting in better models. The paper also explores the connection between dropout and Bayesian inference, showing that Gaussian dropout corresponds to SGVB with local reparameterization, a scale-invariant prior, and proportionally fixed posterior variance. The proposed method allows for more flexible posterior inference and is evaluated on various benchmarks, showing improved performance over standard dropout methods.The paper introduces a local reparameterization technique to reduce the variance of stochastic gradients in variational Bayesian inference (SGVB) for model parameters, while maintaining parallelizability. This technique translates global parameter uncertainty into local noise that is independent across data points in the minibatch, leading to faster convergence. The method is demonstrated through experiments and is shown to be a generalization of Gaussian dropout, where the dropout rates are learned, often resulting in better models. The paper also explores the connection between dropout and Bayesian inference, showing that Gaussian dropout corresponds to SGVB with local reparameterization, a scale-invariant prior, and proportionally fixed posterior variance. The proposed method allows for more flexible posterior inference and is evaluated on various benchmarks, showing improved performance over standard dropout methods.