This paper reviews adaptive Markov chain Monte Carlo (MCMC) algorithms, which aim to improve their performance by adapting the proposal distributions. The authors use simple examples to explain the theoretical foundations of adaptive MCMC and show why they might fail if certain properties are not satisfied. They then discuss stochastic approximation, a framework that allows systematic optimization of criteria and analysis of adaptive MCMC properties. The paper proposes novel adaptive algorithms that are robust and reliable in practice, applied to artificial and high-dimensional scenarios, as well as the classic mine disaster dataset inference problem. MCMC is a general strategy for generating samples from complex high-dimensional distributions. The Metropolis-Hastings algorithm is a key building block, requiring a family of proposal distributions. The choice of these distributions is crucial for the performance of the algorithm. In the toy example, the variance of the estimator is unsatisfactory for values of the proposal variance that are too small or too large. In more realistic scenarios, MCMC algorithms are combinations of several MH updates with different proposal distributions, designed to capture various features of the target distribution. The authors aim to review the theoretical underpinnings and recent advances in optimizing parametrized MCMC transition probabilities. They suggest new algorithms and note that in some cases, the invariant distribution of the transition probabilities might depend on the parameter. However, most arguments and ideas presented here generally apply to this more complex scenario. The choice of a criterion to optimize is the first decision in practice. The authors discuss this in Section 4.1, noting that most sensible optimality or suboptimality criteria can be expressed in terms of expectations with respect to the steady-state distributions of Markov chains. They also suggest new algorithms and illustrate them on examples. To optimize such criteria, one could sequentially run a standard MCMC algorithm for a set of values of the parameter and compute the criterion once equilibrium is reached. However, the authors focus on a technique called controlled MCMC, which belongs to the class of controlled Markov chains. They describe the algorithm and its properties, and discuss its theoretical ergodicity.This paper reviews adaptive Markov chain Monte Carlo (MCMC) algorithms, which aim to improve their performance by adapting the proposal distributions. The authors use simple examples to explain the theoretical foundations of adaptive MCMC and show why they might fail if certain properties are not satisfied. They then discuss stochastic approximation, a framework that allows systematic optimization of criteria and analysis of adaptive MCMC properties. The paper proposes novel adaptive algorithms that are robust and reliable in practice, applied to artificial and high-dimensional scenarios, as well as the classic mine disaster dataset inference problem. MCMC is a general strategy for generating samples from complex high-dimensional distributions. The Metropolis-Hastings algorithm is a key building block, requiring a family of proposal distributions. The choice of these distributions is crucial for the performance of the algorithm. In the toy example, the variance of the estimator is unsatisfactory for values of the proposal variance that are too small or too large. In more realistic scenarios, MCMC algorithms are combinations of several MH updates with different proposal distributions, designed to capture various features of the target distribution. The authors aim to review the theoretical underpinnings and recent advances in optimizing parametrized MCMC transition probabilities. They suggest new algorithms and note that in some cases, the invariant distribution of the transition probabilities might depend on the parameter. However, most arguments and ideas presented here generally apply to this more complex scenario. The choice of a criterion to optimize is the first decision in practice. The authors discuss this in Section 4.1, noting that most sensible optimality or suboptimality criteria can be expressed in terms of expectations with respect to the steady-state distributions of Markov chains. They also suggest new algorithms and illustrate them on examples. To optimize such criteria, one could sequentially run a standard MCMC algorithm for a set of values of the parameter and compute the criterion once equilibrium is reached. However, the authors focus on a technique called controlled MCMC, which belongs to the class of controlled Markov chains. They describe the algorithm and its properties, and discuss its theoretical ergodicity.