Bayesian optimization is a method for optimizing expensive-to-evaluate black-box functions. It uses Gaussian process regression to model the objective function and an acquisition function to decide where to sample next. The acquisition function, such as expected improvement, entropy search, or knowledge gradient, balances exploration and exploitation to find the global optimum. The method is particularly effective for continuous domains with less than 20 dimensions and can handle noisy evaluations. It has been widely applied in machine learning, engineering, and other fields. The paper discusses Gaussian process regression, acquisition functions, and advanced techniques like multi-fidelity optimization, parallel evaluations, and multi-task optimization. It also covers the inclusion of derivative information and the use of Bayesian optimization software. The paper introduces a generalized expected improvement for noisy evaluations and argues that the knowledge gradient acquisition function is the most natural for noisy measurements. The paper concludes with future research directions in Bayesian optimization.Bayesian optimization is a method for optimizing expensive-to-evaluate black-box functions. It uses Gaussian process regression to model the objective function and an acquisition function to decide where to sample next. The acquisition function, such as expected improvement, entropy search, or knowledge gradient, balances exploration and exploitation to find the global optimum. The method is particularly effective for continuous domains with less than 20 dimensions and can handle noisy evaluations. It has been widely applied in machine learning, engineering, and other fields. The paper discusses Gaussian process regression, acquisition functions, and advanced techniques like multi-fidelity optimization, parallel evaluations, and multi-task optimization. It also covers the inclusion of derivative information and the use of Bayesian optimization software. The paper introduces a generalized expected improvement for noisy evaluations and argues that the knowledge gradient acquisition function is the most natural for noisy measurements. The paper concludes with future research directions in Bayesian optimization.