The paper addresses the problem of constrained efficient global optimization (CEGO) for expensive black-box functions, where both the objective and constraints are unknown and can be learned using Gaussian processes. The proposed algorithm, CONFIG (CONstrained efFicient Global Optimization), is designed to handle this problem by exploiting the principle of optimism in the face of uncertainty. Under certain regularity assumptions, CONFIG achieves the same cumulative regret bound as in the unconstrained case and similar cumulative constraint violation upper bounds. For commonly used Matérn and Squared Exponential kernels, the bounds are sublinear, allowing for a convergence rate to the optimal solution of the original constrained problem. CONFIG also naturally provides a scheme to declare infeasibility when the original problem is infeasible. Numerical experiments on sampled instances from Gaussian processes, artificial numerical problems, and a black-box building controller tuning problem demonstrate the competitive performance of CONFIG, showing significant improvements in theoretical guarantees and empirical performance compared to state-of-the-art methods.The paper addresses the problem of constrained efficient global optimization (CEGO) for expensive black-box functions, where both the objective and constraints are unknown and can be learned using Gaussian processes. The proposed algorithm, CONFIG (CONstrained efFicient Global Optimization), is designed to handle this problem by exploiting the principle of optimism in the face of uncertainty. Under certain regularity assumptions, CONFIG achieves the same cumulative regret bound as in the unconstrained case and similar cumulative constraint violation upper bounds. For commonly used Matérn and Squared Exponential kernels, the bounds are sublinear, allowing for a convergence rate to the optimal solution of the original constrained problem. CONFIG also naturally provides a scheme to declare infeasibility when the original problem is infeasible. Numerical experiments on sampled instances from Gaussian processes, artificial numerical problems, and a black-box building controller tuning problem demonstrate the competitive performance of CONFIG, showing significant improvements in theoretical guarantees and empirical performance compared to state-of-the-art methods.