The paper introduces Llambo, a novel approach that integrates Large Language Models (LLMs) into Bayesian optimization (BO) to enhance its efficiency and effectiveness. BO is a powerful method for optimizing complex, expensive-to-evaluate functions, but balancing exploration and exploitation remains a challenge. Llambo leverages LLMs' capabilities in natural language processing, including contextual understanding, few-shot learning, and domain knowledge, to improve key components of BO: warmstarting, surrogate modeling, and candidate point sampling. 1. **Warmstarting**: Llambo uses zero-shot prompting to initialize the optimization process with promising initial points, enhancing search performance and diversity. 2. **Surrogate Modeling**: Llambo employs ICL to enhance the discriminative and generative approaches for surrogate modeling, improving prediction accuracy and uncertainty calibration. 3. **Candidate Point Sampling**: Llambo conditionally generates candidate points based on desired objective values, achieving high-quality points while balancing diversity. Empirical results on hyperparameter tuning tasks demonstrate that Llambo outperforms existing methods, especially in scenarios with limited observations. The approach is modular, allowing individual components to be integrated into existing BO frameworks or used as an end-to-end method. The study highlights the potential of LLMs in enhancing BO, particularly in few-shot learning and sample-efficient exploration.The paper introduces Llambo, a novel approach that integrates Large Language Models (LLMs) into Bayesian optimization (BO) to enhance its efficiency and effectiveness. BO is a powerful method for optimizing complex, expensive-to-evaluate functions, but balancing exploration and exploitation remains a challenge. Llambo leverages LLMs' capabilities in natural language processing, including contextual understanding, few-shot learning, and domain knowledge, to improve key components of BO: warmstarting, surrogate modeling, and candidate point sampling. 1. **Warmstarting**: Llambo uses zero-shot prompting to initialize the optimization process with promising initial points, enhancing search performance and diversity. 2. **Surrogate Modeling**: Llambo employs ICL to enhance the discriminative and generative approaches for surrogate modeling, improving prediction accuracy and uncertainty calibration. 3. **Candidate Point Sampling**: Llambo conditionally generates candidate points based on desired objective values, achieving high-quality points while balancing diversity. Empirical results on hyperparameter tuning tasks demonstrate that Llambo outperforms existing methods, especially in scenarios with limited observations. The approach is modular, allowing individual components to be integrated into existing BO frameworks or used as an end-to-end method. The study highlights the potential of LLMs in enhancing BO, particularly in few-shot learning and sample-efficient exploration.