This paper introduces a novel population-based optimization method called Language-Model-Based Evolutionary Optimizer (LEO) that leverages Large Language Models (LLMs) for numerical optimization. The authors argue that LLMs can perform zero-shot optimization across diverse scenarios, including multi-objective and high-dimensional problems. LEO employs an elitist framework with separate explore and exploit pools to guide the optimization process, helping to mitigate the risk of getting stuck in local optima. The method is evaluated through various benchmark and industrial engineering problems, such as supersonic nozzle shape optimization, heat transfer, and windfarm layout optimization. The results show that LEO performs comparably to state-of-the-art optimization methods, both gradient-based and gradient-free, while highlighting the need for careful handling due to LLMs' imaginative nature and tendency to hallucinate. The paper provides practical guidelines for obtaining reliable answers from LLMs and discusses limitations and potential research directions. Key contributions include the introduction of LEO, its comparison with other optimization methods, and its application to a range of optimization problems.This paper introduces a novel population-based optimization method called Language-Model-Based Evolutionary Optimizer (LEO) that leverages Large Language Models (LLMs) for numerical optimization. The authors argue that LLMs can perform zero-shot optimization across diverse scenarios, including multi-objective and high-dimensional problems. LEO employs an elitist framework with separate explore and exploit pools to guide the optimization process, helping to mitigate the risk of getting stuck in local optima. The method is evaluated through various benchmark and industrial engineering problems, such as supersonic nozzle shape optimization, heat transfer, and windfarm layout optimization. The results show that LEO performs comparably to state-of-the-art optimization methods, both gradient-based and gradient-free, while highlighting the need for careful handling due to LLMs' imaginative nature and tendency to hallucinate. The paper provides practical guidelines for obtaining reliable answers from LLMs and discusses limitations and potential research directions. Key contributions include the introduction of LEO, its comparison with other optimization methods, and its application to a range of optimization problems.