This paper introduces a novel population-based optimization method called Language-Model-Based Evolutionary Optimizer (LEO), leveraging Large Language Models (LLMs) for zero-shot optimization across diverse scenarios, including multi-objective and high-dimensional problems. LEO employs an elitist framework with separate explore and exploit pools of solutions to guide the optimization process. The method is evaluated on benchmark and industrial engineering problems such as supersonic nozzle shape optimization, heat transfer, and windfarm layout optimization. LEO is compared with gradient-based and gradient-free optimization approaches, showing comparable results to state-of-the-art methods. However, the imaginative nature of LLMs and their tendency to hallucinate require careful handling. The paper provides practical guidelines for obtaining reliable answers from LLMs and discusses method limitations and potential research directions. The methodology involves a population-based approach with a novel explore-exploit policy using LLMs for generating new samples. LEO is tested on various benchmark problems, demonstrating its effectiveness in solving high-dimensional problems and engineering applications. The paper also discusses the reasoning capabilities of LLMs, showing that they can generate better candidate solutions when assisted by an elitism criterion. Challenges associated with using LLMs as optimizers are addressed, including the need for careful handling of hallucinations and the importance of context in generating reliable solutions. The paper concludes that LEO is a promising approach for optimization tasks, particularly in complex, high-dimensional problems.This paper introduces a novel population-based optimization method called Language-Model-Based Evolutionary Optimizer (LEO), leveraging Large Language Models (LLMs) for zero-shot optimization across diverse scenarios, including multi-objective and high-dimensional problems. LEO employs an elitist framework with separate explore and exploit pools of solutions to guide the optimization process. The method is evaluated on benchmark and industrial engineering problems such as supersonic nozzle shape optimization, heat transfer, and windfarm layout optimization. LEO is compared with gradient-based and gradient-free optimization approaches, showing comparable results to state-of-the-art methods. However, the imaginative nature of LLMs and their tendency to hallucinate require careful handling. The paper provides practical guidelines for obtaining reliable answers from LLMs and discusses method limitations and potential research directions. The methodology involves a population-based approach with a novel explore-exploit policy using LLMs for generating new samples. LEO is tested on various benchmark problems, demonstrating its effectiveness in solving high-dimensional problems and engineering applications. The paper also discusses the reasoning capabilities of LLMs, showing that they can generate better candidate solutions when assisted by an elitism criterion. Challenges associated with using LLMs as optimizers are addressed, including the need for careful handling of hallucinations and the importance of context in generating reliable solutions. The paper concludes that LEO is a promising approach for optimization tasks, particularly in complex, high-dimensional problems.