January 15, 2024 | Xingyu Wu, Sheng-hao Wu*, Jibin Wu*, Liang Feng*, Senior Member, IEEE, Kay Chen Tan, Fellow, IEEE
This paper provides a comprehensive survey and roadmap of the integration of large language models (LLMs) and evolutionary algorithms (EAs) in the era of LLMs. It categorizes the reciprocal inspiration into two main avenues: LLM-enhanced EA and EA-enhanced LLM. The paper discusses the complementary advantages of LLMs and EAs, including LLMs' ability to provide domain knowledge for EA searches and EAs' ability to enhance LLMs' optimization and search capabilities. It also introduces integrated synergy methods that exemplify the complementarity between LLMs and EAs in various scenarios, such as code generation, software engineering, neural architecture search, and generation tasks. The paper identifies challenges and future directions for researchers and practitioners to unlock the full potential of this innovative collaboration in advancing optimization and artificial intelligence. It also provides a GitHub repository for indexing relevant papers. The paper discusses the potential of LLMs in solving optimization problems, including LLM-assisted black-box optimization as search operators and LLM-assisted optimization algorithm generation. It also explores EA-based prompt engineering and EA-based LLM architecture search, highlighting their roles in enhancing LLM performance and expanding their applications. The paper concludes that while initial studies are promising, significant barriers remain for applying LLMs to real-world complex optimization problems.This paper provides a comprehensive survey and roadmap of the integration of large language models (LLMs) and evolutionary algorithms (EAs) in the era of LLMs. It categorizes the reciprocal inspiration into two main avenues: LLM-enhanced EA and EA-enhanced LLM. The paper discusses the complementary advantages of LLMs and EAs, including LLMs' ability to provide domain knowledge for EA searches and EAs' ability to enhance LLMs' optimization and search capabilities. It also introduces integrated synergy methods that exemplify the complementarity between LLMs and EAs in various scenarios, such as code generation, software engineering, neural architecture search, and generation tasks. The paper identifies challenges and future directions for researchers and practitioners to unlock the full potential of this innovative collaboration in advancing optimization and artificial intelligence. It also provides a GitHub repository for indexing relevant papers. The paper discusses the potential of LLMs in solving optimization problems, including LLM-assisted black-box optimization as search operators and LLM-assisted optimization algorithm generation. It also explores EA-based prompt engineering and EA-based LLM architecture search, highlighting their roles in enhancing LLM performance and expanding their applications. The paper concludes that while initial studies are promising, significant barriers remain for applying LLMs to real-world complex optimization problems.