Evolving Code with A Large Language Model

Evolving Code with A Large Language Model

13 Jan 2024 | Erik Hemberg, Stephen Moskal, Una-May O'Reilly
The paper introduces LLM\_GP, a formalized evolutionary algorithm that uses Large Language Models (LLMs) to evolve code. Unlike traditional Genetic Programming (GP), LLM\_GP employs evolutionary operators that leverage LLMs' pre-trained pattern matching and sequence completion capabilities. The authors present a demonstration-level variant of LLM\_GP and share its code, aiming to demystify the approach and provide a starting point for exploration. They discuss the design and implementation of LLM-based operators, prompt functions, and preparatory steps, highlighting the challenges and considerations in using LLMs for genetic programming. The paper also reviews existing work at the intersection of EAs and LLMs, focusing on LLM\_GP variants, and provides an experimental setup and analysis to evaluate the performance and costs of LLM\_GP. Despite the risks and challenges, the authors argue that LLM\_GP investigations should continue due to the potential benefits and novel insights they offer. They suggest guidelines for conducting LLM\_GP research, emphasizing the need for clear reporting, well-aligned comparisons, and ethical usage.The paper introduces LLM\_GP, a formalized evolutionary algorithm that uses Large Language Models (LLMs) to evolve code. Unlike traditional Genetic Programming (GP), LLM\_GP employs evolutionary operators that leverage LLMs' pre-trained pattern matching and sequence completion capabilities. The authors present a demonstration-level variant of LLM\_GP and share its code, aiming to demystify the approach and provide a starting point for exploration. They discuss the design and implementation of LLM-based operators, prompt functions, and preparatory steps, highlighting the challenges and considerations in using LLMs for genetic programming. The paper also reviews existing work at the intersection of EAs and LLMs, focusing on LLM\_GP variants, and provides an experimental setup and analysis to evaluate the performance and costs of LLM\_GP. Despite the risks and challenges, the authors argue that LLM\_GP investigations should continue due to the potential benefits and novel insights they offer. They suggest guidelines for conducting LLM\_GP research, emphasizing the need for clear reporting, well-aligned comparisons, and ethical usage.
Reach us at info@study.space