Automatic Programming: Large Language Models and Beyond

Automatic Programming: Large Language Models and Beyond

15 May 2024 | MICHAEL R. LYU, BAISHAKHI RAY, ABHIK ROYCHOWDHURY, SHIN HWEI TAN, PATANAMON THONGTANUNAM
Automatic programming has gained popularity due to tools like GitHub Copilot, which rely on Large Language Models (LLMs). However, automatically generated code faces challenges in deployment due to concerns about quality and trust. This article examines the issues of code quality, security, and programmer responsibility in automated programming. It discusses how advances in software engineering, such as program repair and analysis, can enable automatic programming. The article also explores the future of programming environments where programmers may need to switch roles to fully utilize automatic programming. Automated repair of LLM-generated code can help produce higher assurance code with evidence of correctness. The article highlights challenges in automatic programming, including the difficulty of capturing user intent and generating correct code. Recent developments in LLM-based code generation raise trust issues, as the correctness of automatically generated code is uncertain. The article discusses the trust issues for code generated from natural language specifications and the need for systematic program repair to ensure correctness. It also explores the impact of LLMs on non-code artifacts and processes like test generation, code review, and code summarization. The article reviews historical milestones and literature in automatic programming, including code generation, program repair, and software testing. It discusses various approaches to code generation, such as deductive, inductive, and natural language-based methods. It also covers program repair techniques, including search-based, constraint-based, and learning-based approaches. The article highlights the potential of LLMs in intelligent programming, including recent large language code models and their applications in code generation and repair. The article discusses the challenges of program repair, including overfitting and the need for generalization. It presents a concrete approach to program repair using symbolic analysis of test cases. The article also explores the use of LLMs in program repair, including the potential of language models to improve the success rate of code generation and repair. It highlights the importance of ensuring the trustworthiness of code generated by LLMs, including concerns about security vulnerabilities and the need for clear explanations in programming environments. The article concludes with a forward-looking view on the future of programming environments and the role of LLMs in automated programming.Automatic programming has gained popularity due to tools like GitHub Copilot, which rely on Large Language Models (LLMs). However, automatically generated code faces challenges in deployment due to concerns about quality and trust. This article examines the issues of code quality, security, and programmer responsibility in automated programming. It discusses how advances in software engineering, such as program repair and analysis, can enable automatic programming. The article also explores the future of programming environments where programmers may need to switch roles to fully utilize automatic programming. Automated repair of LLM-generated code can help produce higher assurance code with evidence of correctness. The article highlights challenges in automatic programming, including the difficulty of capturing user intent and generating correct code. Recent developments in LLM-based code generation raise trust issues, as the correctness of automatically generated code is uncertain. The article discusses the trust issues for code generated from natural language specifications and the need for systematic program repair to ensure correctness. It also explores the impact of LLMs on non-code artifacts and processes like test generation, code review, and code summarization. The article reviews historical milestones and literature in automatic programming, including code generation, program repair, and software testing. It discusses various approaches to code generation, such as deductive, inductive, and natural language-based methods. It also covers program repair techniques, including search-based, constraint-based, and learning-based approaches. The article highlights the potential of LLMs in intelligent programming, including recent large language code models and their applications in code generation and repair. The article discusses the challenges of program repair, including overfitting and the need for generalization. It presents a concrete approach to program repair using symbolic analysis of test cases. The article also explores the use of LLMs in program repair, including the potential of language models to improve the success rate of code generation and repair. It highlights the importance of ensuring the trustworthiness of code generated by LLMs, including concerns about security vulnerabilities and the need for clear explanations in programming environments. The article concludes with a forward-looking view on the future of programming environments and the role of LLMs in automated programming.
Reach us at info@study.space