EM-Assist: Safe Automated ExtractMethod Refactoring with LLMs

EM-Assist: Safe Automated ExtractMethod Refactoring with LLMs

2024 | Dorin Pomian*, Abhiram Bellur*, Malinda Dilhara Zarina Kurbatova, Egor Bogomolov, Andrey Sokolov, Timofey Bryksin, Danny Dig
EM-Assist is an IntelliJ IDEA plugin that uses Large Language Models (LLMs) to generate, validate, enhance, and rank refactoring suggestions for Extract Method operations. The tool aims to bridge the gap between automated suggestions and real-world developer practices by leveraging LLMs trained on extensive code repositories. EM-Assist filters out invalid and non-useful suggestions, enhancing valid ones using program slicing techniques, and ranks them to provide high-quality options. The plugin then applies the selected suggestion within the IDE, ensuring safe execution. The effectiveness of EM-Assist was evaluated through a quantitative study, comparing it to state-of-the-art tools using a dataset of 1,752 real-world refactorings from open-source projects. EM-Assist achieved a recall rate of 53.4% among its top-5 recommendations, significantly outperforming the previous best tool with a recall rate of 39.4%. A qualitative usability survey of 18 industrial developers also confirmed the tool's positive reception, with 94.4% giving a positive rating. EM-Assist is implemented as an IntelliJ IDEA plugin, supporting Java and Kotlin, and can be extended to support other languages. The tool's architecture allows for easy switching between different LLMs. However, it currently requires sending user code to OpenAI, which can affect response time. Future work includes improving performance and expanding support to more languages and IDEs.EM-Assist is an IntelliJ IDEA plugin that uses Large Language Models (LLMs) to generate, validate, enhance, and rank refactoring suggestions for Extract Method operations. The tool aims to bridge the gap between automated suggestions and real-world developer practices by leveraging LLMs trained on extensive code repositories. EM-Assist filters out invalid and non-useful suggestions, enhancing valid ones using program slicing techniques, and ranks them to provide high-quality options. The plugin then applies the selected suggestion within the IDE, ensuring safe execution. The effectiveness of EM-Assist was evaluated through a quantitative study, comparing it to state-of-the-art tools using a dataset of 1,752 real-world refactorings from open-source projects. EM-Assist achieved a recall rate of 53.4% among its top-5 recommendations, significantly outperforming the previous best tool with a recall rate of 39.4%. A qualitative usability survey of 18 industrial developers also confirmed the tool's positive reception, with 94.4% giving a positive rating. EM-Assist is implemented as an IntelliJ IDEA plugin, supporting Java and Kotlin, and can be extended to support other languages. The tool's architecture allows for easy switching between different LLMs. However, it currently requires sending user code to OpenAI, which can affect response time. Future work includes improving performance and expanding support to more languages and IDEs.
Reach us at info@study.space
Understanding EM-Assist%3A Safe Automated ExtractMethod Refactoring with LLMs