CigaR: Cost-efficient Program Repair with LLMs

CigaR: Cost-efficient Program Repair with LLMs

18 Apr 2024 | Dávid Hidvégi*, Khashayar Etemadi*, Sofia Bobadilla, Martin Monperrus
CIGAR (Cost-efficient Program Repair with LLMs) is a novel LLM-based program repair system designed to minimize computational costs, particularly token usage. The system aims to reduce the cost of automated program repair (APR) by optimizing the prompts and configurations used to interact with large language models (LLMs). CIGAR operates in two main steps: generating a first plausible patch and multiplying plausible patches. It optimizes the prompts to maximize information while minimizing token usage. Experiments on the DEFECTS4J and HUMANEVAL-JAVA datasets show that CIGAR reduces token costs by 73%, with an average of 127k tokens per bug, compared to 467k tokens per bug for the baseline. CIGAR also outperforms the state-of-the-art APR tools in terms of the number of bugs fixed, demonstrating its effectiveness and efficiency. The system's success is attributed to its iterative prompting techniques, reboot strategy, and patch multiplication, which enable it to explore the repair space more effectively and efficiently.CIGAR (Cost-efficient Program Repair with LLMs) is a novel LLM-based program repair system designed to minimize computational costs, particularly token usage. The system aims to reduce the cost of automated program repair (APR) by optimizing the prompts and configurations used to interact with large language models (LLMs). CIGAR operates in two main steps: generating a first plausible patch and multiplying plausible patches. It optimizes the prompts to maximize information while minimizing token usage. Experiments on the DEFECTS4J and HUMANEVAL-JAVA datasets show that CIGAR reduces token costs by 73%, with an average of 127k tokens per bug, compared to 467k tokens per bug for the baseline. CIGAR also outperforms the state-of-the-art APR tools in terms of the number of bugs fixed, demonstrating its effectiveness and efficiency. The system's success is attributed to its iterative prompting techniques, reboot strategy, and patch multiplication, which enable it to explore the repair space more effectively and efficiently.
Reach us at info@study.space
[slides and audio] CigaR%3A Cost-efficient Program Repair with LLMs