CigaR: Cost-efficient Program Repair with LLMs

CigaR: Cost-efficient Program Repair with LLMs

18 Apr 2024 | Dávid Hidvégi*, Khashayar Etemadi*, Sofia Bobadilla, Martin Monperrus
CIGAR is a novel LLM-based program repair tool that focuses on minimizing token cost. It uses three carefully designed prompts: an initiation prompt, an improvement prompt, and a multiplication prompt. The initiation prompt generates a first plausible patch, the improvement prompt refines partial patches, and the multiplication prompt generates more diverse plausible patches. CIGAR reduces token cost by 73% compared to the baseline, spending 127k tokens per bug versus 467k tokens. On bugs fixed by both CIGAR and the baseline, CIGAR saves 96% of the token cost. CIGAR is evaluated on the DEFECTS4J and HUMANEVAL-JAVA datasets, fixing 171/429 (39.8%) of the bugs. It outperforms state-of-the-art APR tools in terms of both effectiveness and efficiency. CIGAR's cost-effective approach enables efficient exploration of the patch search space, leading to a significant reduction in token cost. The tool is publicly available for future research.CIGAR is a novel LLM-based program repair tool that focuses on minimizing token cost. It uses three carefully designed prompts: an initiation prompt, an improvement prompt, and a multiplication prompt. The initiation prompt generates a first plausible patch, the improvement prompt refines partial patches, and the multiplication prompt generates more diverse plausible patches. CIGAR reduces token cost by 73% compared to the baseline, spending 127k tokens per bug versus 467k tokens. On bugs fixed by both CIGAR and the baseline, CIGAR saves 96% of the token cost. CIGAR is evaluated on the DEFECTS4J and HUMANEVAL-JAVA datasets, fixing 171/429 (39.8%) of the bugs. It outperforms state-of-the-art APR tools in terms of both effectiveness and efficiency. CIGAR's cost-effective approach enables efficient exploration of the patch search space, leading to a significant reduction in token cost. The tool is publicly available for future research.
Reach us at info@study.space
[slides and audio] CigaR%3A Cost-efficient Program Repair with LLMs