REVIEW 7 cited by
Aligning the Objective of LLM-based Program Repair
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large language models (LLMs) have achieved decent results on automated program repair (APR). However, the next token prediction training objective of decoder-only LLMs (e.g., GPT-4) is misaligned with the masked span prediction objective of current infilling-style methods, which impedes LLMs from fully leveraging pre-trained knowledge for program repair. In addition, while some LLMs can locate and repair bugs in certain functions using the related artifacts (e.g., test cases), existing methods still depend on statement-level fault localization methods to provide a list of buggy hunks for repair. This restriction hinders LLMs from exploring potential patches beyond the given locations. In this paper, we investigate a new approach to adapt LLMs to program repair. Our core insight is that LLM's APR capability can be greatly improved by simply aligning the output to their training objective and allowing them to refine the whole program without first identifying faulty statements. Based on this insight, we designed D4C, a straightforward prompting framework for APR. D4C can repair 180 bugs correctly in Defects4J, with each patch being sampled only 10 times. This surpasses the SOTA APR methods with perfect fault localization by 10% and reduces the patch sampling number by 90%. Our findings reveal that (1) objective alignment is crucial for fully exploiting LLM's pre-trained capability, and (2) replacing the traditional localize-buggy-hunks-then-repair workflow with direct debugging is more effective for LLM-based APR methods. Thus, we believe this paper introduces a new mindset for harnessing LLMs in APR.
Forward citations
Cited by 7 Pith papers
-
PAIChecker: Uncovering and Checking PR-Issue Misalignment in SWE-Bench-Like Benchmarks
PAIChecker detects PR-issue misalignment in SWE-bench-like benchmarks, which the authors find affects 13.6% of SWE-bench Verified instances and shifts 64.1% of leaderboard rankings.
-
AdaptAgent: A Multi-agent, Domain-Guided Reasoning Framework for Code Adaptation
A multi-agent LLM pipeline that plans code adaptations using summarized intent, domain checklists, and sibling-method context outperforms single-shot prompting and repair baselines on Java adaptation examples.
-
HLSDebugger: Identification and Correction of Logic Bugs in HLS Code with LLM Solutions
HLSDebugger fixes logic bugs in HLS code with 37.6% accuracy, outperforming GPT-4 by over 3x, using an encoder-decoder model trained on a new 300K-sample dataset.
-
Seeing is Fixing: Cross-Modal Reasoning with Multimodal LLMs for Visual Software Issue Fixing
GUIRepair, a cross-modal LLM pipeline that converts issue screenshots into reproduction code and rendered patch screenshots into validation feedback, resolves 157/517 SWE-bench M instances with GPT-4o and 175 with o4-mini.
-
From Empirical Evaluation to Context-Aware Enhancement: Repairing Regression Errors with LLMs
Adding bug-inducing commit context to LLM prompts raises correct regression repairs from 9 to 16 of 99 Java bugs (1.8x).
-
Which Prompting Technique Should I Use? An Empirical Investigation of Prompting Techniques for Software Engineering Tasks
Across ten software engineering tasks and four LLMs, no prompting technique wins consistently; ES-KNN is best on many tasks, some techniques underperform the baseline, and USC is best for code QA and code generation.
-
Automated Repair of C Programs Using Large Language Models
An agent that combines spectrum-based fault localization, test feedback, and chain-of-thought prompting repairs 44.93% of 3,902 Codeflaws C bugs, a 3.61-point gain over GPT-4 with CoT.
Discussion (0). Continue with ORCID to comment.