REVIEW 10 cited by
SpecRover: Code Intent Extraction via LLMs
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
abstract
Autonomous program improvement typically involves automatically producing bug fixes and feature additions. Such program improvement can be accomplished by a combination of large language model (LLM) and program analysis capabilities, in the form of an LLM agent. Since program repair or program improvement typically requires a specification of intended behavior - specification inference can be useful for producing high quality program patches. In this work, we examine efficient and low-cost workflows for iterative specification inference within an LLM agent. Given a GitHub issue to be resolved in a software project, our goal is to conduct iterative code search accompanied by specification inference - thereby inferring intent from both the project structure and behavior. The intent thus captured is examined by a reviewer agent with the goal of vetting the patches as well as providing a measure of confidence in the vetted patches. Our approach SpecRover (AutoCodeRover-v2) is built on the open-source LLM agent AutoCodeRover. In an evaluation on the full SWE-Bench consisting of 2294 GitHub issues, it shows more than 50% improvement in efficacy over AutoCodeRover. Compared to the open-source agents available, our work shows modest cost ($0.65 per issue) in resolving an average GitHub issue in SWE-Bench lite. The production of explanation by SpecRover allows for a better "signal" to be given to the developer, on when the suggested patches can be accepted with confidence. SpecRover also seeks to demonstrate the continued importance of specification inference in automated program repair, even as program repair technologies enter the LLM era.
Forward citations
Cited by 10 Pith papers
-
NoCode-bench: A Benchmark for Evaluating Natural Language-Driven Feature Addition
NoCode-bench provides 634 documentation-change-to-code feature addition tasks, and the best evaluated LLM succeeds on only 28.07% of the human-verified subset.
-
SVRepair: Structured Visual Reasoning for Automated Program Repair
A multimodal program-repair system that converts bug screenshots into semantic scene graphs and iteratively crops to bug regions, reporting 36.47% on SWE-Bench M, 38.02% on MMCode, and 95.12% on CodeVision.
-
Git Context Controller: Manage the Context of LLM-based Agents like Git
GCC, a Git-inspired context management layer, is reported to lift a Claude-based agent to 48.00% on SWE-Bench-Lite and to enable a self-replicating CLI that resolves 40.7% of tasks versus 11.7% without it.
-
Agentic Program Repair from Test Failures at Scale: A Neuro-symbolic approach with static analysis and test execution feedback
Meta's Engineering Agent solved 42.3% of curated internal test-failure tasks offline and landed 25.5% of generated fixes in production over three months.
-
From Empirical Evaluation to Context-Aware Enhancement: Repairing Regression Errors with LLMs
Adding bug-inducing commit context to LLM prompts raises correct regression repairs from 9 to 16 of 99 Java bugs (1.8x).
-
Detection Time Distribution Predicted Using Absorbing Boundary Conditions and Imaginary Potentials
Three inequivalent quantum detection-time proposals yield waveguide arrival distributions showing partial reflection, spin-orientation independence (for tested parameters), and width dependence when spin couples to th...
-
Beyond Accuracy: Behavioral Dynamics of Agentic Multi-Hunk Repair
On 372 multi-hunk bugs, coding agents repair between 26% and 93% of defects, accuracy falls as edits become more divergent and dispersed, and failed repairs consume up to 343% more tokens.
-
ViTAD: Timing Violation-Aware Debugging of RTL Code using Large Language Models
ViTAD uses a graph of signal timing dependencies plus an LLM to diagnose and repair RTL timing violations, reportedly improving success rate by 19.30 points over a basic LLM baseline.
-
Self-Evolving Coding Agents
A systematic survey that defines self-evolving coding agents, organizes systems into a five-category object-centered taxonomy, and analyzes when and on what evidence these agents evolve.
-
Rethinking Autonomy: Preventing Failures in AI-Driven Software Engineering
A survey-plus-benchmark argues that current code LLMs fail safety thresholds and need stronger governance, but the supporting experiment is incomplete.
Discussion (0). Sign in to comment.