The reviewed record of science sign in
Pith

arxiv: 2208.05950 · v2 · pith:6CYVDO3F · submitted 2022-08-11 · cs.SE · cs.LG· cs.PL

Interactive Code Generation via Test-Driven User-Intent Formalization

Shuvendu K. Lahiri , Sarah Fakhoury , Aaditya Naik , Georgios Sakkas , Saikat Chakraborty , Madanlal Musuvathi , Piali Choudhury , Curtis von Veh
show 3 more authors
Jeevana Priya Inala Chenglong Wang Jianfeng Gao
This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel pith:6CYVDO3Frecord.jsonopen to challenge →

classification cs.SE cs.LGcs.PL
keywords codegenerationintentlanguagenaturalsuggestionsuseralgorithm
0
0 comments X
read the original abstract

Large language models (LLMs) have shown great potential in automating significant aspects of coding by producing natural code from informal natural language (NL) intent. However, when interacting with LLMs, users have no guarantees that the code suggestions produced correctly satisfy the intent they provided. In fact, it is hard to define a notion of correctness since natural language can be ambiguous and lacks a formal semantics. In this paper, we propose the workflow of {\it interactive test-driven code generation}, which leverages lightweight user feedback to (a) formalize the user intent using generated tests that can be useful for debugging, and (b) produce an improved set of code suggestions by pruning and ranking candidate code suggestions. We describe a language-agnostic abstract algorithm and a concrete implementation TiCoder. We perform an automated evaluation of TiCoder on the \emph{MBPP} and \emph{HumanEval} code generation benchmarks. Our results are promising with using the OpenAI Codex LLM: our best algorithm improves the \passk{1} code generation accuracy (in absolute percentages) between $22.49\%$ to $37.71\%$ for MBPP and between $24.79\%$ to $53.98\%$ for HumanEval using between 1 to 5 simulated user queries.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. From Exploration to Specification: LLM-Based Property Generation for Mobile App Testing

    cs.SE 2026-04 unverdicted novelty 7.0

    PropGen automates property generation for Android app testing via LLM synthesis from guided exploration and feedback refinement, yielding 912 valid properties and 25 previously unknown bugs across 12 apps.

  2. DuET: Dual Execution for Test Output Prediction with Generated Code and Pseudocode

    cs.SE 2026-04 unverdicted novelty 7.0

    DuET uses dual execution of generated code and pseudocode with majority voting to achieve state-of-the-art test output prediction, boosting Pass@1 by 13.6 percentage points on LiveCodeBench.

  3. HiL-Bench (Human-in-Loop Benchmark): Do Agents Know When to Ask for Help?

    cs.AI 2026-04 unverdicted novelty 7.0

    HiL-Bench shows frontier AI agents fail to ask for help on incomplete tasks, recovering only a fraction of full-information performance, but RL training on Ask-F1 reward improves judgment and transfers across domains.

  4. Ambig-IaC: Multi-level Disambiguation for Interactive Cloud Infrastructure-as-Code Synthesis

    cs.SE 2026-04 unverdicted novelty 7.0

    Ambig-IaC detects structural disagreements in LLM-generated IaC candidates across three hierarchical axes to produce clarification questions, improving structure and attribute accuracy by 18.4% and 25.4% on a new 300-...

  5. Enhancing Program Repair with Specification Guidance and Intermediate Behavioral Signals

    cs.SE 2026-04 unverdicted novelty 6.0

    SpecTune improves LLM-based automated program repair by deriving localized postconditions at execution checkpoints and using alpha and beta signals to produce precise fault-localization and patch-generation guidance.