Pith. sign in

REVIEW 2 major objections 7 minor 46 references

Treefix: Enabling Execution with a Tree of Prefixes

T0 review · 2 major / 7 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Treefix enables the execution of incomplete Python snippets by generating and refining LLM-written code prefixes, reporting 84% and 82% line coverage on two benchmark sets.

desk verdict Solid empirical follow-up to LExecutor with one genuine presentation flaw: the headline '7% improvement' on Stack Overflow holds only for the cumulative prefix set, not for the single-best prefix, and the abstract doesn't say so. read the letter →

arxiv 2501.12339 v2 pith:R5MLYV5C submitted 2025-01-21 cs.SE cs.AI

Beatriz Souza , Michael Pradel This is my paper · ORCID
classification cs.SEcs.AI
keywords learning-guidedexecutioncodeprefixeslargelanguagemodelslinecoveragePythonsnippetsundefinedreferencesdynamicanalysiscoverage-guidedrefinement
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Treefix is trying to establish that incomplete Python code snippets — the kind found in documentation, forums, or extracted from larger projects — can be made executable by asking a large language model to generate code prefixes that initialize the missing pieces. The prefixes are not one-shot guesses: Treefix refines them in three passes, first guided by statically detected undefined references, then by runtime error messages, and finally by annotations of lines not yet covered. The refined prefixes form a tree, and Treefix returns the small subset that together maximize the number of lines executed. On two benchmark collections, the paper reports 84% and 82% line coverage, beating the best prior learning-guided execution approach by 25% and 7%. If this holds, it gives dynamic analyses a way to run previously non-runnable snippets without manual setup.

What carries the argument

The central object is the prefix: a syntactically valid block of import and assignment statements that is prepended to a code snippet to initialize missing variables. The carrying mechanism is the three-step refinement loop. Step 1 statically collects undefined variables, attributes, and methods and prompts the LLM for prefixes that define them. Step 2 executes those prefixes, captures the exception type, message, and line number from any failure, and asks the LLM for a fixed prefix. Step 3 annotates the snippet with '# uncovered' comments on lines not yet executed and asks for prefixes that reach those paths. Each refinement is an edge in a tree of prefixes, and a greedy update keeps the minimal set P whose cumulative coverage is maximal alongside the single best prefix.

What would settle it

A concrete test would be to run Treefix on the same 1,462 snippets while replacing the LLM with a weaker or random code generator, or while withholding one of the three feedback signals, and measure how line coverage changes. More directly, count the fraction of generated prefixes that survive the 10-iteration post-processing and execute without error; if that survival rate is low on a fresh sample of snippets, the coverage result is tied to the particular model's code-synthesis ability rather than to the prefix-tree mechanism itself.

Watch

Extended reading notes

Core claim

The central claim is that the task of executing incomplete code is better solved by generating code than by predicting values. Previous learning-guided execution sampled from a fixed catalog of 23 abstract values with one hard-coded concrete representative each, which limited both realism and branch coverage. Treefix instead has an LLM write a prefix — import statements and assignments that construct concrete values, possibly from third-party libraries — and then iteratively repairs and extends prefixes using three feedback signals. The result is a tree of prefixes whose cumulative coverage exceeds what any single execution can reach, because different prefixes can exercise mutually exclusive branches. The paper reports that this cumulative coverage reaches 84% of lines in open-source functions and 82% in Stack Overflow snippets, with a much larger diversity of produced values than the fixed catalog.

Load-bearing premise

The entire multi-step algorithm depends on the underlying LLM being able to produce syntactically valid, executable Python prefixes from prompts that list undefined references and short error or coverage feedback; if the model frequently writes invalid code or the feedback does not steer it to fix errors, the reported coverage gains would disappear.

Editorial extensions

If this is right

  • Learning-guided execution can be reframed as code generation: LLM-written prefixes produce domain-specific strings, complex objects, and imported-library values, far beyond a fixed value catalog.
  • Coverage can be accumulated across multiple executions, so mutually exclusive branches in a snippet no longer force a sacrifice of one path for another.
  • The returned set of prefixes is small — on average one or two — so downstream dynamic analyses can afford to run the snippet multiple times with different initialization.
  • The per-step ablations imply that each feedback signal adds value, but cost grows with later steps; users can trade coverage for cost by stopping after step 1 or using a cheaper model.
  • A large fraction of snippets (69–71%) can be executed to 100% line coverage, suggesting the approach works not only on curated examples but on realistic extracted functions and forum snippets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Extension, not in the paper: the same prefix-tree loop could apply to other dynamically typed languages, but Treefix's dependency-installation and post-processing heuristics are Python-specific, so porting would require re-engineering rather than simple reuse.
  • The coverage-guided third step resembles coverage-guided fuzzing, but mutating initialization code rather than inputs; a natural testable variant would combine both mutations.
  • The paper's cost analysis shows steps 2 and 3 account for most of the monetary cost; a cheaper distilled model or a smarter stop criterion might retain most of the coverage gain for a fraction of the price.
  • The approach's ceiling is set by the LLM's ability to synthesize valid code and by the feasibility of installing dependencies; if code-generation quality improves, the same algorithm should cover more lines without architectural change.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 7 minor

Summary. Treefix proposes an LLM-based approach to learning-guided execution. Given a syntactically valid Python snippet, it statically identifies undefined references, asks an LLM to generate code prefixes that initialize them, refines those prefixes using runtime error messages, and then uses uncovered-line annotations to request further prefixes targeting missing branches. The final output is a set P of prefixes whose cumulative line coverage is maximized, together with the single-best prefix pbest. The paper evaluates Treefix on 1,000 open-source functions and 462 Stack Overflow snippets, comparing against six baselines, and reports 84% and 82% cumulative coverage on the two datasets, with per-step ablations, a diversity analysis, and cost measurements.

Significance. If the evaluation is made fair, Treefix is a useful advance over LExecutor and related partial-execution techniques: it replaces a fixed inventory of 23 abstract values with LLM-generated code prefixes, closes the loop with execution feedback, and explicitly targets multi-path coverage through a set of prefixes. The paper has several genuine strengths: two datasets from prior work, six baselines, per-step ablations, statistical significance testing with the Wilcoxon signed-rank test, and a replication package with logs and prompts. The central idea is clear and the engineering is non-trivial, especially the dependency-resolution and prefix-post-processing pipeline. However, the headline improvement claim currently rests on an unequal comparison between Treefix's cumulative coverage over a selected set of prefixes and single-execution baselines, and the main result also mixes model choice into the method comparison. These issues are fixable but are load-bearing for the paper's central claim of superiority over the state of the art.

major comments (2)
  1. [Section IV-B, Table II; Abstract and Section I] The headline comparison is set-based versus single-execution. Treefix reports cumulative coverage of the selected prefix set P (0.84 and 0.82), while its own single-best prefix pbest achieves only 0.76 on open-source functions and 0.72 on Stack Overflow snippets. On Stack Overflow, SelfPiCo achieves 0.75, which is higher than pbest, so the advertised 7% improvement over SelfPiCo is entirely an artifact of taking the union of multiple prefixes. The full execution rate column in Table II suffers from the same ambiguity. Please either give the baselines an equal execution budget, e.g., allow LExecutor to make multiple value predictions and take the union of covered lines, allow SelfPiCo to produce multiple candidate executions, or otherwise define a fair multi-execution protocol; or report the comparison as 'Treefix's prefix set versus single-execution baselines' and remove the unqualified 'improves over the best baseline' claim from the abstract and RQ1 conclusions.
  2. [Section IV-A2 and Table II] The SelfPiCo baseline is run with GPT-3.5 while Treefix's main results use GPT-4o. The authors state that SelfPiCo's fine-tuned Code Llama model is not available and that GPT-3.5 reportedly achieves similar performance, but no supporting evidence is given. This conflates the proposed method with the choice of a much stronger LLM, which is a particular concern because Treefix's advantage over SelfPiCo on Stack Overflow is small even in the set-based comparison. Please rerun SelfPiCo with GPT-4o and GPT-4o-mini under a comparable prompting protocol, or clearly frame the result as method-plus-model and provide an ablation that isolates the effect of the model choice.
minor comments (7)
  1. [Abstract and Section I] The numbers '25% and 7% more coverage' are absolute percentage-point differences (0.84-0.59 and 0.82-0.75), not relative improvements; the wording 'relative to' is ambiguous and should be changed to 'by 25 and 7 percentage points.'
  2. [Algorithm 1 and Section III-C] The UPDATE PREFIXES helper and the procedure that selects the returned set P are never described. Please specify the selection rule and clarify in what sense, if any, P is minimal; the current text calls P 'minimal' in Section I but provides no optimality argument.
  3. [Section IV-A2] The sentence that SelfPiCo with GPT-3.5 'report[s] similar performance' to the Code Llama version is not substantiated. Please either provide the comparison or soften the claim, since this is the basis for the main baseline comparison.
  4. [Table II and Section IV-B] The 'Full Execution Rate' column for Treefix should be labeled as cumulative over the prefix set P, to avoid the same set-versus-single ambiguity as the coverage column.
  5. [Section VI] Two paragraphs in the related work contain '[?]' placeholders for automated program repair and learning/LLM citations; please fill in the missing references before publication.
  6. [Section III-E] The text says the conversation history from step 1 is kept as part of the step-2 prompt, but Figure 5 does not show this history. Please clarify exactly what context is included in the prompt.
  7. [Section IV-F] The cost figures depend on OpenAI pricing as of July 2024. Since LLM pricing and model versions change over time, please state the date explicitly and note in the text that the absolute dollar amounts are not stable.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Treefix's coverage claims are measured empirically against external benchmarks and do not reduce to its inputs.

full rationale

Treefix makes no theoretical derivation that could collapse into its own inputs. The central claim is an empirical one: that an LLM-based, feedback-driven prefix generation algorithm achieves higher line coverage than prior learning-guided execution approaches. Coverage is measured by actually executing instrumented snippets and recording executed lines, as described in Section III-G3, against two external datasets of open-source functions and Stack Overflow snippets. The baselines, including LExecutor, are applied and measured independently in the same evaluation harness. The use of datasets from the authors' prior LExecutor paper is not circular: the snippets themselves are external code samples, and the baselines are run fresh on them. The prompts and LLM outputs are not fitted to the target coverage numbers; the paper explicitly reports costs, failures, and cases where Treefix underperforms LExecutor, indicating the results are not forced. The only self-citation of note is the reuse of the LExecutor benchmark and coverage metric, but neither is a load-bearing assumption that predetermines the outcome. The comparison asymmetry between cumulative prefix-set coverage and single-execution baselines is a potential threat to the fairness of the evaluation, but it is a correctness or methodology concern, not a circularity concern, because the reported numbers are still measured rather than derived from definitions or fitted parameters. Under the stated hard rules, no step exhibits self-definition, renamed fits, or a self-citation chain that makes the conclusion equivalent to its inputs.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on hand-chosen hyperparameters (n, k, timeouts) and on standard assumptions about Python execution, static analysis correctness, dependency detection with pipreqs, and the fidelity of the line-coverage instrumentation. No new physical or mathematical entities are introduced.

free parameters (4)
  • n (number of prefixes per LLM query) = 10
    Hand-chosen hyperparameter in Algorithm 1; controls the diversity of generated prefixes per prompt.
  • k (maximum coverage-guidance iterations) = 10
    Hand-chosen budget for Step 3 in Algorithm 1; limits how many rounds of coverage-guided prefix generation are performed.
  • prefix execution timeout = 30 seconds
    Prefixes taking longer than 30 seconds are discarded (Section III-G2).
  • post-processing retry limit = 10
    Each prefix is executed up to 10 times, removing lines that raise errors (Section III-G2).
assumptions (4)
  • domain assumption CPython 3.8 execution semantics
    All snippets and prefixes are executed under Python 3.8; results may differ on other versions.
  • domain assumption Static analysis GET UNDEFINED REFS correctly identifies all undefined references
    Section III-D relies on AST-based scope analysis to find missing variables, attributes, and methods; if it misses any, the prompt will not ask the LLM to define them.
  • domain assumption pipreqs correctly identifies imported third-party dependencies
    Section III-G1 uses pipreqs to find and install dependencies; failures or misidentification prevent execution.
  • domain assumption The _l_ instrumentation accurately measures line coverage
    Section III-G3 adds a call after each line; a line is counted only if it completes without crashing, which is a specific coverage definition.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Treefix: Enabling Execution with a Tree of Prefixes." pith.science (2026). https://pith.science/paper/R5MLYV5C

@misc{pith2026250112339,
  author       = {Pith},
  title        = {Pith review of: Treefix: Enabling Execution with a Tree of Prefixes},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/R5MLYV5C}},
  note         = {Machine review of arXiv:2501.12339}
}
read the original abstract

The ability to execute code is a prerequisite for various dynamic program analyses. Learning-guided execution has been proposed as an approach to enable the execution of arbitrary code snippets by letting a neural model predict likely values for any missing variables. Although state-of-the-art learning-guided execution approaches, such as LExecutor, can enable the execution of a relative high amount of code, they are limited to predicting a restricted set of possible values and do not use any feedback from previous executions to execute even more code. This paper presents Treefix, a novel learning-guided execution approach that leverages LLMs to iteratively create code prefixes that enable the execution of a given code snippet. The approach addresses the problem in a multi-step fashion, where each step uses feedback about the code snippet and its execution to instruct an LLM to improve a previously generated prefix. This process iteratively creates a tree of prefixes, a subset of which is returned to the user as prefixes that maximize the number of executed lines in the code snippet. In our experiments with two datasets of Python code snippets, Treefix achieves 25% and 7% more coverage relative to the current state of the art in learning-guided execution, covering a total of 84% and 82% of all lines in the code snippets.

Figures

Figures reproduced from arXiv: 2501.12339 by the authors.

Figure 1
Figure 1. Example of code to execute and predicted values. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Tree of prefixes generated in different steps of Treefix. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Prompt for undefinedness guidance. Respond strictly with JSON. The JSON should be compatible with the TypeScript type “Response”: ‘‘‘ts interface Response { // Python import statements, one string per import imports: string[]; // Python code to initialize undefined variables, one string per variable initialization: string[]; } ‘‘‘ [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Response specification. with its base object, e.g., d.year, is added to a list, which is returned by GETUNDEFINEDREFS. Based on the statically determined set of undefined refer￾ences, Treefix generates a prompt aimed at predicting code to define those references (GENPR…
Figure 5
Figure 5. Figure 5: Prompt for error guidance. s, and then measures the line coverage achieved by it. Finally, Treefix updates the set of prefixes P and the single-best prefix pbest based on the coverage achieved by the prefixes in P1. E. Step 2: Error Guidance The values predicted in ste…
Figure 7
Figure 7. Figure 7: Prefixes explored by Treefix. the lines with pbest and even 84% with P, which is higher than all considered baselines (statistically significant) and an improvement of 25% over SelfPiCo, i.e., the currently best available approach. Comparing the two LLMs shows that usi…
Figure 9
Figure 9. Figure 9: Example of problematic code. 1) Reasons for Higher Coverage: Across all 1,462 analyzed code snippets, Treefix achieves higher coverage than LExecu￾tor in 707 code snippets: 561 from the open-source functions dataset and 146 from the Stack Overflow dataset. We randomly …
Figure 8
Figure 8. Figure 8: Example of adequate imports and usage of dependen [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 23 canonical work pages

  1. [2]

    Selfpico: Self- guided partial code execution with llms,

    Z. Xue, Z. Gao, S. Wang, X. Hu, X. Xia, and S. Li, “Selfpico: Self- guided partial code execution with llms,” in ISSTA, 2024

    work page 2024

  2. [3]

    Feedback-directed partial exe- cution,

    I. Hayet, A. Scott, and M. d’Amorim, “Feedback-directed partial exe- cution,” in ISSTA, 2024

    work page 2024

  3. [4]

    Changeguard: Validating code changes via pairwise learning-guided execution,

    L. Gr ¨oninger, B. Souza, and M. Pradel, “Changeguard: Validating code changes via pairwise learning-guided execution,” 2024. [Online]. Available: https://arxiv.org/abs/2410.16092

    arXiv 2024

  4. [5]

    Evaluating large language models trained on code,

    M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. de Oliveira Pinto, J. Kaplan, H. Edwards, Y . Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ryder, M. Pavlov, A. Power, L. Kaiser, M. Bavarian, C. Winter, P. Tillet, F. P. Such, D. Cummings, M. Plappert, F. Chantzis, E. Barnes, A. Her...

    arXiv 2021

  5. [6]

    Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation,

    J. Liu, C. S. Xia, Y . Wang, and L. Zhang, “Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation,” in Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023 , A. Oh, T. Nau...

    work page 2023

  6. [7]

    Coeditor: Leveraging contex- tual changes for multi-round code auto-editing,

    J. Wei, G. Durrett, and I. Dillig, “Coeditor: Leveraging contex- tual changes for multi-round code auto-editing,” arXiv preprint arXiv:2305.18584, 2023

    arXiv 2023

  7. [8]

    Grace: Language models meet code edits,

    P. Gupta, A. Khare, Y . Bajpai, S. Chakraborty, S. Gulwani, A. Kanade, A. Radhakrishna, G. Soares, and A. Tiwari, “Grace: Language models meet code edits,” in Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2023, San Francisco, CA, USA, December 3-9, 2023 , S. Ch...

    arXiv 2023

  8. [9]

    Unprecedented code change automation: The fusion of llms and transformation by example,

    M. Dilhara, A. Bellur, T. Bryksin, and D. Dig, “Unprecedented code change automation: The fusion of llms and transformation by example,” in FSE, 2024. [Online]. Available: https://doi.org/10.48550/arXiv.2402. 07138

    doi:10.48550/arxiv.2402 2024

Show all 46 references
  1. [10]

    Type4py: practical deep similarity learning-based type inference for python,

    A. M. Mir, E. Lato ˇskinas, S. Proksch, and G. Gousios, “Type4py: practical deep similarity learning-based type inference for python,” in Proceedings of the 44th International Conference on Software Engineering, ser. ICSE ’22. New York, NY , USA: Association for Computing Mach...

    2022

  2. [11]

    Automated unit test generation for python,

    S. Lukasczyk, F. Kroiß, and G. Fraser, “Automated unit test generation for python,” in SSBSE, 2020, pp. 9–24. [Online]. Available: https://doi.org/10.1007/978-3-030-59762-7 2

    2020 doi

  3. [12]

    American fuzzy lop (afl),

    M. Zalewski, “American fuzzy lop (afl),” https://lcamtuf.coredump.cx/afl/, 2013. [Online]. Available: https: //lcamtuf.coredump.cx/afl/

    2013

  4. [13]

    Coverage-based greybox fuzzing as markov chain,

    M. B ¨ohme, V . Pham, and A. Roychoudhury, “Coverage-based greybox fuzzing as markov chain,” IEEE Trans. Software Eng. , vol. 45, no. 5, pp. 489–506, 2019. [Online]. Available: https: //doi.org/10.1109/TSE.2017.2785841

    2019

  5. [14]

    Fuzz4all: Universal fuzzing with large language models,

    C. S. Xia, M. Paltenghi, J. L. Tian, M. Pradel, and L. Zhang, “Fuzz4all: Universal fuzzing with large language models,” in Proceedings of the 46th IEEE/ACM International Conference on Software Engineering, ICSE 2024, Lisbon, Portugal, April 14- 20, 2024 . ACM, 2024, pp. 126:1–...

    2024

  6. [15]

    Codamosa: Escaping coverage plateaus in test generation with pre-trained large language models,

    C. Lemieux, J. P. Inala, S. K. Lahiri, and S. Sen, “Codamosa: Escaping coverage plateaus in test generation with pre-trained large language models,” in Proceedings of the 45th International Conference on Software Engineering, ser. ICSE ’23. IEEE Press, 2023, p. 919–931. [Onlin...

    2023

  7. [16]

    Coverup: Coverage-guided llm-based test generation,

    J. A. Pizzorno and E. D. Berger, “Coverup: Coverage-guided llm-based test generation,” 2024. [Online]. Available: https://arxiv.org/abs/2403. 16218

    2024

  8. [17]

    Micro execution,

    P. Godefroid, “Micro execution,” in 36th International Conference on Software Engineering, ICSE ’14, Hyderabad, India - May 31 - June 07, 2014, 2014, pp. 539–549

    2014

  9. [18]

    Under-constrained symbolic execution: Correctness checking for real code,

    D. A. Ramos and D. R. Engler, “Under-constrained symbolic execution: Correctness checking for real code,” in 24th USENIX Security Symposium, USENIX Security 15, Washington, D.C., USA, August 12-14, 2015 , J. Jung and T. Holz, Eds. USENIX Association, 2015, pp. 49–64. [Online]....

    2015

  10. [19]

    J-force: Forced execution on javascript,

    K. Kim, I. L. Kim, C. H. Kim, Y . Kwon, Y . Zheng, X. Zhang, and D. Xu, “J-force: Forced execution on javascript,” in Proceedings of the 26th International Conference on World Wide Web, WWW 2017, Perth, Australia, April 3-7, 2017 , R. Barrett, R. Cummings, E. Agichtein, and E....

    2017

  11. [20]

    KLEE: Unassisted and automatic generation of high-coverage tests for complex systems pro- grams,

    C. Cadar, D. Dunbar, and D. R. Engler, “KLEE: Unassisted and automatic generation of high-coverage tests for complex systems pro- grams,” in Symposium on Operating Systems Design and Implementation (OSDI). USENIX, 2008, pp. 209–224

    2008

  12. [21]

    DART: directed automated random testing,

    P. Godefroid, N. Klarlund, and K. Sen, “DART: directed automated random testing,” in Conference on Programming Language Design and Implementation (PLDI). ACM, 2005, pp. 213–223

    2005

  13. [22]

    CUTE: a concolic unit testing engine for C,

    K. Sen, D. Marinov, and G. Agha, “CUTE: a concolic unit testing engine for C,” in European Software Engineering Conference and Interna- tional Symposium on Foundations of Software Engineering (ESEC/FSE). ACM, 2005, pp. 263–272

    2005

  14. [23]

    Evosuite: automatic test suite generation for object-oriented software,

    G. Fraser and A. Arcuri, “Evosuite: automatic test suite generation for object-oriented software,” in SIGSOFT/FSE’11 19th ACM SIGSOFT Symposium on the Foundations of Software Engineering (FSE-19) and ESEC’11: 13th European Software Engineering Conference (ESEC-13), Szeged, Hun...

    2011

  15. [24]

    Codamosa: Escaping coverage plateaus in test generation with pre-trained large language models,

    C. Lemieux, J. P. Inala, S. K. Lahiri, and S. Sen, “Codamosa: Escaping coverage plateaus in test generation with pre-trained large language models,” in 45th International Conference on Software Engineering, ser. ICSE, 2023

    2023

  16. [25]

    Coverup: Coverage-guided llm-based test generation,

    J. A. Pizzorno and E. D. Berger, “Coverup: Coverage-guided llm-based test generation,” 2024

    2024

  17. [26]

    Large language models are zero-shot fuzzers: Fuzzing deep-learning libraries via large language models,

    Y . Deng, C. S. Xia, H. Peng, C. Yang, and L. Zhang, “Large language models are zero-shot fuzzers: Fuzzing deep-learning libraries via large language models,” in Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA , R. Just and G...

    2023

  18. [27]

    Conversational automated program repair,

    C. S. Xia and L. Zhang, “Conversational automated program repair,” CoRR, vol. abs/2301.13246, 2023. [Online]. Available: https://doi.org/10.48550/arXiv.2301.13246

    2023 doi arXiv

  19. [28]

    Less training, more repairing please: revisiting automated program repair via zero-shot learning

    ——, “Less training, more repairing please: revisiting automated program repair via zero-shot learning.” New York, NY , USA: Association for Computing Machinery, 2022. [Online]. Available: https://doi.org/10.1145/3540250.3549101

    2022

  20. [29]

    RepairAgent: An autonomous, LLM-based agent for program repair,

    I. Bouzenia, P. Devanbu, and M. Pradel, “RepairAgent: An autonomous, LLM-based agent for program repair,” Preprint, 2024

    2024

  21. [30]

    Neural software analysis,

    M. Pradel and S. Chandra, “Neural software analysis,” Commun. ACM, vol. 65, no. 1, pp. 86–96, 2022. [Online]. Available: https://doi.org/10.1145/3460348

    2022 doi

  22. [31]

    Deep learning type inference,

    V . J. Hellendoorn, C. Bird, E. T. Barr, and M. Allamanis, “Deep learning type inference,” in Proceedings of the 2018 ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/SIGSOFT , 2018, pp. 152–162. [Onli...

    2018

  23. [32]

    NL2Type: Inferring JavaScript function types from natural language information,

    R. S. Malik, J. Patra, and M. Pradel, “NL2Type: Inferring JavaScript function types from natural language information,” in Proceedings of the 41st International Conference on Software Engineering, ICSE 2019, Montreal, QC, Canada, May 25-31, 2019 , 2019, pp. 304–315. [Online]. ...

    2019

  24. [33]

    Typewriter: Neural type prediction with search-based validation,

    M. Pradel, G. Gousios, J. Liu, and S. Chandra, “Typewriter: Neural type prediction with search-based validation,” in ESEC/FSE ’20: 28th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, Virtual Event, USA, November 8-1...

    2020

  25. [34]

    Typilus: neural type hints,

    M. Allamanis, E. T. Barr, S. Ducousso, and Z. Gao, “Typilus: neural type hints,” in Proceedings of the 41st ACM SIGPLAN International Conference on Programming Language Design and Implementation, PLDI , 2020, pp. 91–105. [Online]. Available: https: //doi.org/10.1145/3385412.3385997

    2020

  26. [35]

    An empirical evaluation of using large language models for automated unit test generation,

    M. Sch ¨afer, S. Nadi, A. Eghbali, and F. Tip, “An empirical evaluation of using large language models for automated unit test generation,” IEEE Transactions on Software Engineering , 2023

    2023

  27. [36]

    Code-aware prompting: A study of coverage guided test generation in regression setting using llm,

    G. Ryan, S. Jain, M. Shang, S. Wang, X. Ma, M. K. Ramanathan, and B. Ray, “Code-aware prompting: A study of coverage guided test generation in regression setting using llm,” in FSE, 2024

    2024

  28. [37]

    Learning deep semantics for test completion,

    P. Nie, R. Banerjee, J. J. Li, R. J. Mooney, and M. Gligoric, “Learning deep semantics for test completion,” in ICSE, 2023

    2023

  29. [38]

    De-hallucinator: Iterative grounding for llm-based code completion,

    A. Eghbali and M. Pradel, “De-hallucinator: Iterative grounding for llm-based code completion,” CoRR, vol. abs/2401.01701, 2024. [Online]. Available: https://doi.org/10.48550/arXiv.2401.01701

    2024 doi arXiv

  30. [39]

    SequenceR: Sequence-to-sequence learning for end-to-end program repair,

    Z. Chen, S. Kommrusch, M. Tufano, L. Pouchet, D. Poshyvanyk, and M. Monperrus, “SequenceR: Sequence-to-sequence learning for end-to-end program repair,” IEEE Trans. Software Eng. , vol. 47, no. 9, pp. 1943–1959, 2021. [Online]. Available: https://doi.org/10.1109/TSE. 2019.2940179

    1943

  31. [40]

    Iter: Iterative neural repair for multi-location patches,

    H. Ye and M. Monperrus, “Iter: Iterative neural repair for multi-location patches,” in ICSE, 2024

    2024

  32. [41]

    Repairllama: Efficient represen- tations and fine-tuned adapters for program repair,

    A. Silva, S. Fang, and M. Monperrus, “Repairllama: Efficient represen- tations and fine-tuned adapters for program repair,” 2024

    2024

  33. [42]

    A deep dive into large language models for automated bug localization and repair,

    S. B. Hossain, N. Jiang, Q. Zhou, X. Li, W.-H. Chiang, Y . Lyu, H. Nguyen, and O. Tripp, “A deep dive into large language models for automated bug localization and repair,” in FSE, 2024

    2024

  34. [43]

    Swe-agent: Agent-computer interfaces enable automated software engineering,

    J. Yang, C. E. Jimenez, K. Lieret, S. Yao, A. Wettig, K. Narasimhan, and O. Press, “Swe-agent: Agent-computer interfaces enable automated software engineering,” 2024

    2024

  35. [44]

    Autocoderover: Autonomous program improvement,

    Y . Zhang, H. Ruan, Z. Fan, and A. Roychoudhury, “Autocoderover: Autonomous program improvement,” 2024

    2024

  36. [45]

    Magis: Llm-based multi-agent framework for github issue resolution,

    W. Tao, Y . Zhou, W. Zhang, and Y . Cheng, “Magis: Llm-based multi-agent framework for github issue resolution,” arXiv preprint arXiv:2403.17927, 2024

    2024 arXiv

  37. [46]

    Our replication package

    “Our replication package.” 2024. [Online]. Available: https://anonymous. 4open.science/r/Treefix-E4D8/

    2024

  38. [2023]

    Available: http://papers.nips.cc/paper files/paper/2023/ hash/43e9d647ccd3e4b7b5baab53f0368686-Abstract-Conference.html

    [Online]. Available: http://papers.nips.cc/paper files/paper/2023/ hash/43e9d647ccd3e4b7b5baab53f0368686-Abstract-Conference.html

    2023

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.