Pith. sign in

REVIEW 3 major objections 5 minor 55 references

Fault Localization via Fine-tuning Large Language Models with Mutation Generated Stack Traces

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Fine-tuning a large language model on tens of thousands of mutation-generated crash stack traces lets it predict the faulty function from a production stack trace alone, with 66.9% accuracy on the SAP HANA code base versus 12.6% and 10.6%…

desk verdict A serious empirical paper with a strong synthetic-data result, but the production claim rests on an authentic evaluation filtered to the easy crashes. read the letter →

arxiv 2501.18005 v3 pith:OVE4ZUNK submitted 2025-01-29 cs.SE cs.LG

Neetha Jambigi , Bartosz Bogacz , Moritz Mueller , Thomas Bach , Michael Felderer This is my paper · ORCID
classification cs.SEcs.LG
keywords faultlocalizationstacktraceslargelanguagemodelsfine-tuningmutation-basedtestingsyntheticcrashdatarootcauseanalysisdatabasesystems
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that a stack trace alone, the only artefact available for many production crashes, contains enough signal to name the function that caused the crash, even when the faulty function is not the innermost frame. Because real crash data is too scarce to train on, the authors generate 64,369 synthetic crashes by mutating the SAP HANA code base and fine-tune open-source large language models to map preprocessed stack traces to a file path and function name. They report 66.9% exact-match accuracy on held-out synthetic HANA crashes for the best fine-tuned model, compared with 12.6% and 10.6% for the inner-frame and fastText baselines, and report transfer to authentic HANA crashes at 35.4% as well as to SQLite at 63% and DuckDB at 74%. The paper also claims that fine-tuned models consistently outperform non-fine-tuned, instruction-tuned LLMs prompted with the same stack traces. If this holds, crash triage in large systems could be automated from the crash report alone, with no failing tests and no source code access at prediction time.

What carries the argument

The load-bearing mechanism is the mutation-to-crash data pipeline: tree-sitter-based mutators, including assignment, comparison, boolean, arithmetic, and line-deletion operators, alter source lines; test suites execute the mutated code; crashes are captured as OS signals; and GDB or HANA's unwinding framework emits a stack trace. Traces are preprocessed by removing dynamic components such as memory addresses and thread identifiers, deduplicated, and paired with the mutated file path and function name as the training label. The LLM is fine-tuned with LoRA adapters on a next-token completion task, learning to generate the file path and function name from the preprocessed trace. The paper conceptualizes this as sparse sampling of the mutation-to-crash mapping, with the LLM acting as an interpolator that can also predict locations never seen during training.

What would settle it

Apply the fine-tuned model to a set of authentic production crashes whose fixes touch multiple files or whose root-cause function is absent from the stack trace, and measure exact-match accuracy. If accuracy on that unfiltered set is at or below the inner-frame or fastText baselines, the synthetic-to-real transfer claim fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that the mapping from a crash stack trace to the mutated function that caused it can be learned by an LLM trained on synthetic crashes, and that the learned mapping generalizes to authentic crashes. The authors frame mutated-program execution as a function from code mutations to crashes, and use the LLM as an interpolator that inverts a sparse sample of this mapping by learning crash-to-mutation associations. The model must output the root-cause function, which may be buried at any depth in the trace or absent from it entirely, rather than simply copying the innermost frame. Fine-tuned models outperform prompted non-fine-tuned models across all experiments, and the approach transfers across three database code bases written in C and C++, which the authors take as evidence that stack-trace-only fault localization is both viable and generalizable.

Load-bearing premise

The central premise is that crashes induced by simple random code mutations are close enough to real crash-causing defects that patterns learned on synthetic traces transfer to authentic production crashes.

Editorial extensions

If this is right

  • Stack-trace-only fault localization becomes practical for production environments that log only crashes, with no failing tests required.
  • Mutation-based synthetic crashes can substitute for rare real crash data when adapting LLMs to a specific code base.
  • Non-local crashes, where the root cause is absent from the trace, are learnable but substantially harder, with lower accuracy than local crashes in every experiment.
  • Small open-source models fine-tuned on domain-specific synthetic data can outperform much larger, general-purpose instruction-tuned models prompted zero-shot.
  • File-level localization is notably easier than function-level localization, with accuracy gains of up to 34 percentage points, so coarse-grained triage may be reliable even when exact function prediction is not.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 66.9% figure is measured on synthetic crashes drawn from the same mutation distribution used for training, while authentic-crash accuracy is 35.4% on a deliberately filtered subset, so real-world performance is likely lower than the headline number suggests.
  • A natural extension is a two-stage cascade that first predicts the file and then the function within that file, since file-level accuracy is much higher and could constrain the search space for function-level localization.
  • Because DropLine and comparison mutants produce the hardest non-local crashes, reweighting training data toward mutation operators that mimic authentic fault patterns may improve transfer to real crashes.
  • The obfuscation experiment indicates the model depends on the semantic content of method names; a testable variant would replace method names with abstract role tokens to separate structural learning from token memorization.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a fault localization approach that uses only stack traces from crashes, fine-tuning open-source LLMs on synthetic crash data generated by mutating the source code of SAP HANA, SQLite, and DuckDB. The mutation pipeline applies mutators to covered code, builds and runs tests, records stack traces for crashes, and maps each trace to the mutated function. Three LLMs (Santacoder-1B, Mistral-7B, CodeLlama-7B) are fine-tuned to output a file path and function name given a preprocessed stack trace. The authors report held-out synthetic accuracy of 66.9% on HANA, 63% on SQLite, and 74% on DuckDB, and 35.4% accuracy on 175 authentic HANA crashes that were filtered to single-file fixes with a fixed method present in the stack trace. They compare against innerframe, fasttext, and prompted non-fine-tuned LLMs, and report that fine-tuned models consistently outperform prompting. The paper also analyzes local vs. non-local crashes, unseen targets, mutator distributions, and average precision.

Significance. If the central claim holds, the approach would be practically valuable for production crash triage in systems where only stack traces and logs are available, and the paper explicitly targets the difficult case where the faulty function is not the innermost frame or is absent from the stack trace. The work has concrete strengths: it releases synthetic crash datasets for SQLite and DuckDB, it evaluates on authentic HANA crashes (albeit a filtered subset), it probes generalization to unseen targets, and it reports negative and null results for prompting baselines. The significance is tempered by two issues: the headline accuracy is measured on held-out synthetic crashes from the same mutation pipeline that generated the training data, and the authentic evaluation is restricted to local, easier crashes. The paper itself acknowledges the latter limitation in Section VII-E. The granularity mismatch between fine-tuned and baseline evaluation targets also weakens the reported performance gap until corrected.

major comments (3)
  1. [Section VI-A2 and VII-E] The authentic-crash evaluation is filtered to 175 crashes whose fixes touch only a single file and where at least one fixed method appears in the stack trace, which by the paper's own definition (Section IV-D) makes all authentic test cases local crashes. This removes exactly the non-local cases that the approach claims to handle (Sections III-B and IV-D). The paper explicitly concedes that 'our evaluation outcomes may be an overestimation of the efficacy of our approach.' The reported 35.4% authentic accuracy therefore does not substantiate transfer to real non-local crashes or to hard real-world faults. The authors should either evaluate on an unfiltered authentic set, report performance stratified by local/non-local, or clearly restrict their production-oriented claims to local crash cases.
  2. [Section VI-C1, VI-D1, and Table III] The headline comparison is not granularity-matched. Fine-tuned models are scored on exact match of file path plus function name (Section VI-B1/VI-D1), while the innerframe and fasttext baselines output only a function name, and the non-fine-tuned LLMs are additionally restricted to local crashes with function-name-only matching. The claimed gap of 66.9% versus 12.6% and 10.6% in Table III therefore conflates prediction granularity with model capability. The paper should report fine-tuned accuracy using function-name-only matching, or alternatively give the baselines the same file+function target, and should compare on identical crash subsets for the non-fine-tuned-LLM comparison.
  3. [Section VI-A1 and VII-A3] The held-out synthetic validation sets are produced by the same mutation pipeline as the training set, so part of the measured accuracy may reflect learning mutation-pipeline artifacts (e.g., which functions are commonly mutated, or stack-trace shapes induced by specific mutators) rather than general fault structure. The unseen-target evidence is thin: Table IV reports only 4, 2, 2 correct predictions on 25 SQLite unseen targets and 1, 4, 0 on 14 DuckDB unseen targets, with no HANA unseen-target numbers. The SQLite (63%) and DuckDB (74%) results are also on synthetic validation sets. The paper should add unseen-target results for HANA, report synthetic accuracy broken down by mutation operator and local/non-local status, and temper the abstract and conclusion claims so that 'generalization' is defined as cross-project synthetic transfer unless supported by real non-local fault data.
minor comments (5)
  1. [Section VII-A1] There is a typo: 'DuckB (to 62%)' should be 'DuckDB (to 62%)'.
  2. [Listing 1] The ROLE text says 'localizing the fault to a method in the stack trace,' but the example crash is non-local and the correct prediction sqlite3_malloc is absent from the shown trace; this contradicts the paper's own definition of non-local crashes in Section IV-D. Please align the prompt wording with the actual task.
  3. [Section VI-D2, Eq. (1)] The average precision formula uses |terms(Pi) ∩ terms(Ti)| without specifying whether terms(Pi) and terms(Ti) are sets or multisets; if duplicates are allowed, the set notation is ambiguous. Please clarify the definition.
  4. [Section VII-C3] The stack-trace obfuscation experiment uses only 10 SAP HANA examples and is presented as a qualitative observation; the paper should explicitly frame this as anecdotal rather than a quantitative evaluation.
  5. [Table I and Section VII-B] The table lists a mutator named 'Delete Blank line,' while the text later refers to 'Drop Line' as a mutator; these terms should be unified to avoid confusion.

Circularity Check

0 steps flagged · score 0.0 of 10

No definitional or construction-level circularity: the mutation-to-stack-trace labels are generated independently of the model, and held-out synthetic and authentic evaluations are genuine predictions; the paper's own stated limitations concern transfer validity, not circularity.

full rationale

The paper's derivation chain is a supervised learning pipeline rather than a self-referential construction. Training data are produced by mutating source code, executing test suites, and recording the resulting stack traces together with the identity of the mutated method (Sections IV-A to IV-C). The label, i.e., the mutated function, is fixed by the mutation process before any model is trained; it is not defined in terms of the model's output. The headline accuracy of 66.9% is measured on a held-out 10% validation split of the same mutation pipeline, with deduplication and random splitting intended to prevent leakage (Section VI-A1). This is standard held-out evaluation, not a reduction of the prediction to the training target by construction. The paper also reports predictions on targets absent from the training set (Table IV) and on 175 authentic HANA crashes (Table III, HANA-Auth); these are genuine held-out predictions because the authentic stack traces were not part of the fine-tuning data, even though the target function names overlap with functions seen during training. The paper explicitly concedes that the authentic evaluation is filtered and 'may be an overestimation of the efficacy of our approach' (Section VII-E), but that is a threat to external validity, not a circularity. The only overlapping self-citation, reference [4] by Bach et al. including co-author Thomas Bach, supports background statements about the scale of SAP HANA; the method's derivation does not depend on that citation. No equation or definition equates a predicted quantity with an input quantity, and no fitted parameter is renamed as a prediction. The baseline asymmetry (file plus function for fine-tuned models versus function name only for non-fine-tuned models) affects comparability but makes the fine-tuned task harder, so it does not constitute a circular advantage. Overall, the paper is self-contained as an empirical evaluation of a supervised model; its limitations are about generalization to real, hard crashes, not about circular reasoning.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the proxy validity of mutation-generated crashes, the sufficiency of stack traces, and ground-truth labels from injected mutations or filtered fixes. These are domain assumptions rather than derived facts; the paper partially acknowledges the first and second in Section VII-E.

free parameters (4)
  • Build/test timeout threshold per codebase = not reported
    Section IV-C: mutations whose build or execution exceeds a per-codebase threshold are discarded; the threshold is set from a successful build time and shapes which crashes enter the dataset.
  • Input token limit = 1024
    Section VI-C2: stack traces are truncated to 1024 tokens for all models, which affects which frames the model sees and therefore the predictions.
  • Fine-tuning hyperparameters (LoRA rank, learning rate, epochs) = not reported
    Only one plot shows accuracy over up to 30 epochs; the exact final training configuration for each model and dataset is not given, so the reported accuracies cannot be reproduced exactly.
  • Manual prompt optimization for non-fine-tuned LLMs = not applicable
    Section VI-C1: prompts are manually tuned per LLM and project using 10 sample cases; baseline performance is sensitive to this hand-tuning.
assumptions (5)
  • domain assumption Mutation operators produce faults representative of real faults.
    Section III-A relies on prior results [40], [41] to justify mutations as a proxy for real faults; if simple mutants do not resemble production crash causes, the synthetic training signal is misleading.
  • domain assumption Only code reachable by the existing test suite can be mutated into observed crashes.
    Section IV-A: coverage-guided mutation means untested code paths never appear in the dataset, so the model cannot learn faults there.
  • domain assumption A stack trace, even one without the faulty function, contains enough signal to identify the fault.
    Section III-B posits this to justify non-local crash prediction; the paper's non-local accuracy is much lower, roughly 0.27 to 0.57 across models and datasets.
  • domain assumption The methods modified in authentic fixes are the ground-truth root cause locations.
    Section VI-A2: authentic crashes are filtered to fixes in a single file, and one fixed method must appear in the stack trace; fixes may still implicate multiple methods and the actual root cause may differ.
  • domain assumption Crash extraction via GDB and HANA's unwinding framework captures the true call stack.
    Section IV-C: stack traces are collected from debugger output; if unwinding is incomplete or corrupted, labels and inputs are wrong.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Fault Localization via Fine-tuning Large Language Models with Mutation Generated Stack Traces." pith.science (2026). https://pith.science/paper/OVE4ZUNK

@misc{pith2026250118005,
  author       = {Pith},
  title        = {Pith review of: Fault Localization via Fine-tuning Large Language Models with Mutation Generated Stack Traces},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OVE4ZUNK}},
  note         = {Machine review of arXiv:2501.18005}
}
read the original abstract

Abrupt and unexpected terminations of software are termed as software crashes. They can be challenging to analyze. Finding the root cause requires extensive manual effort and expertise to connect information sources like stack traces, source code, and logs. Typical approaches to fault localization require either test failures or source code. Crashes occurring in production environments, such as that of SAP HANA, provide solely crash logs and stack traces. We present a novel approach to localize faults based only on the stack trace information and no additional runtime information, by fine-tuning large language models (LLMs). We address complex cases where the root cause of a crash differs from the technical cause, and is not located in the innermost frame of the stack trace. As the number of historic crashes is insufficient to fine-tune LLMs, we augment our dataset by leveraging code mutators to inject synthetic crashes into the code base. By fine-tuning on 64,369 crashes resulting from 4.1 million mutations of the HANA code base, we can correctly predict the root cause location of a crash with an accuracy of 66.9\% while baselines only achieve 12.6% and 10.6%. We substantiate the generalizability of our approach by evaluating on two additional open-source databases, SQLite and DuckDB, achieving accuracies of 63% and 74%, respectively. Across all our experiments, fine-tuning consistently outperformed prompting non-finetuned LLMs for localizing faults in our datasets.

Figures

Figures reproduced from arXiv: 2501.18005 by the authors.

Figure 1
Figure 1. Data creation process. void *sqlite3_malloc(int n){ #ifndef SQLITE_OMIT_AUTOINIT if( sqlite3_initialize()) return 0; #endif return (n) ? 0 : sqlite3Malloc(n);} void *sqlite3_malloc(int n){ #ifndef SQLITE_OMIT_AUTOINIT if( sqlite3_initialize() ) return 0; #endif return n<=0 ? 0 : sqlite3Malloc(n);} Program received signal SIGSEGV, Segmentation fault. fs_register () at src/test_onefile.c:824 824 fs_vfs.base.mxPathname… view at source ↗
Figure 2
Figure 2. Example of Non-local Crash due to Mutation to [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. HANA Validation Accuracy on Santacoder-1B. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Top 3 Mutator distribution in Synthetic HANA Dataset. [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

55 extracted references · 39 canonical work pages

  1. [1]

    A survey on software fault localization,

    W. E. Wong, R. Gao, Y . Li, R. Abreu, and F. Wotawa, “A survey on software fault localization,” IEEE Transactions on Software Engineering, vol. 42, no. 8, pp. 707–740, 2016. (Cited on page: 1)

    work page 2016

  2. [2]

    The SAP HANA Database–An Architecture Overview

    F. Färber, N. May, W. Lehner, P. Große, I. Müller, H. Rauhe, and J. Dees, “The SAP HANA Database–An Architecture Overview.” IEEE Data Eng. Bull., vol. 35, no. 1, pp. 28–33, 2012. (Cited on pages: 1 and 4)

    work page 2012

  3. [3]

    SAP HANA database: data management for modern business applica- tions,

    F. Färber, S. K. Cha, J. Primsch, C. Bornhövd, S. Sigg, and W. Lehner, “SAP HANA database: data management for modern business applica- tions,” ACM Sigmod Record, vol. 40, no. 4, pp. 45–51, 2012. (Cited on page: 1)

    work page 2012

  4. [4]

    Testing very large database management systems: The case of SAP HANA,

    T. Bach, A. Andrzejak, C. Seo, C. Bierstedt, C. Lemke, D. Ritter, D. W. Hwang, E. Sheshi, F. Schabernack, F. Renkes et al., “Testing very large database management systems: The case of SAP HANA,” Datenbank- Spektrum, vol. 22, no. 3, pp. 195–215, 2022. (Cited on page: 1)

    work page 2022

  5. [5]

    Boosting bug-report-oriented fault localization with segmentation and stack-trace analysis,

    C.-P. Wong, Y . Xiong, H. Zhang, D. Hao, L. Zhang, and H. Mei, “Boosting bug-report-oriented fault localization with segmentation and stack-trace analysis,” in 2014 IEEE international Conf. on software maintenance and evolution . IEEE, 2014, pp. 181–190. (Cited on page: 1)

    work page 2014

  6. [6]

    Fault localization and repair for Java runtime exceptions,

    S. Sinha, H. Shah, C. Görg, S. Jiang, M. Kim, and M. J. Harrold, “Fault localization and repair for Java runtime exceptions,” in Proc.gs of the eighteenth international symposium on Software testing and analysis , 2009, pp. 153–164. (Cited on page: 1)

    work page 2009

  7. [7]

    A Preliminary Evaluation of LLM-Based Fault Localization,

    S. Kang, G. An, and S. Yoo, “A Preliminary Evaluation of LLM-Based Fault Localization,” 2023. (Cited on pages: 1 and 2)

    work page 2023

  8. [8]

    Locating crashing faults based on crash stack traces,

    L. Gong, H. Zhang, H. Seo, and S. Kim, “Locating crashing faults based on crash stack traces,” arXiv preprint arXiv:1404.4100 , 2014. (Cited on page: 1)

    arXiv 2014

Show all 55 references
  1. [9]

    Do Stack Traces Help Developers Fix Bugs?

    A. Schroter, A. Schröter, N. Bettenburg, and R. Premraj, “Do Stack Traces Help Developers Fix Bugs?” in 2010 7th IEEE Working Conf. on Mining software repositories (MSR 2010) . IEEE, 2010, pp. 118–121. (Cited on pages: 1 and 3)

    2010

  2. [10]

    An empirical study of fault localization families and their combinations,

    D. Zou, J. Liang, Y . Xiong, M. D. Ernst, and L. Zhang, “An empirical study of fault localization families and their combinations,” IEEE TSE, vol. 47, no. 2, pp. 332–347, 2019. (Cited on page: 1)

    2019

  3. [11]

    Defects4J: A database of existing faults to enable controlled testing studies for Java programs,

    R. Just, D. Jalali, and M. D. Ernst, “Defects4J: A database of existing faults to enable controlled testing studies for Java programs,” in Proc. of the 2014 ISSTA , 2014, pp. 437–440. (Cited on pages: 1 and 3)

    2014

  4. [12]

    Leveraging Stack Traces for Spectrum-based Fault Localization in the Absence of Failing Tests,

    L. Barreto Simedo Pacheco, A. R. Chen, J. Yang et al. , “Leveraging Stack Traces for Spectrum-based Fault Localization in the Absence of Failing Tests,” arXiv e-prints, pp. arXiv–2405, 2024. (Cited on pages: 1 and 2)

    2024

  5. [13]

    Empirical evaluation of the Tarantula automatic fault-localization technique,

    J. A. Jones and M. J. Harrold, “Empirical evaluation of the Tarantula automatic fault-localization technique,” in Proc.of the 20th IEEE/ACM International Conf. on ASE , 2005, pp. 273–282. (Cited on pages: 1 and 2)

    2005

  6. [14]

    The DStar method for effective software fault localization,

    W. E. Wong, V . Debroy, R. Gao, and Y . Li, “The DStar method for effective software fault localization,” IEEE Transactions on Reliability , vol. 63, no. 1, pp. 290–308, 2013. (Cited on pages: 1 and 2)

    2013

  7. [15]

    Few-shot training LLMs for project-specific code-summarization,

    T. Ahmed and P. Devanbu, “Few-shot training LLMs for project-specific code-summarization,” in Proc. of the 37th IEEE/ACM International Conf. on ASE, 2022, pp. 1–5. (Cited on page: 1)

    2022

  8. [16]

    Assessing the quality of GitHub copilot’s code generation,

    B. Yetistiren, I. Ozsoy, and E. Tuzun, “Assessing the quality of GitHub copilot’s code generation,” in Proc. of the 18th International Conf. on Predictive Models and Data Analytics in Software Engineering , 2022, pp. 62–71. (Cited on page: 1)

    2022

  9. [17]

    An initial investigation of ChatGPT unit test generation capability,

    V . Guilherme and A. Vincenzi, “An initial investigation of ChatGPT unit test generation capability,” in 8th Brazilian Symposium on Systematic and Automated Software Testing , 2023, pp. 15–24. (Cited on page: 1)

    2023

  10. [18]

    Repair is nearly generation: Multilingual program repair with LLMs,

    H. Joshi, J. C. Sanchez, S. Gulwani, V . Le, G. Verbruggen, and I. Radiˇcek, “Repair is nearly generation: Multilingual program repair with LLMs,” in Proc. of the AAAI Conf. on Artificial Intelligence , vol. 37, no. 4, 2023, pp. 5131–5140. (Cited on pages: 1 and 2)

    2023

  11. [19]

    Large Language Models for Test-Free Fault Localization,

    A. Z. Yang, C. Le Goues, R. Martins, and V . J. Hellendoorn, “Large Language Models for Test-Free Fault Localization,” 2024. (Cited on pages: 1 and 2)

    2024

  12. [20]

    Large Language Models in Fault Localisation,

    Y . Wu, Z. Li, J. M. Zhang, M. Papadakis, M. Harman, and Y . Liu, “Large Language Models in Fault Localisation,” arXiv preprint arXiv:2308.15276, 2023. (Cited on page: 1)

    2023 arXiv

  13. [21]

    A survey of Large Language Models,

    W. X. Zhao, K. Zhou, J. Li, T. Tang, X. Wang, Y . Hou, Y . Min, B. Zhang, J. Zhang, Z. Dong et al., “A survey of Large Language Models,” arXiv preprint arXiv:2303.18223, 2023. (Cited on pages: 1 and 2)

    2023 arXiv

  14. [22]

    Spectrum-based software fault localization: A survey of techniques, advances, and challenges,

    H. A. de Souza, M. L. Chaim, and F. Kon, “Spectrum-based software fault localization: A survey of techniques, advances, and challenges,” arXiv preprint arXiv:1607.04347 , 2016. (Cited on page: 2)

    2016 arXiv

  15. [23]

    Ask the mutants: Mutating faulty programs for fault localization,

    S. Moon, Y . Kim, M. Kim, and S. Yoo, “Ask the mutants: Mutating faulty programs for fault localization,” in 2014 IEEE Seventh ICST . IEEE, 2014, pp. 153–162. (Cited on page: 2)

    2014

  16. [24]

    Metallaxis-FL: mutation-based fault localization,

    M. Papadakis and Y . Le Traon, “Metallaxis-FL: mutation-based fault localization,” Software Testing, Verification and Reliability , vol. 25, no. 5-7, pp. 605–628, 2015. (Cited on page: 2)

    2015

  17. [25]

    A survey of challenges in spectrum-based software fault localization,

    Q. I. Sarhan and Á. Beszédes, “A survey of challenges in spectrum-based software fault localization,” IEEE Access , vol. 10, pp. 10 618–10 639,

  18. [26]

    Learning to combine multiple ranking metrics for fault localization,

    J. Xuan and M. Monperrus, “Learning to combine multiple ranking metrics for fault localization,” in 2014 IEEE ICSME . IEEE, 2014, pp. 191–200. (Cited on page: 2)

    2014

  19. [27]

    Transforming programs and tests in tandem for fault localization,

    X. Li and L. Zhang, “Transforming programs and tests in tandem for fault localization,” Proc.of the ACM on Programming Languages , vol. 1, no. OOPSLA, pp. 1–30, 2017. (Cited on page: 2)

    2017

  20. [28]

    DeepFL: Integrating multiple fault diagnosis dimensions for deep fault localization,

    X. Li, W. Li, Y . Zhang, and L. Zhang, “DeepFL: Integrating multiple fault diagnosis dimensions for deep fault localization,” in Proc.of the 28th ACM SIGSOFT ISSTA , 2019, pp. 169–180. (Cited on page: 2)

    2019

  21. [29]

    Boosting coverage-based fault localization via graph-based representation learning,

    Y . Lou, Q. Zhu, J. Dong, X. Li, Z. Sun, D. Hao, L. Zhang, and L. Zhang, “Boosting coverage-based fault localization via graph-based representation learning,” in Proc. of the 29th ACM Joint Meeting on Eu- ropean Software Engineering Conf. and Symposium on the Foundations of So...

    2021

  22. [30]

    Fault localization to detect co- change fixing locations,

    Y . Li, S. Wang, and T. N. Nguyen, “Fault localization to detect co- change fixing locations,” in Proc. of the 30th ACM Joint European Software Engineering Conf. and Symposium on the Foundations of Software Engineering, 2022, pp. 659–671. (Cited on page: 2)

    2022

  23. [31]

    Fault localization with code coverage representation learning,

    Y . Li, S. Wang, and T. Nguyen, “Fault localization with code coverage representation learning,” in 2021 IEEE/ACM 43rd ICSE . IEEE, 2021, pp. 661–673. (Cited on page: 2)

    2021

  24. [32]

    Improving fault localization and program repair with deep semantic features and transferred knowledge,

    X. Meng, X. Wang, H. Zhang, H. Sun, and X. Liu, “Improving fault localization and program repair with deep semantic features and transferred knowledge,” in Proc. of the 44th ICSE , 2022, pp. 1169–1180. (Cited on page: 2)

    2022

  25. [33]

    Large Language Models for Software Engineering: A Systematic Literature Review,

    X. Hou, Y . Zhao, Y . Liu, Z. Yang, K. Wang, L. Li, X. Luo, D. Lo, J. Grundy, and H. Wang, “Large Language Models for Software Engineering: A Systematic Literature Review,” 2023. (Cited on page: 2)

    2023

  26. [34]

    Large Language Models for Software Engineering: Survey and Open Problems,

    A. Fan, B. Gokkaya, M. Harman, M. Lyubarskiy, S. Sengupta, S. Yoo, and J. M. Zhang, “Large Language Models for Software Engineering: Survey and Open Problems,” arXiv preprint arXiv:2310.03533 , 2023. (Cited on pages: 2 and 3)

    2023 arXiv

  27. [35]

    GPT-4 technical report,

    OpenAI, “GPT-4 technical report,” ArXiv, vol. abs/2303.08774, 2023. [Online]. Available: https://arxiv.org/abs/2303.08774 (Cited on page: 2)

    2023 arXiv

  28. [36]

    InferFix: End-to-end program repair with LLMs,

    M. Jin, S. Shahriar, M. Tufano, X. Shi, S. Lu, N. Sundaresan, and A. Svyatkovskiy, “InferFix: End-to-end program repair with LLMs,” arXiv preprint arXiv:2303.07263 , 2023. (Cited on page: 2)

    2023 arXiv

  29. [37]

    Sparks of artificial general intelligence: Early experiments with GPT-4,

    S. Bubeck, V . Chandrasekaran, R. Eldan, J. Gehrke, E. Horvitz, E. Kamar, P. Lee, Y . T. Lee, Y . Li, S. Lundberg et al. , “Sparks of artificial general intelligence: Early experiments with GPT-4,” arXiv preprint arXiv:2303.12712, 2023. (Cited on page: 2)

    2023 arXiv

  30. [38]

    Evaluating large language models trained on code,

    M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. d. O. Pinto, J. Kaplan, H. Edwards, Y . Burda, N. Joseph, G. Brockman et al., “Evaluating large language models trained on code,” arXiv preprint arXiv:2107.03374 ,

    arXiv

  31. [39]

    Fairness of ChatGPT and the role of explainable-guided prompts,

    Y . Deldjoo, “Fairness of ChatGPT and the role of explainable-guided prompts,” arXiv preprint arXiv:2307.11761 , 2023. (Cited on page: 2)

    2023 arXiv

  32. [40]

    Are mutants a valid substitute for real faults in software testing?

    R. Just, D. Jalali, L. Inozemtseva, M. D. Ernst, R. Holmes, and G. Fraser, “Are mutants a valid substitute for real faults in software testing?” in Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering , 2014, pp. 654–665. (Cited on...

    2014

  33. [41]

    How Closely are Common Mutation Operators Coupled to Real Faults?

    G. Gay and A. Salahirad, “How Closely are Common Mutation Operators Coupled to Real Faults?” in 2023 IEEE Conf. on Software Testing, Verification and Validation (ICST). IEEE, 2023, pp. 129–140. (Cited on page: 3)

    2023

  34. [42]

    BugsInPy: a database of existing bugs in Python programs to enable controlled testing and debugging studies,

    R. Widyasari, S. Q. Sim, C. Lok, H. Qi, J. Phan, Q. Tay, C. Tan, F. Wee, J. E. Tan, Y . Yieh et al. , “BugsInPy: a database of existing bugs in Python programs to enable controlled testing and debugging studies,” in Proc. of the 28th ACM joint meeting on european software engi...

    2020

  35. [43]

    SQLite - A Small, Fast, Self-Contained, High- Reliability, Full-Featured, SQL Database Engine,

    D. R. Hipp, “SQLite - A Small, Fast, Self-Contained, High- Reliability, Full-Featured, SQL Database Engine,” ACM SIGMOD Record, vol. 29, no. 2, pp. 1–12, 2000. [Online]. Available: https://dl.acm.org/doi/10.1145/354805.354806 (Cited on page: 4)

  36. [44]

    Duckdb: An embeddable analytical database,

    M. Raasveldt and H. Mühleisen, “Duckdb: An embeddable analytical database,” in Proc. of the 2019 International Conf. on Management of Data, 2019, pp. 1981–1984. (Cited on page: 4)

    2019

  37. [45]

    Mixtral of experts,

    A. Q. Jiang, A. Sablayrolles, A. Roux, A. Mensch, B. Savary, C. Bamford, D. S. Chaplot, D. d. l. Casas, E. B. Hanna, F. Bressand et al., “Mixtral of experts,” arXiv preprint arXiv:2401.04088 , 2024. (Cited on page: 6)

    2024 arXiv

  38. [46]

    Llama 2: Open foundation and fine-tuned chat models,

    H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y . Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale et al. , “Llama 2: Open foundation and fine-tuned chat models,” arXiv preprint arXiv:2307.09288, 2023. (Cited on page: 6)

    2023 arXiv

  39. [47]

    Mistral 7b,

    A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. d. l. Casas, F. Bressand, G. Lengyel, G. Lample, L. Saulnier et al., “Mistral 7b,” arXiv preprint arXiv:2310.06825, 2023. (Cited on page: 6)

    2023 arXiv

  40. [48]

    Santacoder: don’t reach for the stars!

    L. B. Allal, R. Li, D. Kocetkov, C. Mou, C. Akiki, C. M. Ferrandis, N. Muennighoff, M. Mishra, A. Gu, M. Dey et al., “Santacoder: don’t reach for the stars!” arXiv preprint arXiv:2301.03988 , 2023. (Cited on page: 6)

    2023 arXiv

  41. [49]

    Code llama: Open foundation models for code,

    B. Roziere, J. Gehring, F. Gloeckle, S. Sootla, I. Gat, X. E. Tan, Y . Adi, J. Liu, R. Sauvestre, T. Remez et al. , “Code llama: Open foundation models for code,” arXiv preprint arXiv:2308.12950 , 2023. (Cited on page: 6)

    2023 arXiv

  42. [50]

    Lora: Low-rank adaptation of large language models,

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” arXiv preprint arXiv:2106.09685, 2021. (Cited on page: 7)

    2021 arXiv

  43. [51]

    Enriching word vectors with subword information,

    P. Bojanowski, E. Grave, A. Joulin, and T. Mikolov, “Enriching word vectors with subword information,” Transactions of the Association for Computational Linguistics, vol. 5, pp. 135–146, 2017. (Cited on page: 7)

    2017

  44. [52]

    BLEU: a method for automatic evaluation of machine translation,

    K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “BLEU: a method for automatic evaluation of machine translation,” in Proceedings of the 40th Annual Meeting on Association for Computational Linguistics . Association for Computational Linguistics, 2002, pp. 311–318. (Cited on page: 7)

    2002

  45. [53]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017. (Cited on page: 8)

    2017

  46. [54]

    Secure hash standard (shs),

    F. Pub, “Secure hash standard (shs),” Fips pub, vol. 180, no. 4, 2012. (Cited on page: 9)

    2012

  47. [2021]

    (Cited on pages: 2 and 3)

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.