Pith. sign in

REVIEW 3 major objections 6 minor 74 references

Your Fix Is My Exploit: Enabling Comprehensive DL Library API Fuzzing with Large Language Models

T0 review · 3 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that defensive checks in one DL library API, when read by an LLM and abstracted to type-level edge cases, can be transferred to similar APIs and produce both higher coverage and real bug discoveries.

desk verdict A real advance in LLM-based fuzzing, with a bug count that needs triage before the headline is trusted. read the letter →

arxiv 2501.04312 v1 pith:KV5LJPJ7 submitted 2025-01-08 cs.SE

Kunpeng Zhang , Shuai Wang , Jitao Han , Xiaogang Zhu , Xian Li , Shaohua Wang , Sheng Wen This is my paper · ORCID
classification cs.SE
keywords deeplearninglibraryfuzzingAPIlargelanguagemodelsedgecaseextractiontype-basedtransfercoveragePyTorchTensorFlow
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

DFUZZ's central claim is that the input checks buried in DL library source code are reusable test inputs: an edge case that triggers a TORCH_CHECK in one API can be lifted to an abstract, type-based form and fed to other APIs with the same parameter types. Because LLMs can read those checks and name the failing inputs, DFUZZ automates what previously required human-written mutators or historical bug corpora. The paper reports that this mechanism covers more APIs than prior LLM-based fuzzers on TensorFlow and PyTorch while using a fraction of the LLM calls, and uncovers 37 bugs in the latest versions, with 8 fixed and 19 acknowledged by developers. A fuzzer that treats fixes as exploits is the practical payoff: the defenders' own guards reveal how APIs fail.

What carries the argument

The machinery is the etype-pattern abstraction: each edge case is paired with a set of parameter types (e.g., {Tensor} or {Int, Tensor}) so that it can be detached from the specific API it came from and matched to any API with the same or a superset of those parameter types. Around this abstraction DFUZZ wraps a three-step pipeline: a Code Extractor that pulls check statements and function interfaces out of the library source; an Analyzer that asks an LLM to read each check and report the checked variables, their types, and the edge case; and a Standardizer that strips variable names to produce context-free edge cases clustered by etype pattern. A debug loop then synthesizes an initial program invoking the target API, and a mutation step selects matched edge cases, prioritizing those that affect early parameters, to generate bug-triggering programs. The abstraction carries the argument because it is what makes knowledge transfer across APIs and across frameworks possible.

What would settle it

Run DFUZZ on a fixed set of matched (edge case, API) pairs and count how often a transferred edge case actually triggers a crash, abort, segfault, runtime error, or CPU/GPU inconsistency; if this transfer success rate is no higher than applying the same edge cases to APIs with mismatched type signatures, the central transferability assumption collapses.

Watch

Extended reading notes

Core claim

The paper's central discovery is that edge cases are not API-specific but type-specific. Reading TORCH_CHECK statements in PyTorch's ATen source, an LLM can articulate what input would violate the check (e.g., 'a complex tensor'), and that description remains valid after variable names are stripped, because the only thing that matters is the parameter's type. DFUZZ clusters these context-free edge cases under 'etype patterns' built from seven primitive types, then matches any API whose parameter types contain that pattern. The empirical payoff is the transfer itself: edge cases extracted exclusively from PyTorch produced 27 of the 37 reported bugs when applied to TensorFlow, and most of those bugs already existed in versions that prior fuzzers had tested. The paper interprets this as evidence that white-box reasoning about checks, rather than black-box mutation, is what unlocks comprehensive DL library fuzzing.

Load-bearing premise

The method stands on the premise that APIs with the same input parameter types share the same edge cases, so an edge case harvested from one API remains a meaningful test input for another API with the same type signature.

Editorial extensions

If this is right

  • A library's own check statements become a reusable fuzz corpus, so the cost of fuzzing a new DL library drops to the cost of extracting its checks and matching them to etype patterns.
  • Coverage need not be proportional to hand-written mutators or web-scraped bug codes; it scales with the number and diversity of input checks in the framework.
  • Cross-framework transfer means a bug pattern discovered in one ecosystem can be tested immediately in another, as demonstrated by PyTorch-harvested edge cases finding TensorFlow bugs.
  • Because most reported bugs are crashes on invalid inputs, DFUZZ's edge-case mutations align with the field's expectation that DL APIs should reject bad inputs gracefully rather than crash.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the seven-type abstraction probably both over-approximates (same type label does not guarantee the same edge case applies) and under-approximates (tensor dtype, shape, layout, and sparsity are erased); a finer type lattice would likely cut false transfers and find more precise bugs.
  • Editorial inference: the same 'fix is my exploit' mechanism could power differential testing across backends or frameworks, since each check is a candidate input to compare implementations against each other.
  • Editorial inference: the paper's coverage numbers justify a cheaper validation study: measure the per-edge-case transfer hit rate directly, rather than only reporting aggregate bugs, to quantify how much of the gain comes from transfer versus from LLM program generation.
  • Editorial inference: if the heuristic holds beyond DL libraries, any software with prevalent inline parameter checks, such as the financial software the paper mentions, could be fuzzed with the same pipeline.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper presents DFUZZ, an LLM-driven fuzzing framework for deep learning libraries. DFUZZ uses GPT-3.5 to extract 'edge cases' (error-triggering conditions) from TORCH_CHECK statements in PyTorch's ATen source code, abstracts them into a context-free form indexed by coarse 'etype patterns' (seven basic types plus compound combinations), and then transfers these edge cases to test other APIs in PyTorch and TensorFlow. A second LLM component generates initial test programs with a debug loop, and a third stage mutates those programs to satisfy the transferred edge cases. Evaluation on PyTorch v1.12 and TensorFlow v2.10 reports higher API coverage than TitanFuzz and FuzzGPT with fewer LLM invocations (Table III), and on latest versions (PyTorch v2.2.1, TensorFlow v2.15) reports 37 bugs, of which 8 are developer-fixed and 19 are replicated by developers (Table V). The authors attribute the TensorFlow bug discoveries to cross-framework transferability of edge cases extracted from PyTorch.

Significance. If the results hold, DFUZZ is a meaningful advance: it achieves higher API coverage than prior LLM-based fuzzers while using only a fraction of the LLM calls, and it reports genuine developer-confirmed fixes (8 fixed bugs), which is strong external evidence that at least some findings are real defects. The paper also gives a concrete mechanism for cross-framework transfer (etype abstraction), and the coverage comparison is performed in the same Docker environment as TitanFuzz, making that part of the evaluation credible. The pilot study on LLM-based check comprehension (Table II) is a useful data point. However, the bug-finding and transferability claims rest on two load-bearing assumptions that are not yet rigorously validated: that abort signals in TensorFlow constitute bugs under a defensible oracle, and that the seven-type etype abstraction transfers edge cases without a high false-positive rate. The lack of a control for false transfers and the absence of a per-bug disaggregation by type/status leave the central 'transferable edge cases find real bugs' claim under-supported, though not refuted.

major comments (3)
  1. [Sec. V-B, Table VI] The bug oracle is not calibrated for TensorFlow abort signals. Table VI reports that 24 of the 27 TensorFlow findings are 'abort signals,' and TensorFlow's CHECK/LOG(FATAL) macros abort the process by design on invalid arguments. The paper cites DocTer's doctrine that 'DL API functions should not crash' as justification, but that is a design preference, not a universal correctness contract. Without a per-case triage showing that these 24 aborts occur on inputs the API is documented or intended to accept (or that maintainers acknowledged them as bugs), the 37-bug headline may substantially overstate the number of real defects. The paper should disaggregate the 8 fixed and 19 replicated bugs by bug type and framework, and explain, for the TensorFlow abort signals specifically, how they were validated against the library's intended error-handling behavior.
  2. [Sec. II-B and Sec. III-A] Observation II ('if APIs have the same input parameter types, they shall likely share edge cases') is implemented through only seven coarse type categories plus 13 compound etype patterns, yet the evaluation validates this heuristic only indirectly through the bugs found. There is no control measuring how often applying a matched edge case to an arbitrary API produces a 'bug' under the paper's oracle, nor a comparison against random mutations with similar input distributions. Without such a control, the observed bug counts could partly reflect aggressive testing with unusual inputs rather than the transferability of semantically meaningful edge cases. The authors should add an ablation or control experiment, e.g., applying the same set of edge cases to APIs with mismatched etype patterns, or comparing bug yield per generated test against random type-matched mutations, to demonstrate that the transferred edge cases are specifically what drives the advantage.
  3. [Sec. V-B, RQ2] The claim that DFUZZ finds bugs missed by TitanFuzz, FuzzGPT, and IvySyn is inferred from the bugs' existence in older versions tested by those tools, rather than from a controlled side-by-side run. This inference assumes that those tools, given their schedules and mutation budgets, would have triggered the bugs if they existed in the code they tested. The paper should either rerun the baselines on the same versions (at least on the specific bug-triggering inputs) or explicitly discuss the limits of the version-existence argument. As it stands, the statement that 'none of these fuzzers discovered them' is a claim about the baselines' efficacy that is not directly tested in this paper.
minor comments (6)
  1. [Sec. III-A, prompt in Fig. 5] The type list is given as '{Tensor, Int, Bool, Str, Float, Scalar, List}' but the text later refers to 'Str' and 'str' inconsistently; the paper should standardize the type names and state explicitly whether Python-level types (e.g., 'str' vs 'Str') are intended.
  2. [Alg. 1, line 11] There is a typo: 'ruturn' should be 'return'.
  3. [Table IV] The table reports 198 successful and 138 failed debugging attempts for PyTorch, but the text says 336 APIs require debugging; please reconcile these numbers and clarify whether 'attempts' refers to APIs or to individual debug iterations.
  4. [Sec. V-C, RQ3] The evaluation of llama2 models replays prompts obtained from bugs discovered by ChatGPT-3.5; this measures the smaller models' ability to reproduce known bug-triggering programs, not their ability to discover new bugs in an end-to-end run. The claim that DFUZZ 'can effectively employ' these LLMs should be softened accordingly.
  5. [Sec. V-B, 'Transferability' paragraph] The statement that edge cases extracted from PyTorch are 'more effective' for TensorFlow than for PyTorch is interesting, but the comparison is confounded by different API sets, fix histories, and testing efforts; please avoid causal language without additional evidence.
  6. [Sec. VII and Sec. II-A] The phrase 'white-box view' is used throughout, but the extraction is performed on source code of PyTorch only; clarify that TensorFlow is tested using edge cases transferred from PyTorch rather than its own source checks.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; DFUZZ's coverage and bug claims are empirically benchmarked against external tools and developer-confirmed fixes.

full rationale

DFUZZ's derivation chain is not circular. The paper's central claims are (i) LLMs can extract edge cases from API checks (validated by a manual ground-truth pilot study on 50 random ATen functions, Table II); (ii) edge cases transfer across etype-matched APIs (Observation II), tested empirically by using PyTorch-derived edge cases against TensorFlow and reporting 27 bugs; and (iii) DFUZZ achieves higher API coverage with fewer LLM calls than TitanFuzz/FuzzGPT (Table III, run in TitanFuzz's Docker environment). Nothing in these steps defines the target result into the inputs. The etype pattern abstraction (Sec. III-A) is a stated heuristic, not a fitted parameter calibrated to the 37-bug figure; the bug count is measured against an external oracle (developer triage, fixes, and replication), and the coverage comparison uses external versions and baselines. The only internal loop--using the same LLM to extract edge cases and to synthesize mutating programs--is a design choice, not an equation reducing output to input. The skeptic's concern that TensorFlow abort signals may overcount as bugs is a threat to bug-oracle validity, not circularity, and the 8 developer-fixed bugs remain independent external evidence. No load-bearing self-citation or uniqueness import appears; prior work is cited for comparison or context, not to justify DFUZZ's transfer rule.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claims rest on three domain assumptions about how DL libraries encode edge cases and how LLM extraction scales; the hand-set loop and sampling parameters are engineering choices rather than fitted constants.

free parameters (3)
  • INIT_MAX = 2
    Maximum tries for generating a fresh initial program; hand-set based on preliminary exploration (Sec. III-B), affects coverage but not fitted to bug targets.
  • DEBUG_MAX = 3
    Maximum debug iterations per initial program; hand-set (Sec. III-B), affects coverage cost.
  • individual edge case sampling rates = 100% for first two parameters, 25% for positions 3-4, 12.5% for later
    Random selection rates for edge cases in mutation (Sec. III-C); hand-chosen and could affect bug discovery.
assumptions (4)
  • domain assumption Inline checks (e.g., TORCH_CHECK) are the primary way DL libraries handle input edge cases.
    Stated in Sec. II-B: 'such inline checks are the primary way to handle edge cases in DL libraries.' If false, check-derived edge cases are incomplete.
  • domain assumption APIs with the same etype pattern share edge cases.
    Observation II in Sec. II-B; load-bearing for cross-API and cross-framework transfer. Validated only indirectly by bug findings, not by a systematic study.
  • domain assumption Seven basic types (Tensor, Int, Bool, Str, Float, Scalar, List) are sufficient to abstract edge-case applicability.
    Sec. III-A Standardizer uses only these types; complex tensor attributes are folded into edge-case text rather than the type pattern.
  • domain assumption LLM extraction accuracy measured on 50 functions and 73 checks generalizes to the full ATen codebase.
    Pilot study in Sec. II-C reports 95%+ accuracy on a small sample; scaling to thousands of checks is assumed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Your Fix Is My Exploit: Enabling Comprehensive DL Library API Fuzzing with Large Language Models." pith.science (2026). https://pith.science/paper/KV5LJPJ7

@misc{pith2026250104312,
  author       = {Pith},
  title        = {Pith review of: Your Fix Is My Exploit: Enabling Comprehensive DL Library API Fuzzing with Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KV5LJPJ7}},
  note         = {Machine review of arXiv:2501.04312}
}
read the original abstract

Deep learning (DL) libraries, widely used in AI applications, often contain vulnerabilities like buffer overflows and use-after-free errors. Traditional fuzzing struggles with the complexity and API diversity of DL libraries such as TensorFlow and PyTorch, which feature over 1,000 APIs. Testing all these APIs is challenging due to complex inputs and varied usage patterns. While large language models (LLMs) show promise in code understanding and generation, existing LLM-based fuzzers lack deep knowledge of API edge cases and struggle with test input generation. To address this, we propose DFUZZ, an LLM-driven fuzzing approach for DL libraries. DFUZZ leverages two insights: (1) LLMs can reason about error-triggering edge cases from API code and apply this knowledge to untested APIs, and (2) LLMs can accurately synthesize test programs to automate API testing. By providing LLMs with a "white-box view" of APIs, DFUZZ enhances reasoning and generation for comprehensive fuzzing. Experimental results show that DFUZZ outperforms state-of-the-art fuzzers in API coverage for TensorFlow and PyTorch, uncovering 37 bugs, with 8 fixed and 19 under developer investigation.

Figures

Figures reproduced from arXiv: 2501.04312 by the authors.

Figure 1
Figure 1. Sample source code in PyTorch. Real-World Example [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. The workflow of DFUZZ. With the key insights presented in Sec. II, we now present the design of DFUZZ, a framework for comprehensive API fuzzing. As in [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The workflow for extracting edge cases. Tensor pool2d(…… IntArrayRef stride_arg, ……) { …… TORCH_CHECK(!stride_arg.empty(), "Stride cannot be empty!"); …… return convert(v_output); } Tensor pool2d(…… const IntArrayRef kernel_arg, ……) { TORCH_CHECK(!stride_arg.empty(), "Stride cannot be empty!"); (a) Original codes (b) Extracted check-related code block { "The 1st TORCH_CHECK statement": { "The statement": "TORCH_CHEC… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Edge case extraction example. reflects the types of input parameters that an API expects in accordance with the analyzed edge case. For example, suppose an edge case’s etype pattern is {tensor, tensor}, then this edge case can be used to test APIs that expect two tenso…
Figure 5
Figure 5. Figure 5: The prompt used by Analyzer. Write a program to generate input data and invoke <API_NAME> to process input data. Here are some information of this api: ``` <API_INFO> ``` [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 7
Figure 7. Figure 7: The prompt of mutation. IV. IMPLEMENTATION The core testing process of DFUZZ is implemented in Python with approximately 4,000 lines of code; see our artifact at [2]. Check Identification. Without losing generality, we extract edge cases from the checks found in the so…
Figure 8
Figure 8. Figure 8: The case study. TABLE VI BUG TYPES. Library abort signals segfaults runtime error inconsistent PyTorch 0 2 5 3 TensorFlow 24 2 0 1 Total 24 4 5 4 TABLE VII EVALUATING OTHER LLMS. Target Library ChatGPT-3.5 Llama-2-70b-chat Llama-2-7b-chat PyTorch 10 3 2 TensorFlow 27 1…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

74 extracted references · 51 canonical work pages

  1. [1]

    https://openai.com/chatgpt, 2024

    Chatgpt. https://openai.com/chatgpt, 2024

    work page 2024

  2. [2]

    https://github.com/DFUZZ-ICSE/DFUZZ, 2024

    Dfuzz. https://github.com/DFUZZ-ICSE/DFUZZ, 2024

    work page 2024

  3. [3]

    https://github.com/meta-llama/llama, 2024

    llama. https://github.com/meta-llama/llama, 2024

    work page 2024

  4. [4]

    http://pytorch.org, 2024

    Pytorch. http://pytorch.org, 2024

    work page 2024

  5. [5]

    https://replicate.com, 2024

    Replicate. https://replicate.com, 2024

    work page 2024

  6. [6]

    https://www.tensorflow.org, 2024

    Tensorflow. https://www.tensorflow.org, 2024

    work page 2024

  7. [7]

    Deep learning-based methods in structural reliability analysis: a review

    Sajad Saraygord Afshari, Chuan Zhao, Xinchen Zhuang, and Xihui Liang. Deep learning-based methods in structural reliability analysis: a review. Measurement Science and Technology , 2023

    work page 2023

  8. [8]

    On hardware security bug code fixes by prompting large language models

    Baleegh Ahmad, Shailja Thakur, Benjamin Tan, Ramesh Karri, and Hammond Pearce. On hardware security bug code fixes by prompting large language models. IEEE Transactions on Information Forensics and Security, 2024

    2024

Show all 74 references
  1. [9]

    Redqueen: Fuzzing with input-to-state correspon- dence

    Cornelius Aschermann, Sergej Schumilo, Tim Blazytko, Robert Gawlik, and Thorsten Holz. Redqueen: Fuzzing with input-to-state correspon- dence. In NDSS, volume 19, pages 1–15, 2019

    2019

  2. [10]

    Program synthesis with large language models

    Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. Program synthesis with large language models. arXiv preprint arXiv:2108.07732, 2021

    2021 arXiv

  3. [11]

    Deep learning and medical diag- nosis: A review of literature

    Mihalj Bakator and Dragica Radosav. Deep learning and medical diag- nosis: A review of literature. Multimodal Technologies and Interaction , 2(3):47, 2018

    2018

  4. [12]

    Directed greybox fuzzing

    Marcel B ¨ohme, Van-Thuan Pham, Manh-Dung Nguyen, and Abhik Roychoudhury. Directed greybox fuzzing. In Proceedings of the 2017 ACM SIGSAC conference on computer and communications security , pages 2329–2344, 2017

    2017

  5. [13]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems , 33:1877–1901, 2020

    1901

  6. [14]

    {IvySyn}: Automated vulnerability discovery in deep learning frameworks

    Neophytos Christou, Di Jin, Vaggelis Atlidakis, Baishakhi Ray, and Vasileios P Kemerlis. {IvySyn}: Automated vulnerability discovery in deep learning frameworks. In 32nd USENIX Security Symposium (USENIX Security 23) , pages 2383–2400, 2023

    2023

  7. [15]

    Large language models are zero-shot fuzzers: Fuzzing deep-learning libraries via large language models

    Yinlin Deng, Chunqiu Steven Xia, Haoran Peng, Chenyuan Yang, and Lingming Zhang. Large language models are zero-shot fuzzers: Fuzzing deep-learning libraries via large language models. In Proceedings of the 32nd ACM SIGSOFT international symposium on software testing and analy...

    2023

  8. [16]

    Large language models are edge-case generators: Crafting unusual programs for fuzzing deep learning libraries

    Yinlin Deng, Chunqiu Steven Xia, Chenyuan Yang, Shizhuo Dylan Zhang, Shujing Yang, and Lingming Zhang. Large language models are edge-case generators: Crafting unusual programs for fuzzing deep learning libraries. In Proceedings of the 46th IEEE/ACM International Conference on...

    2024

  9. [17]

    Fuzzing deep-learning libraries via automated relational api inference

    Yinlin Deng, Chenyuan Yang, Anjiang Wei, and Lingming Zhang. Fuzzing deep-learning libraries via automated relational api inference. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering , pages 4...

    2022

  10. [18]

    Differential testing of cross deep learning framework {APIs}: Revealing inconsistencies and vulnerabilities

    Zizhuang Deng, Guozhu Meng, Kai Chen, Tong Liu, Lu Xiang, and Chunyang Chen. Differential testing of cross deep learning framework {APIs}: Revealing inconsistencies and vulnerabilities. In 32nd USENIX Security Symposium (USENIX Security 23) , pages 7393–7410, 2023

    2023

  11. [19]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 , 2018

    2018 arXiv

  12. [20]

    Large language models of code fail at completing code with potential bugs

    Tuan Dinh, Jinman Zhao, Samson Tan, Renato Negrinho, Leonard Lausen, Sheng Zha, and George Karypis. Large language models of code fail at completing code with potential bugs. Advances in Neural Information Processing Systems, 36, 2024

    2024

  13. [21]

    Codebert: A pre-trained model for programming and natural languages

    Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, et al. Codebert: A pre-trained model for programming and natural languages. arXiv preprint arXiv:2002.08155, 2020. 11

    2002 arXiv

  14. [22]

    {AFL++}: Combining incremental steps of fuzzing research

    Andrea Fioraldi, Dominik Maier, Heiko Eißfeldt, and Marc Heuse. {AFL++}: Combining incremental steps of fuzzing research. In 14th USENIX Workshop on Offensive Technologies (WOOT 20) , 2020

    2020

  15. [23]

    Libafl: A framework to build modular and reusable fuzzers

    Andrea Fioraldi, Dominik Christian Maier, Dongjia Zhang, and Davide Balzarotti. Libafl: A framework to build modular and reusable fuzzers. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, pages 1051–1065, 2022

    2022

  16. [24]

    Grammar-based whitebox fuzzing

    Patrice Godefroid, Adam Kiezun, and Michael Y Levin. Grammar-based whitebox fuzzing. In Proceedings of the 29th ACM SIGPLAN conference on programming language design and implementation , pages 206–215, 2008

    2008

  17. [25]

    Learn&fuzz: Machine learning for input fuzzing

    Patrice Godefroid, Hila Peleg, and Rishabh Singh. Learn&fuzz: Machine learning for input fuzzing. In 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE) , pages 50–59. IEEE, 2017

    2017

  18. [26]

    A survey of deep learning techniques for autonomous driving

    Sorin Grigorescu, Bogdan Trasnea, Tiberiu Cocias, and Gigel Macesanu. A survey of deep learning techniques for autonomous driving. Journal of field robotics , 37(3):362–386, 2020

    2020

  19. [27]

    Muffin: Testing deep learning libraries via neural architecture fuzzing

    Jiazhen Gu, Xuchuan Luo, Yangfan Zhou, and Xin Wang. Muffin: Testing deep learning libraries via neural architecture fuzzing. In Proceedings of the 44th International Conference on Software Engineering , pages 1418–1430, 2022

    2022

  20. [28]

    Longcoder: A long-range pre-trained language model for code completion

    Daya Guo, Canwen Xu, Nan Duan, Jian Yin, and Julian McAuley. Longcoder: A long-range pre-trained language model for code completion. In International Conference on Machine Learning , pages 12098–12107. PMLR, 2023

    2023

  21. [29]

    Audee: Automated testing for deep learning frameworks

    Qianyu Guo, Xiaofei Xie, Yi Li, Xiaoyu Zhang, Yang Liu, Xiaohong Li, and Chao Shen. Audee: Automated testing for deep learning frameworks. In Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering , pages 486–498, 2020

    2020

  22. [30]

    Deep learning-based pedestrian detection in autonomous vehicles: Substantial issues and challenges

    Sundas Iftikhar, Zuping Zhang, Muhammad Asim, Ammar Muthanna, Andrey Koucheryavy, and Ahmed A Abd El-Latif. Deep learning-based pedestrian detection in autonomous vehicles: Substantial issues and challenges. Electronics, 11(21):3551, 2022

    2022

  23. [31]

    Jigsaw: Large language models meet program synthesis

    Naman Jain, Skanda Vaidyanath, Arun Iyer, Nagarajan Natarajan, Suresh Parthasarathy, Sriram Rajamani, and Rahul Sharma. Jigsaw: Large language models meet program synthesis. In Proceedings of the 44th International Conference on Software Engineering , pages 1219–1231, 2022

    2022

  24. [32]

    A context-based automated approach for method name consistency checking and suggestion

    Yi Li, Shaohua Wang, and Tien Nguyen. A context-based automated approach for method name consistency checking and suggestion. In 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE), pages 574–586. IEEE, 2021

    2021

  25. [33]

    Vulnerability detection with fine-grained interpretations

    Yi Li, Shaohua Wang, and Tien N Nguyen. Vulnerability detection with fine-grained interpretations. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering , pages 292–303, 2021

    2021

  26. [34]

    Utango: untangling commits with context-aware, graph-based, code change clustering learning model

    Yi Li, Shaohua Wang, and Tien N Nguyen. Utango: untangling commits with context-aware, graph-based, code change clustering learning model. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering , p...

    2022

  27. [35]

    Contextuality of code repre- sentation learning

    Yi Li, Shaohua Wang, and Tien N Nguyen. Contextuality of code repre- sentation learning. In 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE) , pages 548–559. IEEE, 2023

    2023

  28. [36]

    Commit-level, neural vulnerability detection and assessment

    Yi Li, Aashish Yadavally, Jiaxing Zhang, Shaohua Wang, and Tien N Nguyen. Commit-level, neural vulnerability detection and assessment. In Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering , pages...

    2023

  29. [37]

    Documentation-guided fuzzing for testing deep learning api functions

    Yitong Li. Documentation-guided fuzzing for testing deep learning api functions. Master’s thesis, University of Waterloo, 2020

    2020

  30. [38]

    Guiding enumer- ative program synthesis with large language models

    Yixuan Li, Julian Parsert, and Elizabeth Polgreen. Guiding enumer- ative program synthesis with large language models. arXiv preprint arXiv:2403.03997, 2024

    2024 arXiv

  31. [39]

    Multi-task learning based pre-trained language model for code completion

    Fang Liu, Ge Li, Yunfei Zhao, and Zhi Jin. Multi-task learning based pre-trained language model for code completion. In Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering, pages 473–485, 2020

    2020

  32. [40]

    Nnsmith: Generating diverse and valid test cases for deep learning compilers

    Jiawei Liu, Jinkun Lin, Fabian Ruffy, Cheng Tan, Jinyang Li, Aurojit Panda, and Lingming Zhang. Nnsmith: Generating diverse and valid test cases for deep learning compilers. In Proceedings of the 28th ACM International Conference on Architectural Support for Programming Langua...

    2023

  33. [41]

    Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation

    Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation. Advances in Neural Information Processing Systems , 36, 2024

    2024

  34. [42]

    Computing systems for autonomous driving: State of the art and challenges

    Liangkai Liu, Sidi Lu, Ren Zhong, Baofu Wu, Yongtao Yao, Qingyang Zhang, and Weisong Shi. Computing systems for autonomous driving: State of the art and challenges. IEEE Internet of Things Journal , 8(8):6469–6486, 2020

    2020

  35. [43]

    Ems: History-driven mutation for coverage-based fuzzing

    Chenyang Lyu, Shouling Ji, Xuhong Zhang, Hong Liang, Binbin Zhao, Kangjie Lu, and Raheem Beyah. Ems: History-driven mutation for coverage-based fuzzing. In NDSS, 2022

    2022

  36. [44]

    Deep learning for safe autonomous driving: Current challenges and future directions

    Khan Muhammad, Amin Ullah, Jaime Lloret, Javier Del Ser, and Victor Hugo C de Albuquerque. Deep learning for safe autonomous driving: Current challenges and future directions. IEEE Transactions on Intelligent Transportation Systems, 22(7):4316–4336, 2020

    2020

  37. [45]

    Combining program analysis and statistical language model for code statement completion

    Son Nguyen, Tien Nguyen, Yi Li, and Shaohua Wang. Combining program analysis and statistical language model for code statement completion. In 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE) , pages 710–721. IEEE, 2019

    2019

  38. [46]

    Examining zero-shot vulnerability repair with large language models

    Hammond Pearce, Benjamin Tan, Baleegh Ahmad, Ramesh Karri, and Brendan Dolan-Gavitt. Examining zero-shot vulnerability repair with large language models. In 2023 IEEE Symposium on Security and Privacy (SP), pages 2339–2356. IEEE, 2023

    2023

  39. [47]

    T-fuzz: fuzzing by program transformation

    Hui Peng, Yan Shoshitaishvili, and Mathias Payer. T-fuzz: fuzzing by program transformation. In 2018 IEEE Symposium on Security and Privacy (SP), pages 697–710. IEEE, 2018

    2018

  40. [48]

    Cradle: cross-backend validation to detect and localize bugs in deep learning libraries

    Hung Viet Pham, Thibaud Lutellier, Weizhen Qi, and Lin Tan. Cradle: cross-backend validation to detect and localize bugs in deep learning libraries. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE), pages 1027–1038. IEEE, 2019

    2019

  41. [49]

    Smart greybox fuzzing

    Van-Thuan Pham, Marcel B ¨ohme, Andrew E Santosa, Alexandru R ˘azvan C˘aciulescu, and Abhik Roychoudhury. Smart greybox fuzzing. IEEE Transactions on Software Engineering , 47(9):1980–1997, 2019

    1980

  42. [50]

    On extractive and abstractive neural document summarization with transformer language models

    Jonathan Pilault, Raymond Li, Sandeep Subramanian, and Christopher Pal. On extractive and abstractive neural document summarization with transformer language models. In Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP) , pages 9308–...

    2020

  43. [51]

    Deep neural network perception models and robust autonomous driving systems: practical solutions for mitigation and improvement

    Mohammad Javad Shafiee, Ahmadreza Jeddi, Amir Nazemi, Paul Fieguth, and Alexander Wong. Deep neural network perception models and robust autonomous driving systems: practical solutions for mitigation and improvement. IEEE Signal Processing Magazine, 38(1):22–30, 2020

    2020

  44. [52]

    Deep learning in medical image analysis

    Dinggang Shen, Guorong Wu, and Heung-Il Suk. Deep learning in medical image analysis. Annual review of biomedical engineering, 19:221– 248, 2017

    2017

  45. [53]

    Choose your programming copilot: a comparison of the program synthesis performance of github copilot and genetic programming

    Dominik Sobania, Martin Briesch, and Franz Rothlauf. Choose your programming copilot: a comparison of the program synthesis performance of github copilot and genetic programming. In Proceedings of the genetic and evolutionary computation conference , pages 1019–1027, 2022

    2022

  46. [54]

    Evaluating large language models on medical evidence summa- rization

    Liyan Tang, Zhaoyi Sun, Betina Idnay, Jordan G Nestor, Ali Soroush, Pierre A Elias, Ziyang Xu, Ying Ding, Greg Durrett, Justin F Rousseau, et al. Evaluating large language models on medical evidence summa- rization. npj Digital Medicine , 6(1):158, 2023

    2023

  47. [55]

    Verifying data constraint equivalence in fintech systems

    Chengpeng Wang, Gang Fan, Peisen Yao, Fuxiong Pan, and Charles Zhang. Verifying data constraint equivalence in fintech systems. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE), pages 1329–1341. IEEE, 2023

    2023

  48. [56]

    Eagle: Creating equivalent graphs to test deep learning libraries

    Jiannan Wang, Thibaud Lutellier, Shangshu Qian, Hung Viet Pham, and Lin Tan. Eagle: Creating equivalent graphs to test deep learning libraries. In Proceedings of the 44th International Conference on Software Engineering, pages 798–810, 2022

    2022

  49. [57]

    Deepvd: Toward class-separation features for neural network vulnerability detection

    Wenbo Wang, Tien N Nguyen, Shaohua Wang, Yi Li, Jiyuan Zhang, and Aashish Yadavally. Deepvd: Toward class-separation features for neural network vulnerability detection. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) , pages 2249–2261. IEEE, 2023. 12

    2023

  50. [58]

    Natural is the best: Model-agnostic code simplification for pre-trained large language models

    Yan Wang, Xiaoning Li, Tien N Nguyen, Shaohua Wang, Chao Ni, and Ling Ding. Natural is the best: Model-agnostic code simplification for pre-trained large language models. Proceedings of the ACM on Software Engineering, 1(FSE):586–608, 2024

    2024

  51. [59]

    Deep learning library testing via effective model generation

    Zan Wang, Ming Yan, Junjie Chen, Shuang Liu, and Dongdi Zhang. Deep learning library testing via effective model generation. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering , page...

    2020

  52. [60]

    Free lunch for testing: Fuzzing deep-learning libraries from open source

    Anjiang Wei, Yinlin Deng, Chenyuan Yang, and Lingming Zhang. Free lunch for testing: Fuzzing deep-learning libraries from open source. In Proceedings of the 44th International Conference on Software Engineering, pages 995–1007, 2022

    2022

  53. [61]

    Copiloting the copilots: Fusing large language models with completion engines for automated program repair

    Yuxiang Wei, Chunqiu Steven Xia, and Lingming Zhang. Copiloting the copilots: Fusing large language models with completion engines for automated program repair. In Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of So...

    2023

  54. [62]

    When less is enough: Positive and unlabeled learning model for vulnerability detection

    Xin-Cheng Wen, Xinchen Wang, Cuiyun Gao, Shaohua Wang, Yang Liu, and Zhaoquan Gu. When less is enough: Positive and unlabeled learning model for vulnerability detection. In 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE) , pages 345–357. IEEE, 2023

    2023

  55. [63]

    Using github copilot to solve simple programming problems

    Michel Wermelinger. Using github copilot to solve simple programming problems. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V . 1, pages 172–178, 2023

    2023

  56. [64]

    Docter: Documentation-guided fuzzing for testing deep learning api functions

    Danning Xie, Yitong Li, Mijung Kim, Hung Viet Pham, Lin Tan, Xiangyu Zhang, and Michael W Godfrey. Docter: Documentation-guided fuzzing for testing deep learning api functions. In Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis , pa...

    2022

  57. [65]

    A systematic evaluation of large language models of code

    Frank F Xu, Uri Alon, Graham Neubig, and Vincent Josua Hellendoorn. A systematic evaluation of large language models of code. In Proceedings of the 6th ACM SIGPLAN International Symposium on Machine Programming, pages 1–10, 2022

    2022

  58. [66]

    Selfpico: Self-guided partial code execution with llms

    Zhipeng Xue, Zhipeng Gao, Shaohua Wang, Xing Hu, Xin Xia, and Shanping Li. Selfpico: Self-guided partial code execution with llms. arXiv preprint arXiv:2407.16974 , 2024

    2024 arXiv

  59. [67]

    A learning-based approach to static program slicing

    Aashish Yadavally, Yi Li, Shaohua Wang, and Tien N Nguyen. A learning-based approach to static program slicing. Proceedings of the ACM on Programming Languages , 8(OOPSLA1):83–109, 2024

    2024

  60. [68]

    (partial) program dependence learning

    Aashish Yadavally, Tien N Nguyen, Wenbo Wang, and Shaohua Wang. (partial) program dependence learning. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) , pages 2501–

    2023

  61. [69]

    Fuzzing automatic differentiation in deep-learning libraries

    Chenyuan Yang, Yinlin Deng, Jiayi Yao, Yuxing Tu, Hanchi Li, and Lingming Zhang. Fuzzing automatic differentiation in deep-learning libraries. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE), pages 1174–1186. IEEE, 2023

    2023

  62. [70]

    Thinkrepair: Self-directed automated program repair

    Xin Yin, Chao Ni, Shaohua Wang, Zhenhao Li, Limin Zeng, and Xiaohu Yang. Thinkrepair: Self-directed automated program repair. arXiv preprint arXiv:2407.20898, 2024

    2024 arXiv

  63. [71]

    American fuzzy lop, 2017

    Michal Zalewski. American fuzzy lop, 2017

    2017

  64. [72]

    Planning with large language models for code generation

    Shun Zhang, Zhenfang Chen, Yikang Shen, Mingyu Ding, Joshua B Tenenbaum, and Chuang Gan. Planning with large language models for code generation. arXiv preprint arXiv:2303.05510 , 2023

    2023 arXiv

  65. [73]

    Benchmarking large language models for news summarization

    Tianyi Zhang, Faisal Ladhak, Esin Durmus, Percy Liang, Kathleen McKeown, and Tatsunori B Hashimoto. Benchmarking large language models for news summarization. Transactions of the Association for Computational Linguistics, 12:39–57, 2024

    2024

  66. [74]

    Fuzzing: a survey for roadmap

    Xiaogang Zhu, Sheng Wen, Seyit Camtepe, and Yang Xiang. Fuzzing: a survey for roadmap. ACM Computing Surveys (CSUR) , 54(11s):1–36, 2022. 13

    2022

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.