Pith. sign in

REVIEW 4 major objections 7 minor 2 cited by

ELFuzz: Efficient Input Generation via LLM-driven Synthesis Over Fuzzer Space

T0 review · 4 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read ELFuzz claims that hand-written input grammars can be replaced by an automated LLM evolution loop whose synthesized fuzzers give AFL++ up to 434.8% more edge coverage and trigger up to 216.7% more injected bugs than state-of-the-art…

desk verdict Genuinely new fuzzer-space-guided LLM evolution with a careful evaluation; the main caveat is that 'automated' remains untested for formats the LLM does not already know. read the letter →

arxiv 2506.10323 v5 pith:QRUKMZ3J submitted 2025-06-12 cs.CR cs.SE

Chuyang Chen , Brendan Dolan-Gavitt , Zhiqiang Lin This is my paper
classification cs.CRcs.SE
keywords generation-basedfuzzinglargelanguagemodelsfuzzerspacecoverage-guidedevolutiongrammarsynthesisseedgenerationLLM-drivenmutationvulnerabilitydiscovery
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Generation-based fuzzing works best when test inputs follow the target's grammar and semantic constraints, but those specifications are usually hand-written, expensive, and bespoke. ELFuzz claims to automate this step: starting from a naive random-text generator, an LLM mutates the generator's Python code over many iterations, and a 'fuzzer space' lattice built from set inclusion of covered code edges decides which mutants survive. The final generators are then used to seed AFL++, and on seven benchmarks they outperform grammar synthesizers and hand-written ANTLR4 grammars, with up to 434.8% more edge coverage and 216.7% more triggered injected bugs. A 14-day campaign on cvc5 found five new bugs, three of them exploitable. If true, ELFuzz removes the manual grammar-construction step entirely, replacing it with an automated loop that costs only GPU time.

What carries the argument

The central object is the fuzzer space, a lattice whose elements are candidate fuzzers ordered by the subset relation on their cover sets, i.e., the exact range of code they cover when run for a finite time. The top element is a fuzzer that covers all code, the bottom the weakest, and climbing the lattice means improving fuzzing effectiveness. The evolution loop uses three LLM-driven mutators, splicing (gluing the head of one fuzzer to the tail of another with LLM-generated glue code), completion (truncating and letting the LLM continue), and infilling (removing lines and letting the LLM rewrite them), to generate mutants, then augments the explored part of the lattice by comparing cover sets set-theoretically. Coverage is approximated by feeding 1,000 generated inputs into the SUT and recording covered edges. This lattice-based comparison distinguishes fuzzers that a single coverage value would wrongly treat as equal, and it is the component whose removal hurts performance most in the ablation.

What would settle it

Apply ELFuzz to a SUT with an input format that is absent from any public code or documentation (e.g., a newly invented or proprietary text format), run the 50-iteration evolution, and compare the edge coverage of the synthesized fuzzers' seeds against the naive random seed fuzzer. If coverage does not climb meaningfully above the random baseline, the central claim fails for such SUTs.

Watch

Extended reading notes

Core claim

The paper's central claim is that the task of synthesizing a generation-based fuzzer can be decomposed into small LLM-driven code mutations guided by a lattice it calls the fuzzer space. Each candidate fuzzer is characterized by its cover set, the set of code edges it reaches when run for a limited time, and fuzzer strength is the subset relation between cover sets. Fuzzers with the same cover set are equivalent; a proper superset means strictly stronger; incomparable cover sets mean the fuzzers test different parts of the SUT and cannot substitute for one another. The evolution loop keeps only mutants that are stronger than or incomparable to the current seeds and selects a fixed-size elite that maximizes the union of cover sets. The paper reports that this loop, driven by a local 13B CodeLlama model, synthesizes fuzzers whose seeds outperform grammar-based generators and even hand-written ANTLR4 grammars, and that the fuzzer space guidance contributes up to 62.5% of the effectiveness according to the ablation study.

Load-bearing premise

The evolution loop only works if the LLM's training data already contains enough knowledge of the target input format to make its mutations useful; for obscure, proprietary, or binary formats, the loop has no way to rediscover the grammar and would degrade to random code mutation.

Editorial extensions

If this is right

  • Replacing hand-written grammars with the synthesized fuzzers removes the main manual cost of generation-based fuzzing: the paper reports ELFuzz's core logic is 3,602 lines of Python versus 80k lines for Csmith and 10-13k lines for grammar and constraint synthesizers.
  • The synthesized fuzzers skip grammar-rule instantiation at runtime, so seed generation avoids the overhead of grammar frameworks.
  • The fuzzers are human-readable Python functions, so they can be inspected and extended; the paper demonstrates adaptation to ZEST in five person-days.
  • Fuzzer space guidance matters more than any single mutator: removing it degrades coverage by up to 62.5%, while removing the splicing mutator costs up to 26.2%.
  • The 14-day cvc5 campaign found five new bugs, three reported as exploitable, including a format string injection and two denial-of-service dead loops.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the coverage gains persist on a wider set of SUTs, the economic trade-off for practitioners shifts: synthesis costs tens of hours of GPU time once per SUT, while grammar maintenance is a recurring human cost, so the approach becomes attractive whenever a SUT is fuzzed repeatedly or long-term.
  • The fuzzer space idea is not tied to LLMs: any program-mutation engine could be guided by the same cover-set lattice, so the lattice comparison could be reused by non-LLM evolutionary fuzzing pipelines.
  • The paper's own Section 7 suggests extending evolution to the entire fuzzing cycle; a testable extension would be to keep evolving the input generator during AFL++ runs using runtime feedback, which could maintain corpus diversity beyond the initial seed advantage and possibly close the leveling-off seen on libxml2 and CPython.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper introduces ELFuzz, an LLM-driven evolutionary approach that synthesizes Python input generators (fuzzers) for a system under test. Starting from a naive random-text seed fuzzer, ELFuzz repeatedly mutates candidate fuzzers using CodeLlama-13B via splicing, completion, and infilling, and selects survivors using a poset of approximated edge-cover sets ('fuzzer space') with a max-cover greedy selection. The synthesized fuzzers generate seeds for AFL++. The evaluation covers seven well-known text-format SUTs, compares with Grammarinator+ANTLR grammars, ISLa, ISLearn, and GLADE, and reports up to 434.8% more edge coverage in a 10-minute generation phase, up to 216.7% more injected bugs in 24-hour AFL++ campaigns, and a 14-day cvc5 campaign that found five new bugs. An ablation attributes up to 62.5% of the effect to the fuzzer-space model.

Significance. If the headline results hold, ELFuzz is a useful step toward reducing the manual effort of building generation-based fuzzers: it produces human-readable Python generators, is open-sourced with a Docker artifact, and the controlled experiments use 10 repetitions with standard deviations and FIXREVERTER-injected bugs. The real cvc5 campaign is a valuable existence proof. However, the central claim of fully automated synthesis is materially weakened by the paper's own acknowledgement that LLM training knowledge of the input format is necessary, and the evaluation does not test this boundary. The abstract's characterization of the cvc5 findings is stronger than the evidence in the body. These issues are fixable by scoping claims and adding targeted experiments, so the contribution remains publishable after major revision.

major comments (4)
  1. [§5, §7, Appx. A] The central 'fully automated' claim is not supported as stated. Section 7 explicitly says 'Knowledge about the SUTs in the training set is necessary for the LLM to guess the input format during the synthesis of the fuzzers' and acknowledges that uncommon or binary formats may break the approach. All seven benchmarks (JSON, XML, regex, SQL, Python, SMT-LIB, SVG) are standard text formats that are abundant in CodeLlama-13B's training data, so the evaluation cannot distinguish grammar recall from grammar discovery. The pipeline also involves per-SUT manual input beyond the seed fuzzer: Section 5 says the prompt is prepended with a 'simple hint for the format' (e.g., XML documents start with <?xml...?>), Appendix A shows a seed template with a FORMAT placeholder that 'provides clues', and Section 6.3 describes a hand-collected cvc5 dictionary from regression tests. This is inconsistent with Section 7's statement that the same random-byte seed fuzzer is used for all SUTs. Please either evaluate a SUT whose format is rare or absent in LLM training data, or explicitly scope the contribution to formats already known to the LLM.
  2. [Abstract, §6.4.2, Appx. C] The abstract and introduction say the cvc5 campaign found 'five 0-day bugs (three are exploitable)', but the body is more cautious. Section 6.4.2 states that two of the five bugs involve invalid pointers that 'may possibly be exploited for control flow hijacking (though we have not confirmed this yet)', and Appendix C labels the bugs 'possibly exploitable'. The format string bug is described as 'possibly causing attacks that execute arbitrary commands', and two bugs are denial-of-service dead loops. Two of the five bugs had already been fixed before disclosure. The evidence supports 'five new bugs, three reported to developers, with potential security impact', not 'three exploitable 0-day bugs'. Please align the headline wording with the evidence.
  3. [§3, §4.2, §6.3] The fuzzer-space mechanism relies on strict subset comparisons between approximated cover sets, but no robustness analysis is provided. Definition 1 in Section 3 defines strength via exact cover-set inclusion, while Section 6.3 approximates each cover set by feeding only 1,000 randomly generated inputs to the SUT. Because the candidate fuzzers are stochastic, repeated approximations will differ, so noisy cover sets can produce spurious dominance relations or cause useful mutants to be discarded in Algorithm 1. Please report a sensitivity analysis (e.g., varying the 1,000-input approximation budget, or averaging over repeated approximation runs) to show the evolution guidance is stable.
  4. [§6.2, Figures 7 and 8] The RQ1 headline coverage numbers are not supported with variance information for the generation phase. Section 6.2 says the 24-hour AFL++ phase is repeated 10 times, but does not say the initial 10-minute generation was repeated; Figure 7 has no error bars. Given the stochasticity of all five generators, the 434.8% claim may be based on a single run. In addition, Figure 8 has no mutation-only or random-seed baseline, so the seed-boost effect of ELFuzz is only measured against grammar-based generators and not isolated from the value of providing any valid seeds. Please add repetition for the generation phase and a no-grammar control.
minor comments (7)
  1. [Abstract and §1] The number '216.7.0%' should read '216.7%'.
  2. [Listings 1 and 4] There are typos in the running example: 'parans' should be 'parens' and 'wich' should be 'which'.
  3. [Table 6] In the SQLite column of the GLADE row, the 25% time (562 min) is greater than the 50% time (455 min), which is non-monotonic; please verify these values.
  4. [§3 and Figure 1] The term 'lattice' is used loosely: the explored set of six fuzzers is a subposet of the powerset lattice and is not shown to be closed under meet and join; consider defining the ambient lattice or using 'poset'.
  5. [Algorithm 2] The stated O(T M^2) time complexity is not derived from the nested while/substitution loops; please provide the derivation or correct the bound.
  6. [§6.1] Fuzz4All is listed in Table 2 but is not used as a baseline; since Fuzz4All is LLM-based and not domain-specific, a sentence justifying its exclusion or a direct comparison would strengthen the evaluation.
  7. [§6.3 and Table 5] ELFuzz synthesis takes 19.9–55.5 hours per SUT, while all head-to-head fuzzing comparisons exclude synthesis time; please add a total-budget analysis or explicitly state the amortization assumption.

Circularity Check

0 steps flagged · score 0.0 of 10

No material circularity: ELFuzz's claims are empirical comparisons and an external bug-finding campaign, not derivations that reduce to their inputs.

full rationale

The paper's load-bearing claims are empirical: ELFuzz synthesizes fuzzers via coverage-guided LLM evolution and is then compared against grammar-based baselines on coverage, injected-bug triggering, and a real cvc5 campaign. The coverage-guided loop and the coverage-based evaluation use the same metric, but this is objective alignment, not a constructed equivalence: the evaluation is a head-to-head comparison against baselines that do not optimize that objective, and the five real cvc5 bugs are external evidence independent of the selection criterion. The Section 7 data-contamination passage explicitly states that LLM training knowledge of the SUT input format is necessary and even embraces contamination as the mechanism being leveraged; that is an acknowledged limitation and boundary of the approach, not a circular step. No result is defined in terms of another result, no fitted parameter is relabeled as a prediction, and no load-bearing premise is justified by a self-citation. The fuzzer space concept formalizes coverage-set inclusion as a lattice, but it is used as a selection mechanism and analyzed via ablation rather than being presented as a derived prediction. Under the specified criteria, no circular step can be exhibited.

Assumptions & free parameters 5 free parameters · 5 assumptions · 1 invented entities

The central claim rests on the fidelity of approximated cover sets, on the LLM's prior knowledge of SUT input formats, and on the adequacy of edge coverage as a guiding signal, plus a standard greedy set-cover approximation. The only conceptual invention is the fuzzer space, validated internally. The hand-set experimental parameters (iterations, mutants, survivors, sample size, cvc5 dictionary) shape the reported numbers but were not fitted to the evaluation data.

free parameters (5)
  • evolution iterations = 50
    Hand-chosen synthesis budget; longer evolution would likely improve the fuzzers, so absolute coverage numbers are tied to this setting (§6.3).
  • mutants per iteration = 200
    Hand-chosen population size for the LLM mutation step (§6.3).
  • elite survivor count = 10
    Hand-chosen N for max-cover selection and for the top-k ablation in RQ3 (§4.3, §6.2).
  • cover-set approximation sample count = 1,000
    Each candidate fuzzer's cover set is approximated from 1,000 generated inputs (§6.3); the entire fuzzer space lattice depends on this number.
  • hand-collected cvc5 dictionary = tokens mined from cvc5 regression tests
    Used as the AFL++ dictionary for cvc5 because no pre-built dictionary exists (§6.3); this hand-built resource assists the real-world bug campaign.
assumptions (5)
  • domain assumption Cover sets approximated by generating 1,000 inputs per fuzzer faithfully represent fuzzer strength and the partial order in the fuzzer space.
    Algorithm 1 and §6.3 use a 1,000-input approximation for all cover sets; every survivor selection and every 'stronger than' edge in the lattice depends on this approximation being reliable.
  • domain assumption The LLM has sufficient prior knowledge of each SUT's input format to make useful mutations from a random-text seed fuzzer.
    Stated in §7 as necessary for guessing input formats; the paper notes that uncommon SUTs and binary formats lack this knowledge and would break the synthesis.
  • domain assumption Edge coverage is an adequate proxy for fuzzing effectiveness and for the strength ordering of fuzzers.
    All cover sets, selection decisions, and the RQ1 and RQ2 evaluations use edge coverage or bug counts; the lattice results inherit the limitations of the coverage metric.
  • standard math The greedy set-cover approximation in Algorithm 2 provides near-optimal elite selection.
    Elite selection is the NP-complete max-coverage problem; the paper uses repeated greedy local search and relies on standard approximation behavior.
  • domain assumption Minimizing seeds with afl-cmin and running identical AFL++ campaigns does not bias the comparison across baseline seeds of different sizes and structures.
    RQ1 and RQ2 feed minimized seeds from each fuzzer into the same AFL++ pipeline; corpus composition differences are not controlled beyond minimization.
invented entities (1)
  • fuzzer space (lattice ordered by cover-set inclusion)
    purpose: Formal model for comparing candidate fuzzers and guiding the evolution loop toward stronger generators.
    Introduced in §3 and used throughout §4. Its contribution is measured only by the internal ablation (RQ3, ELFuzz-noFS); there is no external benchmark or falsifiable prediction that isolates the lattice mechanism from coverage guidance in general.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ELFuzz: Efficient Input Generation via LLM-driven Synthesis Over Fuzzer Space." pith.science (2026). https://pith.science/paper/QRUKMZ3J

@misc{pith2026250610323,
  author       = {Pith},
  title        = {Pith review of: ELFuzz: Efficient Input Generation via LLM-driven Synthesis Over Fuzzer Space},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QRUKMZ3J}},
  note         = {Machine review of arXiv:2506.10323}
}
read the original abstract

Generation-based fuzzing produces appropriate test cases according to specifications of input grammars and semantic constraints to test systems and software. However, these specifications require significant manual effort to construct. This paper proposes a new approach, ELFuzz (Evolution Through Large Language Models for Fuzzing), that automatically synthesizes generation-based fuzzers tailored to a system under test (SUT) via LLM-driven synthesis over fuzzer space. At a high level, it starts with minimal seed fuzzers and propels the synthesis by fully automated LLM-driven evolution with coverage guidance. Compared to previous approaches, ELFuzz can 1) seamlessly scale to SUTs of real-world sizes -- up to 1,791,104 lines of code in our evaluation -- and 2) synthesize efficient fuzzers that catch interesting grammatical structures and semantic constraints in a human-understandable way. Our evaluation compared ELFuzz with specifications manually written by domain experts and synthesized by state-of-the-art approaches. It shows that ELFuzz achieves up to 434.8% more coverage over the second best and triggers up to 216.7% more artificially injected bugs, compared to the state-of-the-art. We also used ELFuzz to conduct a real-world fuzzing campaign on the newest version of cvc5 for 14 days, and encouragingly, it found five 0-day bugs (three are exploitable). Moreover, we conducted an ablation study, which shows that the fuzzer space model, the key component of ELFuzz, contributes the most (up to 62.5%) to the effectiveness of ELFuzz. Further analysis of the fuzzers synthesized by ELFuzz confirms that they catch interesting grammatical structures and semantic constraints in a human-understandable way. The results present the promising potential of ELFuzz for more automated, efficient, and extensible input generation for fuzzing.

Figures

Figures reproduced from arXiv: 2506.10323 by the authors.

Figure 1
Figure 1. An illustration of Listing 1’s fuzzer space LLM-driven synthesis over fuzzer space LLM-driven mutation Fuzzer space exploration Naive fuzzer Low coverage High coverage Fuzzer space Low coverage High coverage Fuzzer space Low coverage High coverage Fuzzer space ⋯ Fuzz The SUT Superior Fuzzer [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. illustrates this overall process. Specifically, every iteration of the evolution loop com￾prises three steps as shown by [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. An evolution iteration of ELFUZZ (a) Splicing (b) Completion (c) Infilling tail head tail head tail head middle tail head LLM code head tail LLM code head LLM code head tail [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: LLM-driven mutators ing 6) are the seed fuzzers. The splicing mutator is applied to glue the code before line 7 of FB and the code after line 5 of FD. The completion mutator is used to continue writing FD af￾ter removing the code after line 8. The infilling mutator is …
Figure 6
Figure 6. Figure 6: Fill-in-the-middle query and splicing mutators are implemented via fill-in-the-middle queries [25] supported by the most recently updated LLMs. Fill-in-the-middle queries insert a special FIM token into the queried text, and the LLM will return possible expansions that…
Figure 5
Figure 5. Figure 5: Prompts for the LLM-driven mutation known “big models” such as GPT-4. However, even with such a small model, ELFUZZ has shown significant advantages over existing techniques. We attribute this to the evolution loop that decomposes the fuzzer synthesis task into small s…
Figure 7
Figure 7. Figure 7: Coverage of the inputs generated within 10min [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 8
Figure 8. Figure 8: Coverage trends during mutation-based fuzzing [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 9
Figure 9. Figure 9: Number of the triggered bugs during 24-hour [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]
Figure 10
Figure 10. Figure 10: Unique bugs that only one fuzzer can find [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]
Figure 11
Figure 11. Figure 11: Coverage of the inputs generated by the vari [PITH_FULL_IMAGE:figures/full_fig_p013_11.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Knowdit: Agentic Smart Contract Vulnerability Detection with Auditing Knowledge Summarization

    cs.CR 2026-03 conditional novelty 6.0 of 10

    Knowdit links abstract DeFi semantics to vulnerability patterns in a knowledge graph and drives an agentic specify–harness–fuzz–reflect loop that finds all high-severity and most medium-severity bugs on held-out Code4...

  2. Locus: Agentic Predicate Synthesis for Directed Fuzzing

    cs.CR 2025-08 conditional novelty 6.0 of 10

    Locus uses an LLM agent to synthesize and validate intermediate predicates that make directed fuzzing reach target bug states faster, reporting an average 41.6x speedup across eight fuzzers.

Reference graph

Works this paper leans on

79 extracted references · 52 canonical work pages · cited by 2 Pith papers

  1. [1]

    The AFL++ fuzzing framework

    “The AFL++ fuzzing framework.” [Online]. Available: https://aflplus.plus/

    work page

  2. [2]

    AFL trophy cases

    “AFL trophy cases.” [Online]. Available: https://lcamtuf. coredump.cx/afl/#bugs

    work page

  3. [3]

    antlr/grammars-v4: Grammars written for ANTLR v4

    “antlr/grammars-v4: Grammars written for ANTLR v4.” [Online]. Available: https://github.com/antlr/grammars-v4

    work page

  4. [4]

    clang.cindex — libclang 16.0.6 documentation

    “clang.cindex — libclang 16.0.6 documentation.” [Online]. Available: https://libclang.readthedocs.io/en/latest/_modules/ clang/cindex.html#CursorKind

    work page

  5. [5]

    Format string attack | OWASP foundation

    “Format string attack | OWASP foundation.” [On- line]. Available: https://owasp.org/www-community/attacks/ Format_string_attack

    work page

  6. [6]

    fuzzbench: FuzzBench - Fuzzer benchmarking as a service

    “fuzzbench: FuzzBench - Fuzzer benchmarking as a service.” [Online]. Available: https://github.com/google/fuzzbench/tree/ master

    work page

  7. [7]

    jsoncpp: A C++ library for interacting with JSON

    “jsoncpp: A C++ library for interacting with JSON.” [Online]. Available: https://github.com/open-source-parsers/jsoncpp

    work page

  8. [8]

    NVIDIA A40 GPU for visual computing

    “NVIDIA A40 GPU for visual computing.” [Online]. Available: https://www.nvidia.com/en-us/data-center/a40/

    work page

Show all 79 references
  1. [9]

    OSS-Fuzz

    “OSS-Fuzz.” [Online]. Available: https://google.github.io/ oss-fuzz/

  2. [10]

    system under test - ISTQB glossary

    “system under test - ISTQB glossary.” [Online]. Available: https://glossary.istqb.org/en_US/term/system-under-test

  3. [11]

    codellama/CodeLlama-13b-python-hf · [AUTOMATED] model memory requirements,

    “codellama/CodeLlama-13b-python-hf · [AUTOMATED] model memory requirements,” Nov. 2023. [On- line]. Available: https://huggingface.co/codellama/ CodeLlama-13b-Python-hf/discussions/7

    2023

  4. [12]

    cvc5/cvc5,

    “cvc5/cvc5,” Oct. 2024. [Online]. Available: https://github. com/cvc5/cvc5

    2024

  5. [13]

    GNOME / librsvg · GitLab,

    “GNOME / librsvg · GitLab,” Oct. 2024. [Online]. Available: https://gitlab.gnome.org/GNOME/librsvg

    2024

  6. [14]

    GNOME/libxml2,

    “GNOME/libxml2,” Oct. 2024. [Online]. Available: https: //github.com/GNOME/libxml2

    2024

  7. [15]

    google/fuzzing,

    “google/fuzzing,” Oct. 2024. [Online]. Available: https: //github.com/google/fuzzing

    2024

  8. [16]

    python/cpython,

    “python/cpython,” Oct. 2024. [Online]. Available: https: //github.com/python/cpython

    2024

  9. [17]

    Set cover problem,

    “Set cover problem,” Dec. 2024, page Version ID: 1264796057. [Online]. Available: https://en.wikipedia.org/w/index.php? title=Set_cover_problem&oldid=1264796057

    2024

  10. [18]

    Cover pages: XML and semantic trans- parency,

    “Cover pages: XML and semantic trans- parency,” Feb. 2025. [Online]. Avail- able: https://web.archive.org/web/20250203130605/http://xml. coverpages.org/xmlAndSemantics.html

    2025

  11. [19]

    google/re2,

    “google/re2,” May 2025, original-date: 2014-08- 18T21:21:26Z. [Online]. Available: https://github.com/google/ re2

    2025

  12. [20]

    sqlparser - crates.io: Rust Package Registry,

    “sqlparser - crates.io: Rust Package Registry,” May 2025. [Online]. Available: https://crates.io/crates/sqlparser

    2025

  13. [21]

    A review on grammar-based fuzzing techniques,

    H. Al Salem and J. Song, “A review on grammar-based fuzzing techniques,” International Journal of Computer Science & Se- curity (IJCSS), vol. 13, no. 3, pp. 114–123, 2019

    2019

  14. [22]

    Software engineering for machine learning: a case study,

    S. Amershi, A. Begel, C. Bird, R. DeLine, H. Gall, E. Kamar, N. Nagappan, B. Nushi, and T. Zimmermann, “Software engineering for machine learning: a case study,” in Proceedings of the 41st IEEE/ACM International Conference on Software Engineering: Software Engineering in Pract...

    2019

  15. [23]

    DY Fuzzing: Formal Dolev-Yao models meet cryptographic protocol fuzz testing,

    M. Ammann, L. Hirschi, and S. Kremer, “DY Fuzzing: Formal Dolev-Yao models meet cryptographic protocol fuzz testing,” in Proceedings of the 45th IEEE Symposium on Security and Privacy , ser. SP 2024. IEEE, May 2024, pp. 1481–1499, iSSN: 2375-1207. [Online]. Available: https://...

    2024

  16. [24]

    Synthesizing program input grammars,

    O. Bastani, R. Sharma, A. Aiken, and P. Liang, “Synthesizing program input grammars,” ACM SIGPLAN Notices, vol. 52, no. 6, pp. 95–110, Jun. 2017. [Online]. Available: https: //dl.acm.org/doi/10.1145/3140587.3062349 15

    2017

  17. [25]

    Efficient training of language models to fill in the middle,

    M. Bavarian, H. Jun, N. Tezak, J. Schulman, C. McLeavey, J. Tworek, and M. Chen, “Efficient training of language models to fill in the middle,” Jul. 2022, arXiv:2207.14255 [cs]. [Online]. Available: http://arxiv.org/abs/2207.14255

    2022 arXiv

  18. [26]

    ‘Synthesizing input grammars’: a replication study,

    B. Bendrissou, R. Gopinath, and A. Zeller, “‘Synthesizing input grammars’: a replication study,” in Proceedings of the 43rd ACM SIGPLAN International Conference on Programming Language Design and Implementation , ser. PLDI 2022. New York, NY , USA: Association for Computing Ma...

    2022

  19. [27]

    No grammar, no problem: Towards fuzzing the linux kernel without system-call descriptions,

    A. Bulekov, B. Das, S. Hajnoczi, and M. Egele, “No grammar, no problem: Towards fuzzing the linux kernel without system-call descriptions,” in Proceedings of 2023 Network and Distributed System Security Symposium , ser. NDSS 2023, San Diego, CA, USA, 2023. [Online]. Available:...

    2023

  20. [28]

    A systematic review of fuzzing techniques,

    C. Chen, B. Cui, J. Ma, R. Wu, J. Guo, and W. Liu, “A systematic review of fuzzing techniques,” Computers & Security , vol. 75, pp. 118–137, Jun. 2018. [Online]. Available: https://www.sciencedirect.com/science/article/pii/ S0167404818300658

    2018

  21. [29]

    SyzGen++: Dependency inference for augmenting kernel driver fuzzing,

    W. Chen, Y . Hao, Z. Zhang, X. Zou, D. Kirat, S. Mishra, D. Schales, J. Jang, and Z. Qian, “SyzGen++: Dependency inference for augmenting kernel driver fuzzing,” in Pro- ceedings of the 45th IEEE Symposium on Security and Privacy, ser. SP 2024. IEEE, May 2024. [Online]. Availa...

    2024

  22. [30]

    One engine to fuzz ’em all: Generic language processor testing with semantic validation,

    Y . Chen, R. Zhong, H. Hu, H. Zhang, Y . Yang, D. Wu, and W. Lee, “One engine to fuzz ’em all: Generic language processor testing with semantic validation,” in Proceedings of the 42nd IEEE Symposium on Security and Privacy, ser. SP 2021. San Diego, CA, USA: IEEE, May 2021, pp....

    2021

  23. [31]

    Large language models are zero-shot fuzzers: Fuzzing deep-learning libraries via large language models,

    Y . Deng, C. S. Xia, H. Peng, C. Yang, and L. Zhang, “Large language models are zero-shot fuzzers: Fuzzing deep-learning libraries via large language models,” in Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis, ser. ISSTA 2023. New ...

    2023

  24. [32]

    An experimental comparison of edge, edge-pair, and prime path criteria,

    V . H. S. Durelli, M. E. Delamaro, and J. Offutt, “An experimental comparison of edge, edge-pair, and prime path criteria,” Science of Computer Programming, vol. 152, pp. 99–115, Jan. 2018. [Online]. Available: https://www. sciencedirect.com/science/article/pii/S0167642317302150

    2018

  25. [34]

    AFL++: combining incremental steps of fuzzing research,

    A. Fioraldi, D. Maier, H. Eißfeldt, and M. Heuse, “AFL++: combining incremental steps of fuzzing research,” in Proceed- ings of the 14th USENIX Conference on Offensive Technolo- gies, ser. WOOT 2020. Boston, MA, USA: USENIX Associ- ation, 2020, p. 10

    2020

  26. [35]

    SVG and X3D in the context of the XML family and the semantic web,

    V . Geroimenko, “SVG and X3D in the context of the XML family and the semantic web,” in Visualizing Information Using SVG and X3D , V . Geroimenko and C. Chen, Eds. London: Springer-Verlag, 2005, pp. 3–20. [Online]. Available: http://link.springer.com/10.1007/1-84628-084-2_1

    2005 doi

  27. [36]

    Grammar- based whitebox fuzzing,

    P. Godefroid, A. Kiezun, and M. Y . Levin, “Grammar- based whitebox fuzzing,” in Proceedings of the 29th ACM SIGPLAN Conference on Programming Language Design and Implementation , ser. PLDI 2008. New York, NY , USA: Association for Computing Machinery, Jun. 2008, pp. 206–215. ...

    2008

  28. [37]

    Learn&Fuzz: Machine learning for input fuzzing,

    P. Godefroid, H. Peleg, and R. Singh, “Learn&Fuzz: Machine learning for input fuzzing,” in Proceedings of the 32nd IEEE/ACM International Conference on Automated Software Engineering, ser. ASE 2017. Urbana, IL: IEEE, Oct. 2017, pp. 50–59. [Online]. Available: http://ieeexplore...

    2017

  29. [38]

    Mining input grammars from dynamic control flow,

    R. Gopinath, B. Mathis, and A. Zeller, “Mining input grammars from dynamic control flow,” inProceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ser. ESEC/FSE 2020. New York, NY , USA: A...

    2020

  30. [39]

    Grammarinator: a grammar-based open source fuzzer,

    R. Hodovan, A. Kiss, and T. Gyimothy, “Grammarinator: a grammar-based open source fuzzer,” in Proceedings of the 9th ACM SIGSOFT International Workshop on Automating TEST Case Design, Selection, and Evaluation , ser. A-TEST 2018. New York, NY , USA: Association for Computing M...

    2018

  31. [40]

    Large language models for software engineering: a systematic literature review,

    X. Hou, Y . Zhao, Y . Liu, Z. Yang, K. Wang, L. Li, X. Luo, D. Lo, J. Grundy, and H. Wang, “Large language models for software engineering: a systematic literature review,” Aug. 2023, arXiv:2308.10620 [cs]. [Online]. Available: http://arxiv.org/abs/2308.10620

    2023 arXiv

  32. [41]

    Mining input grammars from dynamic taints,

    M. Höschele and A. Zeller, “Mining input grammars from dynamic taints,” in Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering, ser. ASE 2016. New York, NY , USA: Association for Computing Machinery, Aug. 2016, pp. 720–725. [Online]. Ava...

    2016

  33. [42]

    UTopia: Automatic generation of fuzz driver using unit tests,

    B. Jeong, J. Jang, H. Yi, J. Moon, J. Kim, I. Jeon, T. Kim, W. Shim, and Y . H. Hwang, “UTopia: Automatic generation of fuzz driver using unit tests,” in Proceedings of the 44th IEEE Symposium on Security and Privacy (SP), ser. SP 2023. San Francisco, CA, USA: IEEE, 16 May 202...

    2023

  34. [43]

    Genetic programming as a means for programming computers by natural selection,

    J. R. Koza, “Genetic programming as a means for programming computers by natural selection,” Statistics and Computing, vol. 4, no. 2, pp. 87–112, Jun. 1994. [Online]. Available: https://doi.org/10.1007/BF00175355

    1994 doi

  35. [44]

    Learning highly recursive input grammars,

    N. Kulkarni, C. Lemieux, and K. Sen, “Learning highly recursive input grammars,” in Proceedings of the 36th IEEE/ACM International Conference on Automated Software Engineering, ser. ASE 2021, Nov. 2021, pp. 456–467. [Online]. Available: https://doi.org/10.1109/ASE51524.2021.9678879

    2021

  36. [45]

    Artificial selection methods from evolutionary computing show promise for directed evolution of microbes,

    A. Lalejini, E. Dolson, A. E. V ostinar, and L. Zaman, “Artificial selection methods from evolutionary computing show promise for directed evolution of microbes,”eLife, vol. 11, p. e79665, Aug. 2022, publisher: eLife Sciences Publications, Ltd. [Online]. Available: https://doi...

    2022 doi

  37. [46]

    W. B. Langdon and R. Poli, Foundations of genetic program- ming. Springer Science & Business Media, Mar. 2013

    2013

  38. [47]

    Evolution through large models,

    J. Lehman, J. Gordon, S. Jain, K. Ndousse, C. Yeh, and K. O. Stanley, “Evolution through large models,” Jun. 2022. [Online]. Available: https://arxiv.org/abs/2206.08896v1

    2022 arXiv

  39. [48]

    OSS-fuzz-gen: Automated fuzz target generation,

    D. Liu, O. Chang, J. metzman, M. Sablotny, and M. Maruseac, “OSS-fuzz-gen: Automated fuzz target generation,” May 2024. [Online]. Available: https://github.com/google/oss-fuzz-gen

    2024

  40. [49]

    NNSmith: Generating diverse and valid test cases for deep learning compilers,

    J. Liu, J. Lin, F. Ruffy, C. Tan, J. Li, A. Panda, and L. Zhang, “NNSmith: Generating diverse and valid test cases for deep learning compilers,” in Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Vo...

    2023

  41. [50]

    Is your code generated by ChatGPT really correct? Rigorous evaluation of large language models for code generation,

    J. Liu, C. S. Xia, Y . Wang, and L. Zhang, “Is your code generated by ChatGPT really correct? Rigorous evaluation of large language models for code generation,” Advances in Neural Information Processing Systems, vol. 36, pp. 21 558–21 572, Dec. 2023. [Online]. Available: https...

    2023

  42. [52]

    Data contamination: From memorization to exploitation,

    I. Magar and R. Schwartz, “Data contamination: From memorization to exploitation,” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, S. Muresan, P. Nakov, and A. Villavicencio, Eds., vol. 2. Dublin, Ireland: Association for Computation...

    2022

  43. [53]

    Demystify the fuzzing methods: a comprehensive survey,

    S. Mallissery and Y .-S. Wu, “Demystify the fuzzing methods: a comprehensive survey,” ACM Computing Surveys, vol. 56, no. 3, pp. 71:1–71:38, Oct. 2023. [Online]. Available: https://dl.acm.org/doi/10.1145/3623375

    2023 doi

  44. [54]

    The art, science, and engineering of fuzzing: a survey,

    V . J. M. Manès, H. Han, C. Han, S. K. Cha, M. Egele, E. J. Schwartz, and M. Woo, “The art, science, and engineering of fuzzing: a survey,”IEEE Transactions on Software Engineer- ing, 2019

    2019

  45. [55]

    FuzzBench: an open fuzzer benchmarking platform and service,

    J. Metzman, L. Szekeres, L. Simon, R. Sprabery, and A. Arya, “FuzzBench: an open fuzzer benchmarking platform and service,” in Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering , ser. ...

    2019

  46. [56]

    Large language models in cybersecurity: state-of-the-art,

    F. N. Motlagh, M. Hajizadeh, M. Majd, P. Najafi, F. Cheng, and C. Meinel, “Large language models in cybersecurity: state-of-the-art,” Jan. 2024, arXiv:2402.00891 [cs]. [Online]. Available: http://arxiv.org/abs/2402.00891

    2024 arXiv

  47. [57]

    The mutators reloaded: Fuzzing compilers with large language model generated mutation operators,

    X. Ou, C. Li, Y . Jiang, and C. Xu, “The mutators reloaded: Fuzzing compilers with large language model generated mutation operators,” in Proceedings of the 2024 ACM International Conference on Architectural Support for Programming Languages and Operating Systems , ser. ASPLOS...

    2024

  48. [58]

    Semantic fuzzing with Zest,

    R. Padhye, C. Lemieux, K. Sen, M. Papadakis, and Y . Le Traon, “Semantic fuzzing with Zest,” in Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis, ser. ISSTA 2019. New York, NY , USA: Association for Computing Machinery, 2019, pp. 329...

    2019

  49. [59]

    Model compression via distillation and quantization,

    A. Polino, R. Pascanu, and D. Alistarh, “Model compression via distillation and quantization,” Feb. 2018, arXiv:1802.05668 [cs]. [Online]. Available: http://arxiv.org/abs/1802.05668

    2018 arXiv

  50. [60]

    Grammar-based fuzzing,

    S. Sargsyan, S. Kurmangaleev, M. Mehrabyan, M. Mishechkin, T. Ghukasyan, and S. Asryan, “Grammar-based fuzzing,” in Proceedings of the 2018 Ivannikov Memorial Workshop , ser. IVMEM 2018, Yerevan, Armenia, May 2018, pp. 32–

    2018

  51. [61]

    Available: https://ieeexplore.ieee.org/abstract/ document/8636353

    [Online]. Available: https://ieeexplore.ieee.org/abstract/ document/8636353

  52. [62]

    SoK: Prudent evaluation practices for fuzzing,

    M. Schloegel, N. Bars, N. Schiller, L. Bernhard, T. Scharnowski, A. Crump, A. Ale-Ebrahim, N. Bis- santz, M. Muench, and T. Holz, “SoK: Prudent evaluation practices for fuzzing,” in Proceedings of the 45th IEEE Sympo- sium on Security and Privacy, ser. SP 2024. San Francisco, ...

    2024

  53. [63]

    Input invariants,

    D. Steinhöfel and A. Zeller, “Input invariants,” inProceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software 17 Engineering, ser. ESEC/FSE 2022. New York, NY , USA: Association for Computing Machinery, 2022, pp. 583...

    2022 doi

  54. [64]

    Language-based software testing,

    ——, “Language-based software testing,” Communications of the ACM, vol. 67, no. 4, pp. 80–84, Mar. 2024. [Online]. Available: https://dl.acm.org/doi/10.1145/3631520

    2024 doi

  55. [65]

    sqlite/sqlite,

    T. S. D. Team, “sqlite/sqlite,” May 2025, original-date: 2019-03-18T12:21:01Z. [Online]. Available: https://github. com/sqlite/sqlite

    2025

  56. [66]

    Expectation vs. experience: evaluating the usability of code generation tools powered by large language models,

    P. Vaithilingam, T. Zhang, and E. L. Glassman, “Expectation vs. experience: evaluating the usability of code generation tools powered by large language models,” in Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems, ser. CHI EA 2022. New York, ...

    2022

  57. [67]

    Can large language models write good property-based tests?

    V . Vikram, C. Lemieux, J. Sunshine, and R. Padhye, “Can large language models write good property-based tests?” Jul. 2024, arXiv:2307.04346 [cs]. [Online]. Available: http://arxiv.org/abs/2307.04346

    2024 arXiv

  58. [68]

    A systematic review of fuzzing based on machine learning techniques,

    Y . Wang, P. Jia, L. Liu, C. Huang, and Z. Liu, “A systematic review of fuzzing based on machine learning techniques,” PLOS ONE, vol. 15, no. 8, p. e0237749, Aug. 2020, publisher: Public Library of Science. [Online]. Available: https://journals. plos.org/plosone/article?id=10....

    2020 doi

  59. [69]

    Not all coverage measurements are equal: Fuzzing by coverage accounting for input prioritization

    Y . Wang, X. Jia, Y . Liu, K. Zeng, T. Bao, D. Wu, and P. Su, “Not all coverage measurements are equal: Fuzzing by coverage accounting for input prioritization.” in Proceedings of 2020 Network and Distributed System Security Symposium, ser. NDSS 2020, San Diego, CA, USA, 2020....

    2020

  60. [70]

    Jenetics: Java genetic algorithm library

    F. Wilhelmstötter, “Jenetics: Java genetic algorithm library.” [Online]. Available: https://jenetics.io/

  61. [71]

    Evolutionary computation in the era of large language model: survey and roadmap,

    X. Wu, S.-h. Wu, J. Wu, L. Feng, and K. C. Tan, “Evolutionary computation in the era of large language model: survey and roadmap,” May 2024, arXiv:2401.10034 [cs]. [Online]. Available: http://arxiv.org/abs/2401.10034

    2024 arXiv

  62. [72]

    Fuzz4All: Universal fuzzing with large language models,

    C. S. Xia, M. Paltenghi, J. Le Tian, M. Pradel, and L. Zhang, “Fuzz4All: Universal fuzzing with large language models,” in Proceedings of the 46th IEEE/ACM International Conference on Software Engineering, ser. ICSE 2024. New York, NY , USA: Association for Computing Machinery...

    2024

  63. [73]

    Large language models for cyber security: a systematic literature review,

    H. Xu, S. Wang, N. Li, K. Wang, Y . Zhao, K. Chen, T. Yu, Y . Liu, and H. Wang, “Large language models for cyber security: a systematic literature review,” Jul. 2024, arXiv:2405.04760 [cs]. [Online]. Available: http: //arxiv.org/abs/2405.04760

    2024

  64. [74]

    Finding and un- derstanding bugs in C compilers,

    X. Yang, Y . Chen, E. Eide, and J. Regehr, “Finding and un- derstanding bugs in C compilers,” in Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language De- sign and Implementation, ser. PLDI 2011. New York, NY , USA: Association for Computing Machinery, Jun. 20...

    2011

  65. [75]

    Zeller, R

    A. Zeller, R. Gopinath, M. Böhme, G. Fraser, and C. Holler, The fuzzing book . CISPA Helmholtz Center for Information Security, 2024. [Online]. Available: https: //www.fuzzingbook.org/

    2024

  66. [76]

    How effective are they? Exploring large language model based fuzz driver generation,

    C. Zhang, Y . Zheng, M. Bai, Y . Li, W. Ma, X. Xie, Y . Li, L. Sun, and Y . Liu, “How effective are they? Exploring large language model based fuzz driver generation,” in Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis , ser. ISSTA

  67. [77]

    FixReverter: a realistic bug injection methodology for benchmarking fuzz testing,

    Z. Zhang, Z. Patterson, M. Hicks, and S. Wei, “FixReverter: a realistic bug injection methodology for benchmarking fuzz testing,” in Proceedings of the 31st USENIX Security Symposium, ser. USENIX Security 2022. Boston, MA, USA: USENIX Association, 2022, pp. 3699–3715. [Online]...

    2022

  68. [78]

    IntelliGen: automatic driver synthesis for fuzz testing,

    M. Zhang, J. Liu, F. Ma, H. Zhang, and Y . Jiang, “IntelliGen: automatic driver synthesis for fuzz testing,” in Proceedings of the 43rd IEEE/ACM nternational Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP), ser. ICSE-SEIP 2021, Madrid, Spain, M...

    2021

  69. [79]

    Fuzzing: a survey for roadmap,

    X. Zhu, S. Wen, S. Camtepe, and Y . Xiang, “Fuzzing: a survey for roadmap,” ACM Computing Surveys , vol. 54, no. 11s, pp. 230:1–230:36, 2022. [Online]. Available: https://doi.org/10.1145/3512345 18 Appendix A The Seed Fuzzers Listing 14 presents the seed fuzzer we use (simplif...

    2022 doi

  70. [80]

    A systematic review of fuzzing,

    X. Zhao, H. Qu, J. Xu, X. Li, W. Lv, and G.-G. Wang, “A systematic review of fuzzing,” Soft Computing, vol. 28, no. 6, pp. 5493–5522, Mar. 2024. [Online]. Available: https://doi.org/10.1007/s00500-023-09306-2

    2024 doi

  71. [2024]

    2024, pp

    New York, NY , USA: Association for Computing Machinery, Sep. 2024, pp. 1223–1235. [Online]. Available: https://dl.acm.org/doi/10.1145/3650212.3680355

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.