Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

Validating Network Protocol Parsers with Traceable RFC Document Interpretation

T0 review · 4 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read ParCleanse shows that an LLM can distill RFC documents into a formal message format used as a quasi-oracle for validating protocol parsers, finding 69 bugs (36 confirmed) across nine protocols.

desk verdict A solid, well-engineered tool paper that turns RFCs into executable test oracles via LLMs and finds real developer-confirmed bugs; the traceability claim is stronger than the evidence supports, but it deserves serious peer review. read the letter →

arxiv 2504.18050 v1 pith:ABQ6BDDD submitted 2025-04-25 cs.SE cs.AI

Mingwei Zheng , Danning Xie , Qingkai Shi , Chengpeng Wang , Xiangyu Zhang This is my paper · ORCID
classification cs.SEcs.AI
keywords networkprotocolparserslargelanguagemodelsRFCdocumentsformalspecificationextractiontraceabilityquasi-oracledivide-and-conquerparservalidation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ParCleanse claims that a network protocol parser can be checked against a quasi-oracle — an approximate formal format, extracted automatically from the protocol's own RFC document — instead of relying on hand-written models or other implementations. The paper's pipeline splits an RFC into a tree of sections, uses a large language model to translate each section into a formal message format in a small DSL, merges the sub-formats bottom-up, and then generates one valid packet per format path plus one mutated packet per violated property. Any disagreement between parser and format is traced back to the exact RFC section and triaged by the LLM as either an implementation bug or a format-extraction error, with 97% accuracy on the nine protocols studied. The result is 69 unique bugs (36 confirmed) in C, Go, and Python parsers, mostly silent input-validation faults that crash-free fuzzing and differential testing miss. If the approach carries over, it would let standards documents themselves act as the test oracle across the network protocol ecosystem.

What carries the argument

The load-bearing object is the DocTree, a hierarchy of RFC sections in which each node stores the section's text, an LLM-written summary, and the protocol sub-format extracted from that section; edges record dependencies that the table of contents does not show, such as a TLV section being a child of the packet-format section. The DocTree enables three things at once: divide-and-conquer extraction that keeps LLM context small, bottom-up merging of sub-formats into one complete format, and traceability, since every generated constraint records the section that authored it. The second mechanism is the Format Graph, a DAG of fields and subgraphs built from the merged format; each path through it is encoded as an SMT formula, and the solver produces a positive packet for the path and a negative packet for each individual constraint negation. The combination means every test input and every later diagnosis can cite the exact RFC section that justifies it, which is what lets the pipeline separate parser bugs from its own extraction errors.

What would settle it

Run ParCleanse on an RFC that carries a known erratum or a deliberately planted error in one field constraint, and observe whether a correct parser is reported as buggy; if it is, the oracle has inherited the document's error, showing the load-bearing assumption is wrong.

Watch

Extended reading notes

Core claim

The central discovery is operational: a large language model, guided by a document-structure tree, can convert the natural-language format descriptions of RFCs such as those for Babel, BFD, BGP-4, IPv4/IPv6, ICMPv4/v6, DHCP, and TCP into a formal message grammar whose constraints are precise enough to drive test generation. The paper demonstrates that this grammar, used as a quasi-oracle, catches parsers that accept a packet violating a documented constraint (for example, a Babel Router-Id TLV whose RouterId field is all zeroes, where the RFC states it must satisfy $RouterId \neq 0$) or reject a packet that satisfies every constraint. Because each constraint retains a pointer to the RFC section it came from, suspected violations are re-adjudicated against that section, and the paper reports that this triage is correct for 87 of 90 detected inconsistencies (97%), separating 79 implementation errors from 11 format-extraction errors. The paper further claims that this pipeline outperforms the state of the art: 100% precision/recall on message types versus 89%/55% for the LLM-based fuzzing baseline, and 33 unique new bugs on the C protocols versus 4 for differential analysis, with 68 of 69 total bugs new and 36 confirmed.

Load-bearing premise

The pipeline treats the RFC text as the complete, correct, and unambiguous ground truth, so any error, omission, or ambiguity in the standard is inherited by the oracle and can turn a correct parser into a reported bug.

Editorial extensions

If this is right

  • A parser can be validated directly against its standard document, so bugs shared by every implementation of a protocol — which differential testing cannot see — become detectable.
  • Because every reported inconsistency is traceable to the RFC section that defines the violated property, the paper's ablation shows that without this backtrace inconsistency-classification accuracy drops from 97% to 84% and format-error detection to 0%.
  • Property-level mutation is what finds silent logical bugs: whole-format testing yields mostly crashes (211 of 233 inconsistencies), while fine-grained testing yields 75 logical inconsistencies and 69 unique bugs.
  • Divide-and-conquer via DocTree is essential for long documents: feeding the whole RFC to the LLM drops field-name recall from 95% to 54% and independent-constraint recall from 82% to 26%.
  • The reported 97% inconsistency-classification accuracy and 100% precision/recall on message types indicate that the extracted format can serve as an oracle for these nine protocols.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The oracle inherits the standard's own ambiguities, so the same pipeline could be pointed at RFCs with known errata to make the tool double as a standards-quality checker rather than only a parser validator.
  • Where extraction is weakest — dependent constraints at 44% recall, with TCP's implicit padding alignment as the documented failure mode — the property-level mutation strategy could be extended to cross-field mutations, not just single-constraint negations, to probe interactions between dependent fields.
  • The black-box accept/reject/crash signal could be enriched with partial-parse or warning outputs from parsers, potentially turning silent state corruptions into earlier, observable inconsistencies.
  • Because the DSL is language-independent and the tool needs only an executable, the same DocTree pipeline is a natural candidate for protocols with bit-level layouts (for example TLS or QUIC), where implicit alignment rules are likely to stress the extraction step further.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces ParCleanse, a system that uses GPT-4o to extract formal protocol message formats from RFC documents via a divide-and-conquer DocTree strategy, then uses those formats as a quasi-oracle to validate network protocol parsers. Phase 1 extracts per-section formats and merges them bottom-up; Phase 2 transforms the merged format into a Format Graph and generates positive and negative inputs through property-level mutations solved with Z3; Phase 3, when a parser deviates from the extracted format, backtraces the relevant RFC section and asks the LLM to classify the mismatch as either an implementation error or a format-extraction error, refining the format in the latter case. The evaluation covers nine protocols in C, Go, and Python, reports precision/recall of the extracted formats against a manually built ground truth, compares against ChatAFL and ParDiff, and reports 69 detected bugs with 36 confirmed by developers.

Significance. If the central results hold, this is a useful step toward automating oracle construction and traceability for protocol parser validation. The strongest evidence is the 36 developer-confirmed bugs in widely used implementations, together with a public artifact, a syntax-checked extraction pipeline, and a grounded comparison against established baselines. The main caveat is that the quasi-oracle is an approximate LLM interpretation, not the document itself; the traceability claim therefore needs to be qualified. With that qualification, the paper would be a solid contribution to LLM-assisted software validation and protocol testing.

major comments (4)
  1. [§4.3] The inconsistency-identification step is not independent of the oracle construction. The same GPT-4o model that extracted the disputed constraint in Phase 1 (Algorithm 1, lines 3–8) is given the same RFC section in Phase 3 and asked whether the constraint or the parser is correct. If the model misread the section during extraction, it is likely to repeat that misreading when diagnosing, so the reported 97% identification accuracy (Table 3, §5.3.2) and the abstract's traceability claim do not establish that each reported bug is traceable to the document rather than to the model's interpretation of the document. I ask for an independent adjudication of at least a random sample of the 90 detected inconsistencies (e.g., by protocol experts not involved in building the ground truth, or by an ablation in which the Phase 3 validator is prompted only with the raw section and no knowledge of the extracted format), with results reported separately for format-extraction errors.
  2. [§5.2.2, Table 2] Dependent-constraint extraction is substantially incomplete: the overall recall is 44%, with 0% recall for BFD, DHCP, and TCP and 0% precision/recall for DHCP and TCP. Since Phase 2 (§4.2) generates negative tests by negating extracted constraints, the 56% of ground-truth dependent constraints not extracted are never exercised, which undercuts the 'complete protocol format' claim in §4.1 and the RQ1 conclusion of 'over 90% precision and recall for most elements.' Please quantify how many of the reported bugs concern dependent constraints, report the coverage of ground-truth dependent constraints by the final oracle, and either soften the completeness claims or add experiments on protocols with more dependent constraints.
  3. [§3.2, Definition 2; Abstract] The abstract's statement that 'any bugs we find in a protocol implementation can be traced back to the document' is not supported by Definition 2, which explicitly allows the extracted format F' to deviate from the true format F. A report can be traced to an RFC section, but the section may not entail the extracted constraint; Table 3's own 11 format-extraction errors demonstrate this gap. Please revise the traceability claim to 'traced to a document section and to the LLM-extracted constraint, with the constraint being an approximate interpretation that may require confirmation,' and make the residual risk explicit in the contributions and conclusion.
  4. [§5.3.2, Table 3] The headline count of 69 bugs depends on the Phase 3 classification, which was manually rechecked by the authors at five minutes per inconsistency rather than by independent developers for all cases. The 36 developer confirmations are strong evidence for those specific bugs, but they do not validate the 33 unconfirmed reports. Please report the per-protocol confirmation rate, separate the '69 detected' and '36 confirmed' claims in the abstract and conclusion, and, where possible, provide evidence that the unconfirmed reports were sent to maintainers with sufficient context (e.g., the traceability data) for them to act on.
minor comments (4)
  1. [Table 1] The TCP row lists the description as 'Extensible Authentication Protocol,' which is incorrect for RFC 793; this appears to be a copy-paste error from another protocol.
  2. [Algorithm 1, §4.1.2] The MergeFormats function (lines 11–16) does not run SyntaxChecker on the merged format, even though node-level formats are syntax-checked in lines 3–8; a syntax error introduced during merging would be caught only later in Phase 2, if at all.
  3. [Table 3] The table is very dense because many subcolumns are compressed into a single row; separating the inconsistency-identification and bug-detection parts into two tables, or adding clear column headers, would make it substantially easier to verify the totals.
  4. [§5.3.1] Generating exactly one positive test case per format path may make results sensitive to Z3's chosen assignment; a short sensitivity analysis (e.g., two or three assignments per path for one protocol) would help establish that the 69-bug count is not an artifact of the solver's choices.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the extracted format is explicitly acknowledged as approximate, and the Phase 3 adjudication is designed to detect and correct extraction errors rather than to ratify them by construction.

full rationale

ParCleanse's central derivation is not circular. The oracle F' is explicitly defined as an approximation of the true document-derived format F (Definition 2), and the paper states that observing f(p) != F'(p) does not directly imply f(p) != F(p). Phase 3 is therefore a genuine adjudication step, not a rubber stamp: the LLM is asked to decide whether the format or the parser is correct against the cited RFC section, and the evaluation reports 11 format-extraction errors among the 90 detected inconsistencies, showing that the diagnosis can and does disagree with the extraction. The same GPT-4o model is used for both extraction and diagnosis, which is a legitimate concern about shared hallucination and independence, but it is not a definitional reduction: there is no equation, fitted parameter, or construction that forces the validator to agree with the extracted constraint. The traceability claim is weaker than advertised insofar as a bug is traceable to a section that may not actually support the extracted constraint, but the paper acknowledges this risk explicitly in Section 6 and reports no observed document bugs. External evidence also exists: 36 of the 69 reported bugs were developer-confirmed, and the extraction quality is measured against a manually constructed ground truth and compared with ChatAFL and ParDiff. No load-bearing self-citation chain or imported uniqueness theorem appears. The identified issues are validity and generalization concerns, not circularity, so the score is 0.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The central claim rests on two external anchors: RFC correctness as ground truth and the semantic reliability of GPT-4o in both extracting and judging protocol formats. The evaluation adds a third assumption that the authors' manual ground truth is unbiased. No numeric free parameters are fitted; the method's design choices (single positive test per path, dataset filters, temperature 0) are generation or evaluation choices rather than fitted values.

assumptions (5)
  • domain assumption RFC documents are correct, complete, and authoritative for protocol behavior.
    Section 6 explicitly treats the document as ground truth and assumes high quality; if false, oracle errors propagate to bug reports.
  • ad hoc to paper GPT-4o can semantically translate RFC prose into the DSL format when guided by divide-and-conquer and syntax checking.
    Section 4.1.2 relies on LLMGenerateFormat and LLMSyntaxRefine; the entire pipeline depends on this specific model's capability.
  • domain assumption Parser pass/fail status is observable through status codes or annotated expected endpoints.
    Section 4.2.2 defines how success is detected; incorrect annotation would make oracle comparison invalid.
  • domain assumption The manually written ground truth formats by two expert authors correctly represent the protocols.
    Section 5.2.1 uses two-author consensus as the benchmark; bias could inflate reported precision and recall.
  • standard math Z3 can solve the generated path constraints to produce concrete packet assignments.
    Phase 2 depends on Z3 to materialize positive and negative inputs from path formulas.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Validating Network Protocol Parsers with Traceable RFC Document Interpretation." pith.science (2026). https://pith.science/paper/ABQ6BDDD

@misc{pith2026250418050,
  author       = {Pith},
  title        = {Pith review of: Validating Network Protocol Parsers with Traceable RFC Document Interpretation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ABQ6BDDD}},
  note         = {Machine review of arXiv:2504.18050}
}
read the original abstract

Validating the correctness of network protocol implementations is highly challenging due to the oracle and traceability problems. The former determines when a protocol implementation can be considered buggy, especially when the bugs do not cause any observable symptoms. The latter allows developers to understand how an implementation violates the protocol specification, thereby facilitating bug fixes. Unlike existing works that rarely take both problems into account, this work considers both and provides an effective solution using recent advances in large language models (LLMs). Our key observation is that network protocols are often released with structured specification documents, a.k.a. RFC documents, which can be systematically translated to formal protocol message specifications via LLMs. Such specifications, which may contain errors due to the hallucination of LLMs, are used as a quasi-oracle to validate protocol parsers, while the validation results in return gradually refine the oracle. Since the oracle is derived from the document, any bugs we find in a protocol implementation can be traced back to the document, thus addressing the traceability problem. We have extensively evaluated our approach using nine network protocols and their implementations written in C, Python, and Go. The results show that our approach outperforms the state-of-the-art and has detected 69 bugs, with 36 confirmed. The project also demonstrates the potential for fully automating software validation based on natural language specifications, a process previously considered predominantly manual due to the need to understand specification documents and derive expected outputs for test inputs.

Figures

Figures reproduced from arXiv: 2504.18050 by the authors.

Figure 1
Figure 1. A bug detected by ParCleanse, its fix, and the corresponding documentation. – It features fine-grained property-level input mutations to thoroughly test parser implementa￾tions guided by the extracted specifications. – It leverages a traceable inconsistency identification technique, allowing any identified incon￾sistencies to be traced back to the original specification for a more accurate diagnosis. • We implement … view at source ↗
Figure 2
Figure 2. ParCleanse extracts specifications from documents to build DocTree. Dashed arrows in B indicate hierarchical relationships for forming the protocol format in C . ParCleanse then generates test cases to detect inconsistencies and backtraces the relevant document section for LLM diagnosis. Differential analysis tools like ParDiff [58] and DPIFuzz [38] partially address the need for protocol-specific oracles by compari… view at source ↗
Figure 3
Figure 3. The format syntax of protocol packets To ensure correct interpretation, these elements must be organized in a specific form defined in [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: The pipeline of ParCleanse by Type. For Type = 0, Payload is an empty struct, representing Pad1; for Type = 6, Payload represents RouterIdTLV. 3.2 Problem Statement Definition 1 (Network Protocol Parser). A network protocol parser is a function 𝑓 that maps a network pa…
Figure 5
Figure 5. Figure 5: LLM Output for Generating Section Summaries and Identifying Hierarchy Dependences Node-level Protocol Format Generation (Divide). The protocol format for each DocTree node is generated as described by lines 3–8 in Algorithm 1. First, the content of each node and 𝐷𝑆𝐿 ar…
Figure 6
Figure 6. Figure 6: Transform Protocol Format to Format Graph for Test Generation [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. ProSec: Fortifying Code LLMs with Proactive Security Alignment

    cs.CR 2024-11 conditional novelty 8.0 of 10

    ProSec synthesizes vulnerability-inducing coding scenarios from CWE definitions and uses preference learning on model-generated fixes to make code LLMs up to 35.4% more secure on the PurpleLlama benchmark.

Reference graph

Works this paper leans on

58 extracted references · 26 canonical work pages · cited by 1 Pith paper

  1. [1]

    RFC 793 - Transmission Control Protocol

    1981. RFC 793 - Transmission Control Protocol. https://www.rfc-editor.org/rfc/rfc793.html

    work page 1981

  2. [2]

    Go Networking

    2024. Go Networking. https://github.com/golang/net

    work page 2024

  3. [3]

    IETF DataTracker

    2024. IETF DataTracker. https://datatracker.ietf.org

    work page 2024

  4. [4]

    Impacket

    2024. Impacket. https://github.com/fortra/impacket

    work page 2024

  5. [5]

    Internet Standard

    2024. Internet Standard. https://en.wikipedia.org/wiki/Internet_Standard

    work page 2024

  6. [6]

    Fernando Arnaboldi. 2023. XDiFF. https://github.com/IOActive/XDiFF

    work page 2023

  7. [7]

    Cristian Augusto, Jesús Morán, Antonia Bertolino, Claudio de la Riva, and Javier Tuya. 2024. Software System Testing Assisted by Large Language Models: An Exploratory Study. In Testing Software and Systems (ICTSS ’2024) . Springer-Verlag, 239–255. doi:10.1007/978-3-031-80889-0_17

    work page doi:10.1007/978-3-031-80889-0_17 2024

  8. [8]

    babeld. 2024. babeld. https://github.com/jech/babeld

    work page 2024

Show all 58 references
  1. [9]

    Cristian Cadar, Daniel Dunbar, and Dawson R. Engler. 2008. KLEE: Unassisted and automatic generation of high- coverage tests for complex systems programs. In Proceedings of the 8th USENIX Symposium on Operating Systems Design and Implementation (OSDI ’08) . USENIX, 209–224. ht...

    2008

  2. [10]

    Clarke, Alex Groce, Somesh Jha, and Helmut Veith

    Sagar Chaki, Edmund M. Clarke, Alex Groce, Somesh Jha, and Helmut Veith. 2003. Modular Verification of Software Components in C. In Proceedings of the 25th International Conference on Software Engineering (ICSE ’03) . IEEE, 385–395. doi:10.1109/ICSE.2003.1201217

    2003 arXiv

  3. [11]

    Endadul Hoque, Huangyi Ge, Aniket Kate, Cristina Nita-Rotaru, and Ninghui Li

    Sze Yiu Chau, Omar Chowdhury, Md. Endadul Hoque, Huangyi Ge, Aniket Kate, Cristina Nita-Rotaru, and Ninghui Li. 2017. SymCerts: Practical Symbolic Execution for Exposing Noncompliance in X.509 Certificate Validation Implementations. In IEEE Symposium on Security and Privacy (S...

    2017 doi

  4. [12]

    FRR community. 2024. The FRRouting protocol suite. https://github.com/FRRouting/frr

    2024

  5. [14]

    Yinlin Deng, Chunqiu Steven Xia, Haoran Peng, Chenyuan Yang, and Lingming Zhang. 2023. Large language models are zero-shot fuzzers: Fuzzing deep-learning libraries via large language models. In Proceedings of the 32nd ACM SIGSOFT international symposium on software testing and...

    2023

  6. [15]

    Gregorio Díaz, Fernando Cuartero, Valentín Valero Ruiz, and Fernando L. Pelayo. 2004. Automatic verification of the TLS handshake protocol. In Proceedings of the 2004 ACM Symposium on Applied Computing (SAC ’04) . ACM, 789–794. doi:10.1145/967900.968063

    2004

  7. [16]

    Yangruibo Ding, Marcus J Min, Gail Kaiser, and Baishakhi Ray. 2024. Cycle: Learning to self-refine the code generation. Proceedings of the ACM on Programming Languages 8, OOPSLA1 (2024), 392–418. doi:10.1145/3649825

    2024 doi

  8. [17]

    Fatemeh Erfan, Mohammad Yahyatabar, Martine Bellaiche, and Talal Halabi. 2024. Advanced Smart Contract Vulnera- bility Detection Using Large Language Models. In 2024 8th Cyber Security in Networking Conference (CSNet ’24) . IEEE, 289–296. doi:10.1109/CSNet64211.2024.10851734

    2024

  9. [18]

    Mingyang Geng, Shangwen Wang, Dezun Dong, Haotian Wang, Ge Li, Zhi Jin, Xiaoguang Mao, and Xiangke Liao

  10. [19]

    Patrice Godefroid, Michael Y Levin, and David Molnar. 2012. SAGE: whitebox fuzzing for security testing. Commun. ACM 55, 3 (2012), 40–44. doi:10.1145/2093548.2093564

    2012

  11. [20]

    Jinyao Guo, Chengpeng Wang, Xiangzhe Xu, Zian Su, and Xiangyu Zhang. 2025. RepoAudit: An Autonomous LLM- Agent for Repository-Level Code Auditing. arXiv preprint arXiv:2501.18160 (2025). doi:10.48550/ARXIV.2501.18160

    2025 doi arXiv

  12. [23]

    Haonan Li, Yu Hao, Yizhuo Zhai, and Zhiyun Qian. 2024. Enhancing Static Analysis for Practical Bug Detection: An LLM-Integrated Approach. Proc. ACM Program. Lang. 8, OOPSLA1, Article 111 (2024), 26 pages. doi:10.1145/3649828

    2024 doi

  13. [24]

    Haonan Li, Hang Zhang, Kexin Pei, and Zhiyun Qian. 2025. The Hitchhiker’s Guide to Program Analysis, Part II: Deep Thoughts by LLMs. arXiv preprint arXiv:2504.11711 (2025). doi:10.48550/ARXIV.2308.00245

    2025 doi arXiv

  14. [25]

    Ming Liang, Xiaoheng Xie, Gehao Zhang, Xunjin Zheng, Peng Di, Wei Jiang, Hongwei Chen, Chengpeng Wang, and Gang Fan. 2024. RepoGenix: Dual Context-Aided Repository-Level Code Completion with Language Models. In Proceedings of the 39th IEEE/ACM International Conference on Autom...

    2024

  15. [26]

    Congyu Liu, Sishuai Gong, and Pedro Fonseca. 2023. KIT: Testing OS-Level Virtualization for Functional Interference Bugs. In Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2 (ASPLOS ’23) . A...

    2023

  16. [27]

    Jiawei Liu, Songrun Xie, Junhao Wang, Yuxiang Wei, Yifeng Ding, and Lingming Zhang. 2024. Evaluating Language Models for Efficient Code Generation. In First Conference on Language Modeling (COLM ’24) . https://openreview.net/ forum?id=IBCBMeAhmC

    2024

  17. [28]

    Mosthaf and Andrzej Wasowski

    My M. Mosthaf and Andrzej Wasowski. 2024. From a Natural to a Formal Language with DSL Assistant. In Proceedings of the ACM/IEEE 27th International Conference on Model Driven Engineering Languages and Systems (MODELS ’24) . ACM, 541–549. doi:10.1145/3652620.3687811

    2024

  18. [29]

    Xiaoyue Ma, Lannan Luo, and Qiang Zeng. 2024. From One Thousand Pages of Specification to Unveiling Hidden Bugs: Large Language Model Assisted Fuzzing of Matter IoT Devices. In Proceedings of the 33rd USENIX Conference on Security Symposium (USENIX Security ’24) . USENIX, 4783...

    2024

  19. [30]

    Stephen McQuistin, Mladen Karan, Prashant Khare, Colin Perkins, Gareth Tyson, Matthew Purver, Patrick Healey, Waleed Iqbal, Junaid Qadir, and Ignacio Castro. 2021. Characterising the IETF through the lens of RFC deployment. In Proceedings of the 21st ACM Internet Measurement C...

    2021

  20. [31]

    Ruijie Meng, Martin Mirchev, Marcel Böhme, and Abhik Roychoudhury. 2024. Large language model guided protocol fuzzing. In Proceedings of the 31st Annual Network and Distributed System Security Symposium (NDSS ’24) . The Internet Society. doi:10.14722/ndss.2024.24556

    2024

  21. [32]

    MITRE. 2022. CWE Top 25 Most Dangerous Software Weaknesses. https://cwe.mitre.org/top25/archive/2022/2022_ cwe_top25.html

    2022

  22. [33]

    MITRE. 2024. CWE-20: Improper Input Validation. https://cwe.mitre.org/data/definitions/20.html

    2024

  23. [34]

    Madanlal Musuvathi and Dawson R. Engler. 2004. Model Checking Large Network Protocol Implementations. In Proceedings of the 1st Conference on Symposium on Networked Systems Design and Implementation (NSDI ’24) . USENIX, 155–168. http://www.usenix.org/events/nsdi04/tech/musuvathi.html

    2004

  24. [35]

    OpenAI. 2024. GPT-4o. https://platform.openai.com/docs/models/gpt-4o

    2024

  25. [36]

    Joshua Pereyda. 2023. BooFuzz. https://github.com/jtpereyda/boofuzz

    2023

  26. [37]

    Tahina Ramananandro, Antoine Delignat-Lavaud, Cédric Fournet, Nikhil Swamy, Tej Chajed, Nadim Kobeissi, and Jonathan Protzenko. 2019. EverParse: Verified Secure Zero-Copy Parsers for Authenticated Message Formats. In Proceedings of the 28th USENIX Conference on Security Sympos...

    2019

  27. [38]

    Gaganjeet Singh Reen and Christian Rossow. 2020. DPIFuzz: a differential fuzzing framework to detect DPI elusion strategies for QUIC. In Proceedings of the 36th Annual Computer Security Applications Conference (ACSAC ’20) . ACM, 332–344. doi:10.1145/3427228.3427662

    2020

  28. [39]

    Microsoft Research. 2020. everparse. https://project-everest.github.io/everparse/3d-lang.html

    2020

  29. [40]

    Richard Rutledge and Alessandro Orso. 2022. Automating Differential Testing with Overapproximate Symbolic Execution. In 2022 15th IEEE Conference on Software Testing, Verification and Validation (ICST ’22) . IEEE, 256–266. doi:10.1109/ICST53961.2022.00035

    2022

  30. [41]

    Gabriel Ryan, Siddhartha Jain, Mingyue Shang, Shiqi Wang, Xiaofei Ma, Murali Krishna Ramanathan, and Baishakhi Ray. 2024. Code-Aware Prompting: A Study of Coverage-Guided Test Generation in Regression Setting using LLM. Proc. ACM Softw. Eng. 1, FSE, Article 43 (2024), 21 pages...

    2024 doi

  31. [42]

    Qingkai Shi, Junyang Shao, Yapeng Ye, Mingwei Zheng, and Xiangyu Zhang. 2023. Lifting Network Protocol Implemen- tation to Precise Format Specification with Security Applications. In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security (CCS ’23...

    2023

  32. [43]

    Qingkai Shi, Xiao Xiao, Rongxin Wu, Jinguo Zhou, Gang Fan, and Charles Zhang. 2018. Pinpoint: Fast and precise sparse value flow analysis for million lines of code. InProceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’18) . ...

    2018

  33. [44]

    Benjamin Steenhoek, Md Mahbubur Rahman, Richard Jiles, and Wei Le. 2023. An Empirical Study of Deep Learning Models for Vulnerability Detection. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE ’23) . IEEE, 2237–2248. doi:10.1109/ICSE48619.2023.00188

    2023

  34. [45]

    Nikhil Swamy, Tahina Ramananandro, Aseem Rastogi, Irina Spiridonova, Haobin Ni, Dmitry Malloy, Juan Vazquez, Michael Tang, Omar Cardona, and Arti Gupta. 2022. Hardening attack surfaces with formally proven binary format parsers. In 43rd ACM SIGPLAN International Conference on ...

    2022

  35. [46]

    Octavian Udrea and Cristian Lumezanu. 2006. Rule-Based Static Analysis of Network Protocol Implementations. In Proceedings of the 15th Conference on USENIX Security Symposium (USENIX Security ’06) . USENIX. https://www.usenix. , Vol. 1, No. 1, Article . Publication date: April...

    2006

  36. [47]

    Chengpeng Wang, Jipeng Zhang, Rongxin Wu, and Charles Zhang. 2024. DAInfer: Inferring API Aliasing Specifications from Library Documentation via Neurosymbolic Optimization. Proc. ACM Softw. Eng. 1, FSE (2024), 2469–2492. doi:10.1145/3660816

    2024 doi

  37. [48]

    Chengpeng Wang, Wuqi Zhang, Zian Su, Xiangzhe Xu, Xiaoheng Xie, and Xiangyu Zhang. 2024. LLMDFA: Analyzing Dataflow in Code with Large Language Models. In Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems (NeurIPS ...

    2024

  38. [49]

    Chengpeng Wang, Wuqi Zhang, Zian Su, Xiangzhe Xu, and Xiangyu Zhang. 2024. Sanitizing Large Language Models in Bug Detection with Data-Flow. In Findings of the Association for Computational Linguistics (EMNLP ’24) . Association for Computational Linguistics, 3790–3805. doi:10....

    2024 doi

  39. [50]

    Jincheng Wang, Le Yu, and Xiapu Luo. 2024. LLMIF: Augmented Large Language Model for Fuzzing IoT Devices. In 2024 IEEE Symposium on Security and Privacy (S&P ’24) . IEEE, 881–896. doi:10.1109/SP54263.2024.00211

    2024

  40. [51]

    Cheng Wen, Jialun Cao, Jie Su, Zhiwu Xu, Shengchao Qin, Mengda He, Haokun Li, Shing-Chi Cheung, and Cong Tian. 2024. Enchanting program specification synthesis by large language models using static analysis and program verification. In International Conference on Computer Aide...

    2024 doi

  41. [52]

    Yi Wu, Nan Jiang, Hung Viet Pham, Thibaud Lutellier, Jordan Davis, Lin Tan, Petr Babkin, and Sameena Shah. 2023. How Effective Are Neural Networks for Fixing Security Vulnerabilities. In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analys...

    2023

  42. [53]

    Chunqiu Steven Xia, Matteo Paltenghi, Jia Le Tian, Michael Pradel, and Lingming Zhang. 2024. Fuzz4All: Universal Fuzzing with Large Language Models. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering (ICSE ’24) . ACM, Article 126, 13 pages. do...

    2024

  43. [54]

    Danning Xie, Byungwoo Yoo, Nan Jiang, Mijung Kim, Lin Tan, Xiangyu Zhang, and Judy S Lee. 2023. Impact of Large Language Models on Generating Software Specifications. arXiv preprint arXiv:2306.03324 (2023). doi:10.48550/ARXIV. 2306.03324

    2023 doi arXiv

  44. [55]

    Chenyuan Yang, Yinlin Deng, Runyu Lu, Jiayi Yao, Jiawei Liu, Reyhaneh Jabbarvand, and Lingming Zhang. 2024. WhiteFox: White-Box Compiler Fuzzing Empowered by Large Language Models. 8, OOPSLA2 (2024). doi:10.1145/ 3689736

    2024

  45. [56]

    Zhe Yang, Hao Peng, Yanling Jiang, Xingwei Li, Haohua Du, Shuhai Wang, and Jianwei Liu. 2025. ChatHTTPFuzz: large language model-assisted IoT HTTP fuzzing. International Journal of Machine Learning and Cybernetics (2025), 1–22. doi:10.1007/s13042-024-02527-3

    2025 doi

  46. [57]

    Yuntong Zhang, Haifeng Ruan, Zhiyu Fan, and Abhik Roychoudhury. 2024. Autocoderover: Autonomous program improvement. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA ’24). ACM, 1592–1604. doi:10.1145/3650212.3680384

    2024

  47. [58]

    Mingwei Zheng, Qingkai Shi, Xuwei Liu, Xiangzhe Xu, Le Yu, Congyu Liu, Guannan Wei, and Xiangyu Zhang. 2024. ParDiff: Practical Static Differential Analysis of Network Protocol Parsers. In Proc. ACM Program. Lang. (OOPSLA ’24) . ACM, 1208–1234. doi:10.1145/3649854

    2024 doi

  48. [59]

    Mingwei Zheng, Jun Yang, Ming Wen, Hengcheng Zhu, Yepang Liu, and Hai Jin. 2021. Why Do Developers Remove Lambda Expressions in Java?. In 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE ’21). IEEE, 67–78. doi:10.1109/ASE51524.2021.9678600

    2021

  49. [60]

    Qihao Zhu, Daya Guo, Zhihong Shao, Dejian Yang, Peiyi Wang, Runxin Xu, Y Wu, Yukun Li, Huazuo Gao, Shirong Ma, et al. 2024. DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence. arXiv preprint arXiv:2406.11931 (2024). doi:10.48550/ARXIV.2406.119...

    2024 doi arXiv

  50. [2023]

    arXiv preprint arXiv:2304.11384 (2023)

    An Empirical Study on Using Large Language Models for Multi-Intent Comment Generation. arXiv preprint arXiv:2304.11384 (2023). doi:10.48550/ARXIV.2304.11384

    2023 doi arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.