Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

Testing against documented intent catches 29 real business-logic bugs that code-centric tests miss.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 05:21 UTC pith:QF4TRV3L

load-bearing objection Likely real effect, but the headline numbers sit on a table inconsistency and an unreleased dataset—worth refereeing, not yet citable as settled. the 4 major comments →

arxiv 2604.23509 v2 pith:QF4TRV3L submitted 2026-04-26 cs.SE

Uncovering Business Logic Bugs via Semantics-Driven Unit Test Generation

Chen Yang , Junjie Chen This is my paper
classification cs.SE
keywords business logic bugssemantics-driven test generationproduct requirement documentsLLM-based unit teststest oraclefunctionality DSLbusiness scenariosGo
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that business-logic bugs — cases where code is syntactically valid but violates what the software is supposed to do — stay invisible to unit test generators that read only source code. SeGa converts product requirement documents into a structured knowledge base of "functionalities," retrieves the ones relevant to each method under test, and derives fine-grained "business scenarios" with explicit preconditions, actions, expected outcomes, and constraints. Those scenarios act as a semantic oracle that lets an LLM write tests asserting intended behavior, which can contradict the flawed implementation. In a study of 60 real bugs from four industrial Go projects, SeGa detected 29 bugs versus 4–7 for four LLM-based generators, at higher precision, and in six production repositories it surfaced 16 previously unknown bugs that developers fixed.

Core claim

SeGa's central claim is that the mismatch between implemented code and documented business semantics is discoverable at the unit level if the test generator is given the right slice of the requirements. The paper shows that raw PRDs are too noisy to use directly, but a two-level structured representation — functionality entries grouping related requirements, refined into concrete business scenarios with preconditions, triggering actions, expected outcomes, and semantic constraints — provides an actionable oracle. Using these scenarios, SeGa generated tests that exposed 29 of 60 real-world business logic bugs, compared to 4–7 for code-centric LLM generators, and during deployment uncovered 16

What carries the argument

The carrying mechanism is the business-scenario DSL: each scenario isolates one requirement from a retrieved functionality and states it as preconditions, a triggering action, expected outcomes, and structured semantic constraints. This turns a natural-language requirement into an executable oracle that can disagree with the implementation. Around it sits the functionality DSL knowledge base (which groups requirements under a business intent), an agent that summarizes a focal method's intent to retrieve relevant functionalities, and a standalone compilation-repair component that keeps generated tests executable.

Load-bearing premise

The load-bearing premise is that the product requirement documents are a complete, current, and unambiguous description of intended behavior; if a requirement is missing, stale, or open to multiple readings, SeGa's tests will assert the wrong behavior.

What would settle it

Take two sets of real business-logic bugs matched in complexity — one whose requirements are fully and unambiguously documented in the PRD, one whose requirements are undocumented or ambiguous. If SeGa's detection rate on the undocumented set falls to the level of the code-centric baselines, the advantage is an artifact of documentation quality, not of the semantic machinery.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If requirements are available, unit tests can be generated against intended behavior rather than implemented behavior, exposing a bug class that coverage-driven tools miss.
  • Retrieving only the requirements relevant to a focal method matters more than feeding the generator more semantic context.
  • A structured, scenario-level representation of requirements (preconditions, actions, outcomes, constraints) is what makes the oracle actionable for LLM-based test generation.
  • Standalone compilation repair improves the executability of LLM-generated tests without distracting the generation agent.
  • Deploying semantics-driven test generation in CI/CD can find business-logic bugs before merge, in code that already passes existing tests.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • SeGa's effectiveness is bounded by PRD quality; the same machinery could be repurposed to flag ambiguous, incomplete, or drifting requirements as a documentation-quality check.
  • The retrieval-plus-scenario pattern could generalize to other specification artifacts (API docs, regulatory rules) and other languages, provided a similar structured oracle can be built.
  • A natural extension is a feedback loop that uses developer accept/reject decisions on reported bugs to prune or correct future scenarios, reducing the documented false positives from oracle hallucination.
  • Comparing SeGa on bugs whose requirements are fully documented versus undocumented would isolate how much of the advantage comes from semantics versus test construction quality.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. SeGa is a semantics-driven unit-test generation technique for business logic bugs. It constructs a structured DSL knowledge base of "functionality entries" from product requirement documents (PRDs), uses an LLM-based semantic reasoning agent to retrieve the functionalities relevant to each focal method, refines them into business scenarios with explicit preconditions, actions, expected outcomes, and semantic constraints, and generates executable unit tests with a separate compilation-repair component. The evaluation on four industrial Go projects containing 60 real-world bugs reports that SeGa detects 29 bugs versus 7, 7, 6, and 4 for CHATTESTER, SymPrompt, HITS, and RATester, with aggregate precision 0.73 versus 0.54-0.57, plus an ablation study and a deployment on 6 production repositories that found 16 previously unknown bugs confirmed and fixed by developers. The paper also reports lessons learned about relevance, false positives, PRD completeness, oracle hallucination, and deployment in CI/CD.

Significance. If the quantitative claims hold, this is a valuable industrial experience paper for a practically important but under-served bug class. The design is not circular: the oracles come from external PRDs and the benchmark uses independently maintained bug annotations, and the paper includes direct comparisons, a component ablation, and a deployment study with developer confirmation. Those are real strengths. However, the headline numbers currently rest on an inconsistent subject table and on non-released proprietary data, so the central 29-vs-7/7/6/4 result is not yet independently verifiable from the manuscript. The paper's honest analysis of false-positive causes and its lessons learned are additional strengths, but the load-bearing evaluation data need to be made transparent before the result can be fully accepted.

major comments (4)
  1. [§4.1 / Table 1 / Table 2] Section 4.1 states that Subject-1..4 contain 12/9/37/2 bugs, while Table 1's Bugs column reads 37/9/2/12. These cannot both be correct. Cross-checking Table 2 with §5.1.2: SeGa has recall 0.42 on Subject-1 and is said to detect 5 bugs there, implying a denominator of 12, not 37; on Subject-3, recall 0.43 and 16 detected bugs imply a denominator of 37, not 2. Thus Table 1 appears to swap Subject-1 and Subject-4 bug counts. Please correct Table 1 and provide per-subject TP/FP/FN counts (ideally a confusion matrix) so that the aggregate 29 bugs and precision 0.73 can be recomputed. This is material because all reported numbers are small integers and the headline advantage over baselines is only 22-25 bugs.
  2. [§4.3.1 / §5.1.2] The TP/FP labels in RQ1 are assigned by "manual inspection" comparing failures with annotated bug ranges, but the paper does not say who performed the matching, whether they were blind to the technique, whether there was independent verification, or what inter-rater agreement was. In an industrial study with proprietary data, this creates a risk of labeling bias favoring the proposed method. Please report the labeling protocol, inter-rater reliability (or at least a second annotator's agreement), and per-subject confusion matrices. Without these, the precision comparison (0.73 versus 0.54-0.57) cannot be independently validated.
  3. [§4.2 / §4.4 / §7] CHATTESTER, SymPrompt, and HITS are adapted from Java/Python to Go, and RATester's local LLM calls are replaced; no adapted prompts, implementations, or artifacts are released. The statement that the authors "strictly followed" the original papers is not sufficient to rule out adaptation bias, especially because the weakness of the baselines is the central result. Please release the adapted code/prompts and report per-technique compilation success rates, numbers of generated and executed tests, and how the one-test-file-per-method configuration interacts with each baseline's intended workflow. Otherwise the 22-25 bug advantage may partly reflect weaker baseline configurations.
  4. [§5.1.2 / §6 / §7] The paper honestly documents that false positives arise from ambiguous, incomplete, or outdated PRDs and from "oracle hallucination," but it does not quantify how sensitive the central result is to PRD completeness. A failure analysis of the 31 missed bugs is needed: how many were missed because the PRD lacked or obscured the relevant requirement, versus failures in retrieval, scenario derivation, test generation, or execution? Such an analysis would clarify the scope of the claimed advantage and directly inform the lessons in Section 6.
minor comments (6)
  1. [Abstract / §1 / §9] The claim that precision improves by 26.9%-34.3% does not match the rounded aggregate values in Table 2: relative improvements are about 28.1%, 32.7%, and 35.2% against 0.57, 0.55, and 0.54, and the absolute improvement is 18-19 percentage points. Please state whether the percentages are relative or absolute and recompute them consistently.
  2. [Figure 3] The overlap diagram is hard to read: the repeated zero labels and ambiguous intersections make it difficult to verify the 29/7/7/6/4 counts. Please replace or supplement it with a table of per-technique detected-bug IDs or a standard Venn-diagram data table.
  3. [Table 1] The subject identifiers in Table 1 use inconsistent capitalization (subject-1 vs Subject-1) and the Bugs column is inconsistent with Section 4.1. Also, the "Requirements" column (e.g., 106 requirements for a 115,048-token PRD) seems surprisingly coarse; clarify the granularity of a "requirement" in this table.
  4. [§7] Typo: "realatively" should be "relatively" in the overhead discussion.
  5. [References [18] and [42]] References [18] and [42] are cited as prior work for designing the functionality DSL, but the cited titles appear to concern binary lifting and FFT semantics lifting, which seem unrelated to requirements engineering DSLs. Please verify and replace these citations.
  6. [§5.3.2] The statement that buggy code changes involve "91.35 lines on average" is overly precise for a small sample; report the median and range as well, or round to a meaningful precision.

Circularity Check

0 steps flagged

No significant circularity: SeGa's oracle comes from external PRDs and the evaluation uses independently annotated, developer-confirmed bugs.

full rationale

SeGa's central claim (29 detected bugs vs 7/7/6/4 for baselines; precision 0.73) is not circular. The oracles are external: functionality entries and business scenarios are derived from product requirement documents (Sections 3.1 and 3.2.2), and the motivating example explicitly shows the PRD overriding the implemented behavior. The ground truth is independent: Section 4.1 states each bug is 'precisely annotated' and reviewed by developers, and RQ3 counts only issues 'confirmed and subsequently fixed by the developers' (Section 5.3). No parameter is fitted to the benchmark, and the compared techniques are not derived from SeGa's outputs. The only mild concern is that Section 3.2.2 says scenario derivation 'jointly analyzes the focal method's code context together with the retrieved functionality,' so an LLM could in principle leak implemented behavior into expected outcomes; however, no specific reduction is demonstrated, the paper's example shows the PRD side overriding the implementation, and Section 6 candidly lists oracle hallucination and PRD ambiguity as false-positive sources rather than using them to manufacture success. The inconsistent bug counts between Section 4.1 and Table 1 are a serious reporting/correctness issue, but they do not constitute circularity. Self-citations appear only in related work and implementation conventions and are not load-bearing.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

SeGa introduces no new physical or domain entities; its DSL schemas are part of the proposed method rather than independent postulated objects. The central free parameters are engineering bounds chosen by the authors; no target bug-detection outcome is fitted into a parameter.

free parameters (4)
  • Agent interaction budget = 50
    Section 3.3.1 caps the semantic reasoning/test generation agent at 50 interactions; chosen by hand, affects how much context the agent can gather and thus bug detection.
  • Compilation repair attempts = 3
    Section 3.3.2 stops the repair loop after three attempts; an engineering bound that affects executability of generated tests.
  • LLM temperature = 0
    Section 7 sets temperature to zero to reduce randomness; a control choice, not fitted to the bug-detection outcome.
  • Test files per focal method = 1
    Section 4.4 configures every technique to generate one test file per focal method for fairness; this bound influences measured recall and precision.
axioms (4)
  • domain assumption PRDs contain and accurately express the intended business semantics required for detecting business logic bugs.
    Section 2.1 defines business semantics as sourced from PRDs; Sections 5.1.2 and 6 note that incomplete/ambiguous PRDs cause false positives, so the central claim depends on PRD quality.
  • domain assumption A generated test failure is a true business-logic bug only if human matching maps it to an annotated ground-truth bug.
    Section 4.3.1 defines TP/bug matching via manual inspection; no inter-rater reliability or blinding is reported, so ground-truth matching is an unverified assumption.
  • domain assumption The adapted Go implementations of CHATTESTER, SymPrompt, and HITS faithfully represent the original techniques.
    Section 4.4 states the authors adapted public implementations following the original papers, but no released port artifact is available for independent validation.
  • domain assumption The proprietary repositories used in the study are not present in the LLM's training data.
    Section 7 says the industrial partner confirmed exclusion; this is not independently verifiable and matters for data-leakage interpretation.

pith-pipeline@v1.3.0-alltime-deepseek · 19400 in / 15438 out tokens · 146512 ms · 2026-08-04T05:21:29.994927+00:00 · methodology

0 comments
read the original abstract

Business logic bugs violate intended business semantics and are particularly prevalent in enterprise software. Yet most existing unit test generation techniques are code-centric, making such bugs difficult to expose. We present SeGa, a semantics-driven unit test generation technique for uncovering business logic bugs. SeGa constructs a semantic knowledge base from product requirement documents, represented as a set of functionality entries that group related requirements under a common business intent. Given a focal method, SeGa retrieves the relevant functionality entries and derives fine-grained business scenarios with explicit preconditions, triggering actions, expected outcomes, and semantic constraints to guide LLM-based test generation. We evaluate SeGa on four industrial Go projects containing 60 real-world business logic bugs. SeGa detects 22-25 more bugs than four state-of-the-art LLM-based techniques and improves precision by 26.9%-34.3%. Deployment across 6 production repositories further uncovers 16 previously unknown business logic bugs that were confirmed and fixed by developers. From our industrial study, we summarize a series of lessons and suggestions for practical use and future research.

Figures

Figures reproduced from arXiv: 2604.23509 by Chen Yang, Junjie Chen.

Figure 1
Figure 1. Figure 1: Motivating Example 2.2 Motivating Example view at source ↗
Figure 2
Figure 2. Figure 2: Overview of SeGa concerned only with the Item Operation Management functionality. When all functionalities are provided as context, LLM-based generators (e.g., CHATTESTER, SymPrompt, HITS, and RATester) are easily distracted by irrelevant information and generate tests that either miss the bug or trigger spurious failures. This motivates a semantic retrieval mechanism that identifiesthe specific functional… view at source ↗
Figure 3
Figure 3. Figure 3: illustrates the overlap of bugs detected by different techniques. From the figure, SeGa identifies 29 bugs in total, while CHATTESTER, SymPrompt, HITS, and RATester detect 7, 7, 6, and 4 bugs, respectively. Notably, SeGa covers nearly all bugs found by the other techniques (missing only one uniquely detected by SymPrompt) and also discovers the largest number of bugs that no baseline can find. This highlig… view at source ↗
Figure 4
Figure 4. Figure 4: Simplified and desensitized versions of two previously unknown business logic bugs view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Context Matters: Improving the Practical Reliability of LLM-Based Unit Test Generation

    cs.SE 2026-07 conditional novelty 6.0

    CATGen improves LLM unit-test reliability by combining structured project-context retrieval, deterministic test-class skeletons, and static-analysis repair, beating six baselines on compilation success, coverage, and cost.

Reference graph

Works this paper leans on

43 extracted references · 11 linked inside Pith · cited by 1 Pith paper

  1. [1]

    LangChain

    2026. LangChain. https://www.langchain.com/

    2026

  2. [2]

    Shreya Bhatia, Tarushi Gandhi, Dhruv Kumar, and Pankaj Jalote. 2024. System test case design from requirements specifications: Insights and challenges of using chatgpt.arXiv preprint arXiv:2412.03693(2024)

    Pith/arXiv arXiv 2024

  3. [3]

    Xiang Cheng, Fan Sang, Yizhuo Zhai, Xiaokuan Zhang, and Taesoo Kim. 2025. RUG: Turbo LLM for Rust Unit Test Generation. In2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE). IEEE Computer Society, 634–634

    2025

  4. [4]

    Bei Chu, Yang Feng, Kui Liu, Hange Shi, Zifan Nan, Zhaoqiang Guo, and Baowen Xu. 2025. Synergizing Program Analysis and LLMs to Enhance Rust Unit Test Coverage. (2025)

    2025

  5. [5]

    Elizabeth Dinella, Gabriel Ryan, Todd Mytkowicz, and Shuvendu K Lahiri. 2022. Toga: A neural method for test oracle generation. InProceedings of the 44th International Conference on Software Engineering. 2130–2141

    2022

  6. [6]

    FRET Tool Documentation. 2026. Formal Requirements Elicitation Tool (FRET). https://en.wikipedia.org/wiki/FRET_ (software)

    2026

  7. [7]

    Gordon Fraser and Andrea Arcuri. 2011. Evosuite: automatic test suite generation for object-oriented software. InProceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering. 416–419

    2011

  8. [8]

    Sujin Jang, Yeonhee Ryou, Heewon Lee, and Kihong Heo. 2025. UnitCon: Synthesizing Targeted Unit Tests for Java Runtime Exceptions.Proceedings of the ACM on Software Engineering2, FSE (2025), 2053–2074

    2025

  9. [9]

    René Just, Darioush Jalali, and Michael D Ernst. 2014. Defects4J: A database of existing faults to enable controlled testing studies for Java programs. InProceedings of the 2014 international symposium on software testing and analysis. 437–440

    2014

  10. [10]

    Caroline Lemieux, Jeevana Priya Inala, Shuvendu K Lahiri, and Siddhartha Sen. 2023. Codamosa: Escaping coverage plateaus in test generation with pre-trained large language models. In2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE). IEEE, 919–931

    2023

  11. [11]

    Dianshu Liao, Xin Yin, Shidong Pan, Chao Ni, Zhenchang Xing, and Xiaoyu Sun. 2025. Navigating the Labyrinth: Path-Sensitive Unit Test Generation with Large Language Models. (2025)

    2025

  12. [12]

    Yun Lin, You Sheng Ong, Jun Sun, Gordon Fraser, and Jin Song Dong. 2021. Graph-based seed object synthesis for search-based unit testing. InProceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 1068–1080

    2021

  13. [13]

    Jinwei Liu, Chao Li, Rui Chen, Shaofeng Li, Bin Gu, and Mengfei Yang. 2025. STRUT: Structured Seed Case Guided Unit Test Generation for C Programs using LLMs.Proceedings of the ACM on Software Engineering2, ISSTA (2025), 2113–2135

    2025

  14. [14]

    Andrea Lops, Fedelucio Narducci, Azzurra Ragone, and Michelantonio Trizio. 2024. AgoneTest: Automated creation and assessment of Unit tests leveraging Large Language Models. InProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering. 2440–2441

    2024

  15. [15]

    Stephan Lukasczyk and Gordon Fraser. 2022. Pynguin: Automated unit test generation for python. InProceedings of the ACM/IEEE 44th International Conference on Software Engineering: Companion Proceedings. 168–172

    2022

  16. [16]

    Nikitha Medeshetty, Ahmad Nauman Ghazi, Sadi Alawadi, and Fahed Alkhabbas. 2025. From Requirements to Test Cases: An NLP-Based Approach for High-Performance ECU Test Case Automation.arXiv preprint arXiv:2505.00547 (2025)

    Pith/arXiv arXiv 2025

  17. [17]

    Zifan Nan, Zhaoqiang Guo, Kui Liu, and Xin Xia. 2025. Test intention guided llm-based unit test generation. In2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE). IEEE Computer Society, 779–779

    2025

  18. [18]

    Nico Naus, Freek Verbeek, Sagar Atla, and Binoy Ravindran. 2024. Poster: Formally Verified Binary Lifting to P-Code. InProceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security. 4973–4975

    2024

  19. [19]

    Carlos Pacheco and Michael D Ernst. 2007. Randoop: feedback-directed random testing for Java. InCompanion to the 22nd ACM SIGPLAN conference on Object-oriented programming systems and applications companion. 815–816

    2007

  20. [20]

    Lilly Raamesh and GV Uma. 2010. Reliable mining of automatically generated test cases from software requirements specification (SRS).arXiv preprint arXiv:1002.1199(2010)

    Pith/arXiv arXiv 2010

  21. [21]

    Sanjai Rayadurgam and Mats Per Erik Heimdahl. 2001. Test-sequence generation from formal requirement models. InProceedings Sixth IEEE International Symposium on High Assurance Systems Engineering. Special Topic: Impact of Networking. IEEE, 23–31

    2001

  22. [22]

    Gabriel Ryan, Siddhartha Jain, Mingyue Shang, Shiqi Wang, Xiaofei Ma, Murali Krishna Ramanathan, and Baishakhi Ray

  23. [23]

    ByteDance Seed, Yuyu Zhang, Jing Su, Yifan Sun, Chenguang Xi, Xia Xiao, Shen Zheng, Anxiang Zhang, Kaibo Liu, Daoguang Zan, et al. 2025. Seed-coder: Let the code model curate data for itself.arXiv preprint arXiv:2506.03524(2025). , Vol. 1, No. 1, Article . Publication date: April 2026. Uncovering Business Logic Bugs via Semantics-Driven Unit Test Generati...

    Pith/arXiv arXiv 2025

  24. [24]

    Richa Sharma and KK Biswas. 2014. Automated generation of test cases from logical specification of software requirements. In2014 9th International Conference on Evaluation of Novel Approaches to Software Engineering (ENASE). IEEE, 1–8

    2014

  25. [25]

    Jiho Shin, Sepehr Hashtroudi, Hadi Hemmati, and Song Wang. 2024. Domain adaptation for code model-based unit test case generation. InProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis. 1211–1222

    2024

  26. [26]

    Phil Stocks and David Carrington. 1993. Test template framework: A specification-based testing case study. In Proceedings of the 1993 ACM SIGSOFT international symposium on Software testing and analysis. 11–18

    1993

  27. [27]

    Chunhui Wang, Fabrizio Pastore, Arda Goknil, and Lionel C Briand. 2020. Automatic generation of acceptance test cases from use case specifications: an nlp-based approach.IEEE Transactions on Software Engineering48, 2 (2020), 585–616

    2020

  28. [28]

    Zejun Wang, Kaibo Liu, Ge Li, and Zhi Jin. 2024. HITS: High-coverage LLM-based Unit Test Generation via Method Slicing.arXiv preprint arXiv:2408.11324(2024)

    Pith/arXiv arXiv 2024

  29. [29]

    Ratnadira Widyasari, Sheng Qin Sim, Camellia Lok, Haodi Qi, Jack Phan, Qijin Tay, Constance Tan, Fiona Wee, Jodie Ethelda Tan, Yuheng Yieh, et al . 2020. Bugsinpy: a database of existing bugs in python programs to enable controlled testing and debugging studies. InProceedings of the 28th ACM joint meeting on european software engineering conference and sy...

    2020

  30. [30]

    Chunqiu Steven Xia, Matteo Paltenghi, Jia Le Tian, Michael Pradel, and Lingming Zhang. 2024. Fuzz4all: Univer- sal fuzzing with large language models. InProceedings of the IEEE/ACM 46th International Conference on Software Engineering. 1–13

    2024

  31. [31]

    Zhuokui Xie, Yinghao Chen, Chen Zhi, Shuiguang Deng, and Jianwei Yin. 2023. ChatUniTest: a ChatGPT-based automated unit test generation tool.arXiv preprint arXiv:2305.04764(2023)

    Pith/arXiv arXiv 2023

  32. [32]

    Chen Yang, Junjie Chen, Bin Lin, Ziqi Wang, and Jianyi Zhou. 2024. Advancing Code Coverage: Incorporating Program Analysis with Large Language Models.ACM Transactions on Software Engineering and Methodology(2024)

    2024

  33. [33]

    Chen Yang, Ziqi Wang, Yanjie Jiang, Lin Yang, Yuteng Zheng, Jianyi Zhou, and Junjie Chen. 2025. Reflective Unit Test Generation for Precise Type Error Detection with Large Language Models. InProceedings of the 40th IEEE/ACM International Conference on Automated Software Engineering

    2025

  34. [34]

    Chen Yang, Ziqi Wang, Lin Yang, Dong Wang, Shutao Gao, Yanjie Jiang, and Junjie Chen. 2026. WiseUT: An Intelligent Framework for Unit Test Generation. In2026 IEEE/ACM 48th International Conference on Software Engineering: Companion Proceedings (ICSE-Companion)

    2026

  35. [35]

    Chen Yang, Lin Yang, Ziqi Wang, Dong Wang, Jianyi Zhou, and Junjie Chen. 2025. Clarifying Semantics of In-Context Examples for Unit Test Generation. InProceedings of the 40th IEEE/ACM International Conference on Automated Software Engineering

    2025

  36. [36]

    Lin Yang, Chen Yang, Shutao Gao, Weijing Wang, Bo Wang, Qihao Zhu, Xiao Chu, Jianyi Zhou, Guangtai Liang, Qianxiang Wang, et al. 2024. On the evaluation of large language models in unit test generation. InProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering. 1607–1619

    2024

  37. [37]

    Zhenzhen Yang, Rubing Huang, Chenhui Cui, Nan Niu, and Dave Towey. 2025. Requirements-based test generation: A comprehensive survey.ACM Transactions on Software Engineering and Methodology(2025)

    2025

  38. [38]

    Xin Yin, Chao Ni, Xinrui Li, Liushan Chen, Guojun Ma, and Xiaohu Yang. 2025. Enhancing LLM’s Ability to Generate More Repository-Aware Unit Tests Through Precise Contextual Information Injection.arXiv preprint arXiv:2501.07425 (2025)

    Pith/arXiv arXiv 2025

  39. [39]

    Xin Yin, Chao Ni, Xiaodan Xu, and Xiaohu Yang. 2024. What you see is what you get: Attention-based self-guided automatic unit test generation.arXiv preprint arXiv:2412.00828(2024)

    Pith/arXiv arXiv 2024

  40. [40]

    Zhiqiang Yuan, Yiling Lou, Mingwei Liu, Shiji Ding, Kaixin Wang, Yixuan Chen, and Xin Peng. 2023. No more manual tests? evaluating and improving chatgpt for unit test generation.arXiv preprint arXiv:2305.04207(2023)

    Pith/arXiv arXiv 2023

  41. [41]

    Junwei Zhang, Xing Hu, Shan Gao, Xin Xia, David Lo, and Shanping Li. 2025. Less is More: On the Importance of Data Quality for Unit Test Generation.arXiv preprint arXiv:2502.14212(2025)

    Pith/arXiv arXiv 2025

  42. [42]

    Naifeng Zhang, Sanil Rao, Mike Franusich, and Franz Franchetti. 2025. Towards Semantics Lifting for Scientific Computing: A Case Study on FFT.arXiv preprint arXiv:2501.09201(2025). , Vol. 1, No. 1, Article . Publication date: April 2026

    Pith/arXiv arXiv 2025

  43. [2024]

    Code-aware prompting: A study of coverage-guided test generation in regression setting using llm.Proceedings of the ACM on Software Engineering1, FSE (2024), 951–971

    2024