Pith. sign in

REVIEW 10 cited by

CoverUp: Effective High Coverage Test Generation for Python

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2403.16218 v4 pith:JMSQMWEP submitted 2024-03-24 cs.SE cs.AIcs.LGcs.PL

Juan Altmayer Pizzorno , Emery D. Berger This is my paper · ORCID
classification cs.SEcs.AIcs.LGcs.PL
keywords coverupcoveragetestbranchgenerationlinepythontests
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Testing is an essential part of software development. Test generation tools attempt to automate the otherwise labor-intensive task of test creation, but generating high-coverage tests remains challenging. This paper proposes CoverUp, a novel approach to driving the generation of high-coverage Python regression tests. CoverUp combines coverage analysis, code context, and feedback in prompts that iteratively guide the LLM to generate tests that improve line and branch coverage. We evaluate our prototype CoverUp implementation across a benchmark of challenging code derived from open-source Python projects and show that CoverUp substantially improves on the state of the art. Compared to CodaMosa, a hybrid search/LLM-based test generator, CoverUp achieves a per-module median line+branch coverage of 80% (vs. 47%). Compared to MuTAP, a mutation- and LLM-based test generator, CoverUp achieves an overall line+branch coverage of 89% (vs. 77%). We also demonstrate that CoverUp's performance stems not only from the LLM used but from the combined effectiveness of its components.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. PROGRESS: Property-Guided Regression Search for Semantic Falsification

    cs.SE 2026-07 conditional novelty 7.0 of 10

    PROGRESS embeds LLM-generated jqwik properties as search objectives in EvoSuite's DynaMOSA, detecting 58% of injected Java bugs that regression assertions miss and satisfying hard preconditions 4x more often than stan...

  2. Sakura: An Approach for Generating Complex Tests from Natural Language Test Descriptions

    cs.SE 2026-05 unverdicted novelty 7.0 of 10

    Sakura is a multi-agent system that generates structurally complex tests from NL descriptions, achieving 50-78% higher compilability and 38-66% higher coverage overlap than baselines on 1,464 scenarios from 20 Apache ...

  3. From Resource Flow to Executable Tests: Petri-Net-Guided LLM Test Generation for Concurrent Stateful Rust APIs

    cs.SE 2026-07 conditional novelty 6.0 of 10

    Petri-net models of Rust APIs can guide LLMs to produce executable concurrent tests whose structure stays faithful to the modeled resource flow.

  4. Multi-Agent LLM Collaboration for Unit Test Generation via Human-Testing-Inspired Workflows

    cs.SE 2026-07 conditional novelty 6.0 of 10

    A three-agent LLM workflow plus a test-specialized knowledge graph generates unit tests that beat prior LLM and search-based tools on coverage and mutation score.

  5. On the risk of coding before testing: An empirical study on LLM-based test generation workflow

    cs.SE 2026-07 conditional novelty 6.0 of 10

    Generating LLM tests after faulty LLM code cuts fault detection to about 14% versus about 25% when tests are generated from the task description alone, across models and prompting strategies.

  6. Hamster: A Large-Scale Study and Characterization of Developer-Written Tests

    cs.SE 2025-09 conditional novelty 6.0 of 10

    A large-scale study of 1.7M developer-written Java tests shows they commonly use fixtures, mocking, structured inputs, and multi-class scope, features largely missing from automated test generation tools.

  7. Combining Type Inference and Automated Unit Test Generation for Python

    cs.SE 2025-07 conditional novelty 6.0 of 10

    A proxy-based type tracing technique integrated into Pynguin infers parameter and return types during test generation and improves branch coverage.

  8. How well LLM-based test generation techniques perform with newer LLM versions?

    cs.SE 2026-01 conditional novelty 5.0 of 10

    With newer LLMs, a plainly prompted generation loop matches or beats four engineered test-generation tools on coverage and mutation score, and a class-then-method hybrid cuts LLM queries by about 20%.

  9. LLM-Based Agentic Systems for Software Engineering: Challenges and Opportunities

    cs.SE 2026-01 unverdicted novelty 2.0 of 10

    A survey of LLM-based multi-agent systems across the software development life cycle, plus a research agenda for orchestration, human coordination, cost, and data.

  10. A Contemporary Survey of Large Language Model Assisted Program Analysis

    cs.SE 2025-02 conditional novelty 1.0 of 10

    A review that catalogs how large language models are used in static, dynamic, and hybrid program analysis, and outlines open challenges.

Pith tools