The reviewed record of science sign in
Pith

arxiv: 2402.09171 · v1 · pith:UCSDDPYY · submitted 2024-02-14 · cs.SE

Automated Unit Test Improvement using Large Language Models at Meta

Nadia Alshahwan , Jubin Chheda , Anastasia Finegenova , Beliz Gokkaya , Mark Harman , Inna Harper , Alexandru Marginean , Shubho Sengupta
show 1 more author
Eddy Wang
This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel pith:UCSDDPYYrecord.jsonopen to challenge →

classification cs.SE
keywords metatesttestgen-llmdeploymentimprovementinstagramclassescode
0
0 comments X
read the original abstract

This paper describes Meta's TestGen-LLM tool, which uses LLMs to automatically improve existing human-written tests. TestGen-LLM verifies that its generated test classes successfully clear a set of filters that assure measurable improvement over the original test suite, thereby eliminating problems due to LLM hallucination. We describe the deployment of TestGen-LLM at Meta test-a-thons for the Instagram and Facebook platforms. In an evaluation on Reels and Stories products for Instagram, 75% of TestGen-LLM's test cases built correctly, 57% passed reliably, and 25% increased coverage. During Meta's Instagram and Facebook test-a-thons, it improved 11.5% of all classes to which it was applied, with 73% of its recommendations being accepted for production deployment by Meta software engineers. We believe this is the first report on industrial scale deployment of LLM-generated code backed by such assurances of code improvement.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. RepairAgent: An Autonomous, LLM-Based Agent for Program Repair

    cs.SE 2024-03 conditional novelty 8.0

    RepairAgent autonomously repairs 164 bugs on Defects4J including 39 not fixed by prior techniques by treating an LLM as an agent that invokes tools via a finite state machine and dynamic prompts.

  2. Mutation-Guided Unit Test Generation with a Large Language Model

    cs.SE 2025-06 conditional novelty 6.0

    MUTGEN incorporates mutation feedback into LLM prompts and uses iteration to generate unit tests that achieve higher mutation scores than EvoSuite or vanilla LLM prompting on 204 benchmark subjects.

  3. Large Language Models: A Survey

    cs.CL 2024-02 accept novelty 3.0

    The paper surveys key large language models, their training methods, datasets, evaluation benchmarks, and future research directions in the field.