Pith. sign in

REVIEW 3 cited by

Evaluating Large Language Models in Detecting Test Smells

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2407.19261 v2 pith:ND4AUJOX submitted 2024-07-27 cs.SE

Keila Lucas , Rohit Gheyi , Elvys Soares , Márcio Ribeiro , Ivan Machado This is my paper · ORCID
classification cs.SE
keywords smellstestlargetypesadvancedchatgpt-4detectinggemini
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Test smells are coding issues that typically arise from inadequate practices, a lack of knowledge about effective testing, or deadline pressures to complete projects. The presence of test smells can negatively impact the maintainability and reliability of software. While there are tools that use advanced static analysis or machine learning techniques to detect test smells, these tools often require effort to be used. This study aims to evaluate the capability of Large Language Models (LLMs) in automatically detecting test smells. We evaluated ChatGPT-4, Mistral Large, and Gemini Advanced using 30 types of test smells across codebases in seven different programming languages collected from the literature. ChatGPT-4 identified 21 types of test smells. Gemini Advanced identified 17 types, while Mistral Large detected 15 types of test smells. Conclusion: The LLMs demonstrated potential as a valuable tool in identifying test smells.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Model Context Protocol (MCP) Tool Descriptions Are Smelly! Towards Improving AI Agent Efficiency with Augmented MCP Tool Descriptions

    cs.SE 2026-02 conditional novelty 6.0 of 10

    Most MCP tool descriptions (97.1%) contain quality smells, and augmenting them improves agent success by a median of 5.85 percentage points at a 67.46% increase in execution steps.

  2. How Propense Are Large Language Models at Producing Code Smells? A Benchmarking Study

    cs.SE 2024-12 reject novelty 6.0 of 10

    A new benchmark scores LLMs' tendency to generate code smells by aggregating token-level probabilities over smell locations in existing code.

  3. Benchmarking LLM for Code Smells Detection: OpenAI GPT-4.0 vs DeepSeek-V3

    cs.SE 2025-04 reject novelty 4.0 of 10

    On a small synthetic four-language dataset, GPT-4.0 detects code smells with higher precision than DeepSeek-V3, while both models miss most annotated smells and the cost comparison is unreliable.

Pith tools