Pith. sign in

REVIEW 5 major objections 6 minor 32 references

Investigating the Performance of Small Language Models in Detecting Test Smells in Manual Test Cases

T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Small language models can detect test smells in manual, natural-language test cases, with Phi-4 reaching pass@2 of 97%.

desk verdict The reported pass@2 numbers are recall-like: with a positive-only dataset and a per-smell prompt, always answering YES would score 100%; the paper still has value as a benchmark artifact if the evaluation is fixed. read the letter →

arxiv 2507.13035 v1 pith:Y4WKDOIS submitted 2025-07-17 cs.SE

Keila Lucas , Rohit Gheyi , Márcio Ribeiro , Fabio Palomba , Luana Martins , Elvys Soares This is my paper · ORCID
classification cs.SE
keywords testsmellssmalllanguagemodelsmanualtestingnaturalcasesdirectionalstimuluspromptingPhi-4pass@kmeta
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper attempts to show that small language models—models small enough to run on a local machine—can detect test smells in manual, natural-language test cases, a task previously handled by hand-written rules or cloud-based large models. The authors prompt three models with the conceptual definition of each smell and ask whether a given test sentence exhibits it, scoring the answer against a manually curated set of 261 sentences from real Ubuntu tests. They report that Phi-4 reaches 97% pass@2 accuracy, Gemma3 and Llama3.2 around 91%, and that the models also provide explanations and sometimes refactoring suggestions without being asked. If the result holds, testing teams could screen manual test descriptions for ambiguity, redundancy, and missing checks at low cost and without sending proprietary test data to a cloud service.

What carries the argument

Directional Stimulus Prompting (DSP) is the mechanism: each query embeds a conceptual definition of one test smell, an instruction to judge the sentence against that definition, and the test sentence itself; the model then answers yes or no with a justification. Scoring uses pass@k, the probability that at least one of k sampled responses is correct, and semantic consistency measures stability across repeated runs. The same prompt structure carries the argument because it converts an open detection problem into a per-concept classification, and changing the definition in the prompt is what lets a model handle a new smell without writing new rules.

What would settle it

Run the same three models on a set that mixes the 261 annotated sentences with an equal number of clean sentences (no smell) and some multi-smell sentences, and score a response as correct only when the model names exactly the annotated smell and says no for clean sentences; a large drop in pass@2 under this stricter scoring would show the original numbers were inflated by a yes-bias.

Watch

Extended reading notes

Core claim

The central claim is that three small language models run locally—Phi-4, Gemma3, and Llama3.2—can identify test smells in natural-language manual test sentences when prompted with the conceptual definition of the smell, without rule lists or syntactic analyzers. On 261 sentences curated from Ubuntu manual test cases, with each sentence assigned one of seven smells, Phi-4 reaches pass@2 of 97%, Gemma3 91.1%, and Llama3.2 91.0%; Phi-4 is also the most stable across temperature settings and has the highest semantic consistency, at 90%. The paper further reports that the models spontaneously give explanations and sometimes suggest refactorings, and that a Meta Prompting refinement recovers all six missed Ambiguous Test and all eleven missed Eager Action sentences in the examined subset.

Load-bearing premise

The load-bearing premise is that the manually curated annotations are a correct and complete oracle—every sentence contains exactly one of the seven smells, and a response is correct when its justification matches that annotation—so noisy labels or unreported false positives on the other six prompts would change the reported pass@2 values.

Editorial extensions

If this is right

  • Manual test suites can be screened locally: a developer could run Phi-4 against natural-language test steps and get a smell flag, a justification, and a refactoring hint in most cases.
  • Detection becomes concept-driven rather than tool-bound: adding a new smell or translating the catalog only requires editing the prompt's definition, not rebuilding a rule engine.
  • Lighter models remain useful: Gemma3 and Llama3.2 reach roughly 91% at a fraction of Phi-4's parameter count, which matters for memory- or cost-constrained environments.
  • Meta Prompting can recover missed detections: model-generated prompt structures fixed all examined Ambiguous Test and Eager Action false negatives, suggesting accuracy gains are available without model changes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The per-smell scoring means each sentence is tested against seven prompts, but only the prompt for the annotated smell determines a hit; false positives on the other six are unreported, so real-world precision is probably lower than the pass@2 numbers suggest.
  • Because the dataset is a single operating system's manually written tests, the claimed accuracy is best read as an upper bound for heterogeneous real-world suites; smells that depend on surrounding sentences, such as Misplaced Action and Misplaced Verification, are explicitly harder for single-sentence prompts.
  • The spontaneous refactoring suggestions are an interesting byproduct, but their usefulness is untested; a study where practitioners accept or reject the suggestions would convert this qualitative advantage into a measurable one.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper evaluates three small language models (Gemma3, Llama3.2, and Phi-4) on detecting seven test smells in 261 natural-language sentences extracted from 143 Ubuntu manual test cases. Using Directional Stimulus Prompting, the authors report pass@2 values of 97% for Phi-4 and about 91% for Gemma3 and Llama3.2, along with the observation that the models sometimes provide explanations and improvement suggestions. The paper also explores a meta-prompting refinement on a subset of previously missed sentences. The central claim is that locally run small language models can reproduce the curated smell annotations on this dataset at high rates, offering a low-cost and privacy-preserving alternative to rule-based and cloud-LLM approaches.

Significance. If the reported detection numbers were discriminative, this would be a useful practical contribution: a lightweight, local alternative for detecting natural-language test smells, with the added value of model-generated explanations and refactoring hints. The study has concrete strengths: it uses a real-world dataset, covers a published seven-smell catalog, reports an 11-temperature sweep, and makes the generated responses available as artifacts. However, the core quantitative claim is currently stronger than the evaluation supports. Because the dataset is positive-only and success is scored only against the single annotated smell, the headline pass@2 values are recall-like agreement measures rather than detection accuracy; a degenerate yes-to-all model would obtain 100% on the metric as defined. The per-smell results also contain very small cells, the meta-prompting section evaluates a selected false-negative subset without a control, and no baseline comparison with existing NLP-based detectors is provided. These issues are load-bearing but fixable with additional analysis, so the paper has a defensible core that needs substantial supplementary evaluation.

major comments (5)
  1. [Section 2 (Test smell detection workflow) and Section 3, Table 4] The evaluation protocol makes the reported pass@2 values recall-like rather than detection accuracy. All 261 sentences are annotated with exactly one smell (Table 1), the prompts in Figure 2 are applied per smell, and Section 4.2 explicitly states that all sentences are affected by test smells. A response is counted correct only when the model's justification matches the annotated smell; a 'yes' answer on any of the other six smell prompts is never scored as an error. Under this protocol, a degenerate model that answers 'yes' to all seven prompts for every sentence would achieve 100% on the metric as defined. The paper reports no precision, F1, false-positive counts, or the distribution of yes/no responses across the seven prompts. Please report these quantities, add negative-control sentences, and re-frame the claims (e.g., as recall with respect to the annotated smell), or the main quantitative contribution is not established.
  2. [Section 3, Table 4] The per-smell scores are based on very small cells: Conditional Test has 9 instances, Misplaced Action has 8, and Misplaced Precondition has 5. For example, Phi-4's 20.0% on Misplaced Precondition corresponds to 1 of 5 correct responses while Gemma3 and Llama3.2 show 100.0%. Without confidence intervals or a significance test, the aggregate claim that Phi-4 'achieved the best results' has no uncertainty quantification. Please report binomial confidence intervals (or an equivalent) and test whether the cross-model differences are statistically meaningful, especially for the small smell categories.
  3. [Section 4.4] The meta-prompting analysis does not currently support the conclusion that 'The Meta Prompting strategy enhances the detection of test smells.' Phi-4 generates the refined prompt and is then evaluated only on 19 sentences that were previously False Negatives, with no comparison against the original DSP prompt on the same subset and no held-out evaluation. The reported 6/6 and 11/11 success rates are therefore compatible with regression toward the mean on a selected subset. Please present this section as exploratory, or add a controlled comparison on a fixed evaluation set.
  4. [Section 4.3] The authors acknowledge in Section 4.3 that Misplaced Action, Misplaced Verification, and Misplaced Precondition require cross-sentence context and cannot be reliably identified from an isolated sentence, yet the evaluation in Section 3 applies all seven smells sentence-by-sentence. This is an internal inconsistency in the evaluation design. Either evaluate these three smell types with the full test case as context, or exclude them from the aggregate accuracy; otherwise the 97% headline includes a class of smells that the authors themselves argue is outside the scope of the input representation used.
  5. [Section 5 (Related Work) and Section 2] The empirical comparison lacks a baseline. The related work reports an F-measure of 83.7% for the Manual Test Alchemist tool (Aranda et al.) and describes keyword-based detection with NALABS, but the paper does not compare the SLMs against these existing natural-language test smell detectors. A precision/recall-style comparison against at least one existing tool is needed to support the claim that SLMs are an efficient alternative to rule-based and syntactic detection approaches rather than an uncalibrated point estimate.
minor comments (6)
  1. [Section 3, Table 4] The table header 'Accuracy (pass@2)' is misleading because pass@2 is a probability of at least one correct sample in two generations, not an accuracy measure. Please clarify the terminology and also state whether the temperature-sweep numbers in Section 3 use pass@1 or pass@2 and how the 'best temperature' was selected for the main results.
  2. [Section 4.1] In the 'Example of Phi-4 explanation with DSP' box, the sentence 'The machine logins successfully' is presented in a discussion of ambiguity, but the model's suggested output focuses on expected outcomes. Please indicate which test smell this sentence is annotated with and how this response was scored.
  3. [Section 2 and Section 4.5] The response analysis relies on two authors manually reviewing model outputs, but the paper does not state whether the yes/no decision was extracted automatically or assigned manually. This distinction matters for reproducibility; please describe the scoring procedure precisely.
  4. [Section 3] The Semantic Consistency metric is introduced without a formal definition. Please specify how SC is computed (e.g., exact-match on repeated outputs, semantic similarity, or reviewer judgment) and what threshold, if any, is used to interpret the reported percentages.
  5. [Table 3 and Section 6] There are small typos: 'Out/2024' in Table 3 should likely be 'Oct/2024', 'LLama' in Section 6 should be 'Llama3.2', and 'testcase1600_Ristretto' in Section 2 should have a space. Please also check the citation for the Meta Prompting reference, since [8] appears to be a different prompting method from the one described.
  6. [Section 3 and Section 4.2] The counts reported in the text are not fully reconciled: Section 3 says 253 sentences were detected by at least one model and 78 by all three, while Section 4.2 says Llama3.2 generated 80 negative responses. Please report these numbers in a table or clarify how they relate to the 261-sentence dataset.

Circularity Check

1 steps flagged · score 2.0 of 10

No load-bearing circularity: the headline pass@2 numbers are an empirical evaluation against an externally curated dataset, and the only self-referential step is the exploratory Meta Prompting re-evaluation, which does not drive the central claims.

  1. other [Section 4.4, 'Can We Do It Better? The Role of Meta Prompting']
    "We employed Meta Prompting to re-evaluate 19 sentences using the Phi-4 model. The model detected Ambiguous Test in six sentences (6/6) and Eager Action in 11 sentences (11/11). The re-evaluated sentences were False Negatives that the model had previously missed the smells with DSP."

    This is an in-sample self-check rather than an independent evaluation. The prompt is generated by Phi-4, the responses are produced by Phi-4, and the 19 target sentences are selected precisely because Phi-4 had previously failed on them. The reported 6/6 and 11/11 successes therefore do not provide independent evidence that Meta Prompting improves detection; the evaluation set and the prompt both come from the same model being evaluated. However, the section is explicitly exploratory and is not used to derive the paper's headline pass@2 figures.

full rationale

The central claim of the paper is an empirical measurement: three SLMs are prompted to identify seven test smells in 261 sentences from Ubuntu manual test cases, and the labels come from the externally published Aranda et al. dataset. No parameter is fitted to the data, no prediction is derived from an assumed first principle, and the benchmark annotations are not constructed from the model outputs. The self-citations to Aranda et al. and to the authors' earlier LLM work are not load-bearing circularity: the dataset is an independently falsifiable artifact of real Ubuntu test cases, and the earlier work is related work rather than the evidence for the present results. The main validity threat is that the evaluation is positive-only: every sentence is annotated with exactly one smell, the model is prompted separately for each of the seven smells, and a response that says 'yes' to the annotated smell is scored as success while 'yes' on the other six prompts is never counted as an error. Under that protocol a degenerate model answering 'yes' to every prompt would score 100% pass@2, so the reported 91-97% figures are better interpreted as recall-like agreement than as discriminative detection accuracy. That is a serious evaluation-design concern, but it is not circularity in the sense of a result being equivalent to its inputs by construction; the model outputs are still empirical and the ground truth is external. The one genuinely self-referential step is the Meta Prompting analysis in Section 4.4, where Phi-4 generates the prompt and is then evaluated on its own false negatives, but this is exploratory and does not drive the headline numbers. Overall, the central derivation chain is self-contained and the score is low.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

No new physical or mathematical entities are introduced. The evaluation choices above are the main load-bearing assumptions; the absence of a baseline and of false-positive counts means the central numbers are not yet calibrated.

free parameters (2)
  • pass@k window (k=2) = k=2
    The headline result uses two generated attempts per sentence. For a binary detection question with random guessing, pass@2 has a 75% success floor, so the reported 91-97% is closer to chance than the raw numbers suggest.
  • temperature sweep and best temperature = 0.2 for Phi-4 best temperature
    The paper evaluates 11 temperatures and reports the best temperature result; pass@2 may aggregate across attempts, and selecting the best-performing temperature after seeing outputs inflates expected performance without correcting for multiple comparisons.
assumptions (4)
  • domain assumption The Aranda et al. [3] annotations are a correct oracle for the presence of the seven test smells.
    Section 2 Dataset selection states each sentence has been manually curated by Aranda et al.; the paper scores model responses against this annotation as ground truth.
  • domain assumption Every sentence in the dataset is affected by test smells and by exactly one of the seven types.
    Section 4.2 states 'All sentences in our dataset are affected by test smells'; Table 1 partitions the 261 sentences by one smell type, and the evaluation treats detection of that annotation as correctness.
  • domain assumption Semantic agreement between a model's justification and the annotated smell is a valid correctness judgment.
    Section 2 states 'Responses were considered correct when the model's justification matched the test smell annotation in the manual baseline.'
  • domain assumption The 261 Ubuntu sentences are representative of manual test cases in general.
    The study generalizes from Ubuntu manual tests; Section 4.5 acknowledges this limitation and that a larger dataset is needed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Investigating the Performance of Small Language Models in Detecting Test Smells in Manual Test Cases." pith.science (2026). https://pith.science/paper/Y4WKDOIS

@misc{pith2026250713035,
  author       = {Pith},
  title        = {Pith review of: Investigating the Performance of Small Language Models in Detecting Test Smells in Manual Test Cases},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Y4WKDOIS}},
  note         = {Machine review of arXiv:2507.13035}
}
read the original abstract

Manual testing, in which testers follow natural language instructions to validate system behavior, remains crucial for uncovering issues not easily captured by automation. However, these test cases often suffer from test smells, quality issues such as ambiguity, redundancy, or missing checks that reduce test reliability and maintainability. While detection tools exist, they typically require manual rule definition and lack scalability. This study investigates the potential of Small Language Models (SLMs) for automatically detecting test smells. We evaluate Gemma3, Llama3.2, and Phi-4 on 143 real-world Ubuntu test cases, covering seven types of test smells. Phi-4 achieved the best results, reaching a pass@2 of 97% in detecting sentences with test smells, while Gemma3 and Llama3.2 reached approximately 91%. Beyond detection, SLMs autonomously explained issues and suggested improvements, even without explicit prompt instructions. They enabled low-cost, concept-driven identification of diverse test smells without relying on extensive rule definitions or syntactic analysis. These findings highlight the potential of SLMs as efficient tools that preserve data privacy and can improve test quality in real-world scenarios.

Figures

Figures reproduced from arXiv: 2507.13035 by the authors.

Figure 1
Figure 1. Ubuntu test case 1600_Ristretto. Prompt design. The queries to the models were guided by DSP. It is a prompting technique designed to optimize the LLM’s response in generating the desired outcome. In this approach, we provide a stimulus or clue related to the specific concept of test smell that we aim to detect in the test sentences. In DSP, we structure the prompt by presenting the concept of the test smell, a guid… view at source ↗
Figure 2
Figure 2. The Directional Stimulus Prompting [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. Improving Prompt Quality for Test Smell Detection [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

32 extracted references · 24 canonical work pages

  1. [1]

    Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J Hewett, Mojan Javaheripi, Piero Kauffmann, et al. 2024. Phi-4 technical report. arXiv preprint arXiv:2412.08905(2024)

    arXiv 2024

  2. [2]

    David H Ackley, Geoffrey E Hinton, and Terrence J Sejnowski

    work page

  3. [3]

    Manoel Aranda, Naelson Oliveira, Elvys Soares, Márcio Ribeiro, Davi Romão, Ullyanne Patriota, Rohit Gheyi, Emerson Souza, and Ivan Machado. 2024. A Catalog of Transformations to Remove Smells From Natural Language Tests. InInternational Conference on Evaluation and Assessment in Software Engineering. ACM, 7–16

    work page 2024

  4. [4]

    Manoel Aranda, Naelson Oliveira, Elvys Soares, Márcio Ribeiro, Davi Romão, Ullyanne Patriota, Rohit Gheyi, Emer- son Souza, and Ivan Machado. 2024. Manual Test Al- chemist. https://github.com/easy-software-ufal/manual-test- alchemist/tree/main/testcases Retrieved April 19, 2024

    work page 2024

  5. [5]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evalu- ating large language models trained on code. arXiv preprint arXiv:2107.03374 (2021)

    arXiv 2021

  6. [6]

    2016.Reducing system testing effort by focusing on commonalities in test procedures

    Benedikt Hauptmann. 2016.Reducing system testing effort by focusing on commonalities in test procedures. Ph.D. Dissertation. Technische Universität München

    work page 2016

  7. [7]

    Benedikt Hauptmann, Maximilian Junker, Sebastian Eder, Lars Heinemann, Rudolf Vaas, and Peter Braun. 2013. Hunting for smells in natural language tests. InInternational Conference on Software Engineering. IEEE Computer Society, 1217–1220

    work page 2013

  8. [8]

    Yutai Hou, Hongyuan Dong, Xinghao Wang, Bohan Li, and Wanx- iang Che. 2022. MetaPrompting: Learning to learn better prompts. arXiv preprint arXiv:2209.11486(2022)

    arXiv 2022

Show all 32 references
  1. [9]

    2025.Ollama: Run large language models locally

    Ollama Inc. 2025.Ollama: Run large language models locally. https://ollama.com

    2025

  2. [10]

    Katharina Juhnke, Alexander Nikic, and Matthias Tichy. 2021. Clustering Natural Language Test Case Instructions as Input for Deriving Automotive Testing DSLs. J. Object Technol.20, 3 (2021), 5–1

    2021

  3. [11]

    Nildo Silva Junior, Luana Martins, Larissa Rocha, Heitor Costa, and Ivan Machado. 2021. How are test smells treated in the wild? A tale of two empirical studies.Journal of Software Engineering Research and Development9 (2021), 9–1

    2021

  4. [12]

    Nildo Silva Junior, Larissa Rocha, Luana Almeida Martins, and Ivan Machado. 2020. A survey on test practitioners’ awareness of test smells. InIberoamerican Conference on Software Engineer- ing. Curran Associates, 462–475

    2020

  5. [13]

    Zekun Li, Baolin Peng, Pengcheng He, Michel Galley, Jianfeng Gao, and Xifeng Yan. 2023. Guiding Large Language Models via Directional Stimulus Prompting. arXiv:2302.11520 [cs.CL] https://arxiv.org/abs/2302.11520

    2023 arXiv

  6. [14]

    Zhenyan Lu, Xiang Li, Dongqi Cai, Rongjie Yi, Fangming Liu, Xiwen Zhang, Nicholas D Lane, and Mengwei Xu. 2024. Small language models: Survey, measurements, and insights. arXiv preprint arXiv:2409.15790 (2024)

    2024 arXiv

  7. [15]

    Keila Lucas, Rohit Gheyi, Márcio Ribeiro, Fabio Palomba, Luana Martins, and Elvys Soares. 2025. Investigating the Performance of Small Language Models in Detecting Test Smells in Manual Test Cases (artifacts). https://doi.org/10.5281/zenodo.15484997

    2025 doi

  8. [16]

    Keila Lucas, Rohit Gheyi, Elvys Soares, Márcio Ribeiro, and Ivan Machado. 2024. Evaluating Large Language Models in Detecting Test Smells. InBrazilian Symposium on Software Engineering. 672–678

    2024

  9. [17]

    Rian Melo, Pedro Simões, Rohit Gheyi, Marcelo d’Amorim, Márcio Ribeiro, Gustavo Soares, Eduardo Almeida, and Elvys Soares. 2025. Agentic SLMs: Hunting Down Test Smells. arXiv:2504.07277 [cs.SE] https://arxiv.org/abs/2504.07277

    2025

  10. [18]

    Meta AI. 2024. Llama 3.2: Revolutionizing edge AI and vision with open, customizable models. https://ai.meta.com/blog/llama-3-2- connect-2024-vision-edge-mobile-devices/ Accessed: 2025-05-20

    2024

  11. [19]

    Shuyin Ouyang, Jie M Zhang, Mark Harman, and Meng Wang

  12. [20]

    Max Peeperkorn, Tom Kouwenhoven, Dan Brown, and Anna Jordanous. 2024. Is temperature the creativity parameter of large language models? arXiv preprint arXiv:2405.00492(2024)

    2024 arXiv

  13. [21]

    Myron David Lucena Campos Peixoto, Davy de Medeiros Baia, Nathalia Nascimento, Paulo Alencar, Baldoino Fonseca, and Már- cio Ribeiro. 2024. On the Effectiveness of LLMs for Manual Test Verifications. arXiv preprint arXiv:2409.12405(2024)

    2024 arXiv

  14. [22]

    Kostadin Rajkovic and Eduard Enoiu. 2022. Nalabs: Detecting bad smells in natural language requirements and test specifications. arXiv preprint arXiv:2202.05641(2022)

    2022 arXiv

  15. [23]

    June Sallou, Thomas Durieux, and Annibale Panichella. 2024. Breaking the Silence: the Threats of Using LLMs in Software En- gineering. InInternational Conference on Software Engineering - New Ideas and Emerging Results. ACM/IEEE

    2024

  16. [24]

    Elvys Soares, Manoel Aranda, Naelson Oliveira, Márcio Ribeiro, Rohit Gheyi, Emerson Souza, Ivan Machado, André L. M. Santos, Baldoino Fonseca, and Rodrigo Bonifácio. 2023. Manual Tests Do Smell! Cataloging and Identifying Natural Language Test Smells. In International Symposiu...

    2023

  17. [25]

    Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovi- cova, Alexandre Ramé, Morgane Rivière, et al. 2025. Gemma 3 Technical Report.arXiv preprint arXiv:2503.19786(2025)

    2025 arXiv

  18. [26]

    Ubuntu. 2024. Ubuntu Manual Tests in Launchpad. https: //launchpad.net/ubuntumanual-tests

    2024

  19. [27]

    Chien van Nguyen, Xuan Shen, Ryan Aponte, Yu Xia, Samyadeep Basu, Zhengmian Hu, Jian Chen, Mihir Parmar, Sasidhar Kuna- puli, Joe Barrow, et al. 2024. A survey of small language models. arXiv preprint arXiv:2410.20011(2024)

    2024 arXiv

  20. [28]

    Alvaro Veizaga, Seung Yeob Shin, and Lionel C Briand. 2024. Au- tomated smell detection and recommendation in natural language requirements. IEEE Transactions on Software Engineering50, 4 (2024), 695–720

    2024

  21. [29]

    Di Wu, Fangwen Mu, Lin Shi, Zhaoqiang Guo, Kui Liu, Weiguang Zhuang, Yuqi Zhong, and Li Zhang. 2024. iSMELL: Assembling LLMs with Expert Toolsets for Code Smell Detection and Refac- toring. In Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engi...

    2024

  22. [30]

    Yanming Yang, Xing Hu, Xin Xia, and Xiaohu Yang. 2024. The lost world: Characterizing and detecting undiscovered test smells. ACM Transactions on Software Engineering and Methodology 33, 3 (2024), 1–32

    2024

  23. [1985]

    A learning algorithm for Boltzmann machines.Cognitive science 9, 1 (1985), 147–169

    1985

  24. [2025]

    ACM Transactions on Software Engineering and Methodology34, 2 (2025), 1–28

    An empirical study of the non-determinism of chatgpt in code generation. ACM Transactions on Software Engineering and Methodology34, 2 (2025), 1–28

    2025

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.