Pith. sign in

REVIEW 4 major objections 3 minor 21 references

HyClone: Bridging LLM Understanding and Dynamic Execution for Semantic Code Clone Detection

T0 review · 4 major / 3 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims a two-stage LLM-screening plus execution-validation pipeline outperforms direct LLM classification for semantic code clone detection.

desk verdict The abstract and full text are different papers; the HyClone claims are nowhere in the submitted manuscript, so the central result is unverifiable and the submission is not reviewable in this state. read the letter →

arxiv 2508.01357 v1 pith:PZBUYKP3 submitted 2025-08-02 cs.SE

Yunhao Liang , Ruixuan Ying , Takuya Taniguchi , Guwen Lyu , Zhe Cui This is my paper
classification cs.SE
keywords semanticcodeclonedetectionType4cloneslargelanguagemodelsexecution-basedvalidationtestinputgenerationfunctionalequivalencePythonprogramstwo-stagepipeline
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper's abstract argues that semantic code clones — fragments with identical behavior but different syntax, known as Type 4 clones — are detected more reliably by a two-stage process than by asking a large language model to judge pairs directly. In the first stage, the LLM screens out pairs that are obviously not clones. In the second, remaining pairs are tested for functional equivalence by running both programs on LLM-generated inputs and comparing the executions. The claim is that this hybrid pipeline improves precision, recall, and F1 over direct LLM-based detection. Note that the full text supplied for this record is a different manuscript, so the experimental part of this claim is not inspectable here.

What carries the argument

The central mechanism is the two-stage pipeline: an LLM-based semantic screening stage that filters out obvious non-clones, followed by an execution-based validation stage that uses LLM-generated test inputs to perform cross-execution validation. Cross-execution validation means running both code fragments on the same generated inputs and comparing their observable behavior, so that functional equivalence is decided by execution rather than by the LLM's syntactic impression.

What would settle it

Compare the two-stage pipeline with direct LLM classification on a ground-truth set of Python function pairs that includes near-miss non-clones sharing syntax but differing in one untested branch; if the pipeline labels those pairs as clones because the generated inputs never hit the differing branch, the central claim collapses. The supplied text contains no such evaluation.

Watch

Extended reading notes

Core claim

The discovery the authors are trying to establish is that semantic clone detection should not rely on the LLM's verdict alone. Instead, they propose a screening-plus-execution design: an LLM first discards clearly non-clone pairs based on semantic analysis, and then every surviving pair is cross-executed on LLM-generated test inputs to decide whether the two programs are functionally equivalent. The stated result is that this two-stage division of labor achieves significantly better precision, recall, and F1 than directly applying an LLM to classify pairs as clones or non-clones.

Load-bearing premise

The load-bearing premise is that the test inputs generated by the LLM reach the code branches where semantically different programs actually diverge; if the tests miss those branches, the execution stage will wrongly confirm clone pairs.

Editorial extensions

If this is right

  • If the framework works as stated, semantic clone detectors can be built without hand-written test oracles: the LLM supplies the inputs and the execution comparison supplies the oracle.
  • The execution stage should suppress false positives caused by LLMs being misled by surface syntax, since functionally equivalent pairs must agree on actual behavior.
  • The screening stage should reduce execution cost, because only pairs that survive semantic filtering pay the price of running both programs.
  • The paper's stated future directions — cross-language clone detection and scaling to large applications — are natural next steps if the current result holds.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An unstated extension is to measure how well the LLM-generated test inputs cover the behavioral space of each pair; if coverage is low, adding targeted or adversarial test generation would likely improve the validation stage.
  • The screening-then-executing pattern is transferable to other software engineering tasks where a cheap semantic prefilter can be followed by an expensive behavioral check, such as duplicate bug report detection or program repair validation.
  • The full-text section of this record is a different manuscript on gamma/hadron separation for Cherenkov astronomy, so the abstract's experimental claims are not backed by the supplied text; the claimed improvement over direct LLM detection should be read as unverified.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. The submission presents an abstract claiming a two-stage framework ('HyClone') that combines LLM-based screening with execution-based validation for semantic code clone detection, and claims that experiments show significant improvements in precision, recall, and F1-score over direct LLM-based detection. However, the full text supplied is a different manuscript: 'Testing Gamma/Hadron Separation for Ultra-High-Energy Cherenkov Astronomy', a Cherenkov telescope gamma/hadron separation study using CORSIKA simulations and boosted decision trees. None of the HyClone framework, its components, datasets, baselines, or experimental results appear in the submitted text.

Significance. If the claimed HyClone framework existed and performed as stated, a hybrid LLM-screening plus execution-validation pipeline for semantic clone detection could be a practically useful contribution. However, the submitted manuscript contains no implementation details, no experimental evaluation, and no artifact; the only measurable result in the full text (AUC 0.91 for gamma/hadron separation at 1 PeV) is unrelated to code clone detection. There are no machine-checked proofs, reproducible code, or falsifiable predictions for the claimed method, so the paper's central contribution is unsupported.

major comments (4)
  1. [Abstract / Full text] The central claim of the abstract—that HyClone combines LLM-based screening and execution-based validation for semantic clone detection in Python—is not present anywhere in the full text. The body describes a Cherenkov astronomy study with no mention of LLMs, code clones, or execution-based validation. This is a load-bearing mismatch: the paper's central contribution cannot be evaluated from the submitted artifact.
  2. [Abstract, 'Our experimental evaluation'] The abstract asserts 'significant improvements in precision, recall, and F1-score compared to direct LLM-based detection,' but the manuscript contains no such evaluation. There are no reported precision, recall, or F1 values, no dataset description, no baseline description, and no comparison to direct LLM detection. The only quantitative performance result in the full text is the gamma/hadron separation AUC of 0.91 in the Results section, which is irrelevant to the claimed framework.
  3. [Abstract, second stage] The claimed execution-based validation assumes that LLM-generated test inputs adequately cover the behavior of the code pairs, so that divergent branches of non-clones are exposed. No coverage analysis, test-generation strategy, or justification for this assumption is supplied anywhere in the manuscript. Additionally, because the same LLM is used both to screen pairs and to generate test inputs, there is a potential bias that is not discussed.
  4. [Full text] No implementation, dataset, or artifact is provided for HyClone. The full text's acknowledgments, references, and author list correspond to the astronomy paper, not to a software engineering clone-detection study. Consequently, the claimed results are not reproducible from the submitted material.
minor comments (3)
  1. [Title and metadata] The manuscript title, author list, abstract, and body content are mutually inconsistent: the title and body describe gamma/hadron separation for the Dark100 array, while the abstract describes a code clone detection framework. This appears to be a submission or file-composition error.
  2. [Full text structure] The full text has unnumbered sections, making precise reference to specific claims difficult; I have cited by section title and by nearby content.
  3. [References] The reference list contains only astronomy and cosmic-ray simulation citations, with no references on code clone detection, LLM-based code analysis, or software engineering; there is no scholarly grounding for the abstract's claimed framework.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the HyClone claims are unsupported by the supplied full text, but absence of support is not circularity.

full rationale

The claimed derivation chain for HyClone—LLM-based screening followed by LLM-generated test inputs and cross-execution validation—is not present in the supplied full text. The body is an unrelated Cherenkov astronomy paper reporting CORSIKA simulations and BDT-based gamma/hadron separation (Sections 1–4). Consequently, there are no equations in which a predicted quantity is defined in terms of the target, no fitted parameter is renamed as a prediction, and no load-bearing self-citation chain to inspect. The only conceivable circularity would be that the same LLM is used both to screen pairs and to generate test inputs, but the abstract does not define the final clone verdict as the screening output; it describes functional equivalence assessed through separate cross-execution, which is an external behavioral check. A mismatch between the abstract's claimed framework and the supplied full text is a serious verifiability and integrity concern, but it is not a reduction of a derivation to its own inputs. Under the provided rubric, absence of a supporting derivation is not circularity, so the score is 0.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

The abstract introduces only a methodological framework, not new physical or conceptual entities. The two listed assumptions are the load-bearing premises on which the claimed effectiveness of the execution-based stage depends.

assumptions (2)
  • domain assumption LLM-generated test inputs provide sufficient behavioral coverage to reveal functional differences between code fragments.
    The second stage of the claimed framework relies on these test inputs for cross-execution validation; if coverage is inadequate, false positives would pass the validation.
  • domain assumption Execution-based validation is a reliable proxy for functional equivalence.
    The framework equates identical outputs on generated inputs with semantic equivalence, which may miss differences that occur outside the tested inputs.

how reviews work

0 comments
Cite this review

Pith. "Pith review of HyClone: Bridging LLM Understanding and Dynamic Execution for Semantic Code Clone Detection." pith.science (2026). https://pith.science/paper/PZBUYKP3

@misc{pith2026250801357,
  author       = {Pith},
  title        = {Pith review of: HyClone: Bridging LLM Understanding and Dynamic Execution for Semantic Code Clone Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PZBUYKP3}},
  note         = {Machine review of arXiv:2508.01357}
}
read the original abstract

Code clone detection is a critical task in software engineering, aimed at identifying duplicated or similar code fragments within or across software systems. Traditional methods often fail to capture functional equivalence, particularly for semantic clones (Type 4), where code fragments implement identical functionality despite differing syntactic structures. Recent advances in large language models (LLMs) have shown promise in understanding code semantics. However, directly applying LLMs to code clone detection yields suboptimal results due to their sensitivity to syntactic differences. To address these challenges, we propose a novel two-stage framework that combines LLM-based screening with execution-based validation for detecting semantic clones in Python programs. In the first stage, an LLM evaluates code pairs to filter out obvious non-clones based on semantic analysis. For pairs not identified as clones, the second stage employs an execution-based validation approach, utilizing LLM-generated test inputs to assess functional equivalence through cross-execution validation. Our experimental evaluation demonstrates significant improvements in precision, recall, and F1-score compared to direct LLM-based detection, highlighting the framework's effectiveness in identifying semantic clones. Future work includes exploring cross-language clone detection and optimizing the framework for large-scale applications.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

21 extracted references · 16 canonical work pages

  1. [1]

    IEEE access 7, 86121--86144 (2019)

    Ain, Q.U., Butt, W.H., Anwar, M.W., Azam, F., Maqbool, B.: A systematic review on code clone detection. IEEE access 7, 86121--86144 (2019)

    work page 2019

  2. [2]

    In: Proceedings

    Baxter, I.D., Yahin, A., Moura, L., Sant'Anna, M., Bier, L.: Clone detection using abstract syntax trees. In: Proceedings. International Conference on Software Maintenance (Cat. No. 98CB36272). pp. 368--377. IEEE (1998)

    work page 1998

  3. [3]

    arXiv preprint arXiv:2308.01191 (2023)

    Dou, S., Shan, J., Jia, H., Deng, W., Xi, Z., He, W., Wu, Y., Gui, T., Liu, Y., Huang, X.: Towards understanding the capability of large language models on code clone detection: A survey. arXiv preprint arXiv:2308.01191 (2023)

    arXiv 2023

  4. [4]

    In: 29th International Conference on Software Engineering (ICSE'07)

    Jiang, L., Misherghi, G., Su, Z., Glondu, S.: Deckard: Scalable and accurate tree-based detection of code clones. In: 29th International Conference on Software Engineering (ICSE'07). pp. 96--105. IEEE (2007)

    work page 2007

  5. [5]

    Juergens, E., Deissenboeck, F., Hummel, B., Wagner, S.: Do code clones matter? In: 2009 IEEE 31st International Conference on Software Engineering. pp. 485--495. IEEE (2009)

    work page 2009

  6. [6]

    In: Proceedings of the 32nd IEEE/ACM International Conference on Program Comprehension

    Khajezade, M., Wu, J.J., Fard, F.H., Rodr \' guez-P \'e rez, G., Shehata, M.S.: Investigating the efficacy of large language models for code clone detection. In: Proceedings of the 32nd IEEE/ACM International Conference on Program Comprehension. pp. 161--165 (2024)

    work page 2024

  7. [7]

    Koschke, R.: Survey of research on software clones (2007)

    work page 2007

  8. [8]

    In: Proceedings eighth working conference on reverse engineering

    Krinke, J.: Identifying similar code with program dependence graphs. In: Proceedings eighth working conference on reverse engineering. pp. 301--309. IEEE (2001)

    work page 2001

Show all 21 references
  1. [9]

    Concurrency and Computation: Practice and Experience 28(6), 1900--1917 (2016)

    Li, H., Kwon, H., Kwon, J., Lee, H.: Clorifi: software vulnerability discovery using code clone verification. Concurrency and Computation: Practice and Experience 28(6), 1900--1917 (2016)

    2016

  2. [10]

    In: 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE)

    Li, J., Tao, C., Jin, Z., Liu, F., Li, G.: Zc 3: Zero-shot cross-language code clone detection. In: 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE). pp. 875--887. IEEE (2023)

    2023

  3. [11]

    arXiv preprint arXiv:2404.13340 (2024)

    Li, K., Yuan, Y.: Large language models as test case generators: Performance evaluation and enhancement. arXiv preprint arXiv:2404.13340 (2024)

    2024 arXiv

  4. [12]

    In: International Conference on Intelligent Computing

    Liang, Y., Gan, C., Ying, R., Cui, Z.: Exploring behavior-driven development for code generation. In: International Conference on Intelligent Computing. pp. 41--51. Springer (2025)

    2025

  5. [13]

    In: International Conference on Intelligent Computing

    Liang, Y., Ying, R., Taniguchi, T., Gan, C., Cui, Z.: Recode: Leveraging reliable self-generated tests and fine-grained execution feedback to enhance llm-based code generation. In: International Conference on Intelligent Computing. pp. 510--521. Springer (2025)

    2025

  6. [14]

    arXiv preprint arXiv:2412.19437 (2024)

    Liu, A., Feng, B., Xue, B., Wang, B., Wu, B., Lu, C., Zhao, C., Deng, C., Zhang, C., Ruan, C., et al.: Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437 (2024)

    2024 arXiv

  7. [15]

    In: 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE)

    Liu, J., Zeng, J., Wang, X., Liang, Z.: Learning graph-based code representations for source-level functional similarity detection. In: 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE). pp. 345--357. IEEE (2023)

    2023

  8. [16]

    Science of computer programming 74(7), 470--495 (2009)

    Roy, C.K., Cordy, J.R., Koschke, R.: Comparison and evaluation of code clone detection techniques and tools: A qualitative approach. Science of computer programming 74(7), 470--495 (2009)

    2009

  9. [17]

    In: 2020 IEEE 27th International Conference on Software Analysis, Evolution and Reengineering (SANER)

    Wang, W., Li, G., Ma, B., Xia, X., Jin, Z.: Detecting code clones with graph neural network and flow-augmented abstract syntax tree. In: 2020 IEEE 27th International Conference on Software Analysis, Evolution and Reengineering (SANER). pp. 261--271. IEEE (2020)

    2020

  10. [18]

    arXiv preprint arXiv:2406.04531 (2024)

    Wang, W., Yang, C., Wang, Z., Huang, Y., Chu, Z., Song, D., Zhang, L., Chen, A.R., Ma, L.: Testeval: Benchmarking large language models for test case generation. arXiv preprint arXiv:2406.04531 (2024)

    2024 arXiv

  11. [19]

    In: Proceedings of the 31st IEEE/ACM international conference on automated software engineering

    White, M., Tufano, M., Vendome, C., Poshyvanyk, D.: Deep learning code fragments for code clone detection. In: Proceedings of the 31st IEEE/ACM international conference on automated software engineering. pp. 87--98 (2016)

    2016

  12. [20]

    arXiv preprint arXiv:2505.09388 (2025)

    Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., et al.: Qwen3 technical report. arXiv preprint arXiv:2505.09388 (2025)

    2025 arXiv

  13. [21]

    In: 2024 4th International Conference on Code Quality (ICCQ)

    Zhang, Z., Saber, T.: Assessing the code clone detection capability of large language models. In: 2024 4th International Conference on Code Quality (ICCQ). pp. 75--83. IEEE (2024)

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.