pith. sign in

arxiv: 2606.26604 · v1 · pith:SANJRAUZnew · submitted 2026-06-25 · 💻 cs.SE

Quantum Mutant Equivalence via Transpilation

Jos\'e Campos , Andriy Miranskyy This is my paper

Pith reviewed 2026-06-26 04:37 UTC · model grok-4.3

classification 💻 cs.SE
keywords quantum mutation testingequivalent mutantstranspilationOpenQASMquantum circuitsmutant detectionsoftware testing
0
0 comments X

The pith

Transpiling quantum circuits under identical settings and comparing OpenQASM identifies equivalent mutants at 100% precision.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Mutation testing for quantum programs is hindered by equivalent mutants that differ in syntax but match in behavior, so no test can kill them. The paper presents Transpiler-Based Equivalence as a method that runs the original circuit and each mutant through the same transpiler configuration then checks whether the emitted OpenQASM files are identical. On 348299 surviving mutants that include 92011 known equivalents, the method correctly flags 29536 of the equivalents. It records 100% precision and 82% accuracy while leaving the remaining equivalents undetected. Removing these flagged mutants from the pool improves the reliability of mutation scores that quantum testing researchers compute.

Core claim

Transpiler-Based Equivalence identifies equivalent quantum mutants by transpiling the original and mutated circuits under the same configuration and comparing the resulting OpenQASM code. Evaluated on 348299 surviving mutants including 92011 equivalents, the method detects 29536 equivalents at 100% precision and 82% accuracy.

What carries the argument

Transpiler-Based Equivalence (TBE), which detects semantic equivalence by identical-configuration transpilation followed by direct OpenQASM comparison.

If this is right

  • 32.1% of the 92011 known equivalent mutants can be removed from consideration before test execution.
  • Mutation scores become more accurate once these unkillable mutants are excluded.
  • The approach processes hundreds of thousands of mutants without introducing false positives.
  • Surviving mutants that pass TBE can be prioritized for further manual or automated analysis.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The method could be combined with other equivalence detectors to raise recall beyond the reported 32%.
  • The same transpilation-comparison idea might apply directly to classical mutation testing in languages that have stable compilers.
  • Accuracy figures rest on the independence of the 92011 ground-truth labels from the TBE procedure itself.

Load-bearing premise

Transpiling the original and mutant circuits under the exact same configuration and comparing the resulting OpenQASM code is sufficient to detect semantic equivalence.

What would settle it

A pair of circuits known to be semantically distinct that nevertheless produce identical OpenQASM output after transpilation under the configuration used in the evaluation.

Figures

Figures reproduced from arXiv: 2606.26604 by Andriy Miranskyy, Jos\'e Campos.

Figure 1
Figure 1. Figure 1: reports the results of the state-vector analysis. Overall, 92,011 (26.4%) of the 348,299 surviving mutants previously generated by Mendiluze Usandizaga et al. [6] are equivalent. Most mutants classified as equivalent (90,166) are produced by mutation operators that add new gates to the original circuit. Additionally, the state-vector results suggest that mutating a circuit by removing or replacing a gate i… view at source ↗
Figure 2
Figure 2. Figure 2: True positives overlap among all basis gates sets. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
read the original abstract

Mutation testing evaluates test suite quality by introducing artificial faults (mutants) and checking whether tests detect (kill) them. A central challenge is the equivalent mutant problem: some mutants are syntactically different from the original program but semantically identical to it and therefore cannot be killed by any test. If left unidentified, such mutants waste testing effort and distort mutation scores. In quantum software, mutation testing is increasingly used, but the equivalent mutant problem remains unsolved. A recent study generated more than 700,000 quantum circuit mutants and found that roughly half survived the available tests, making it unclear whether these survivors reflect weak tests or semantic equivalence. We propose Transpiler-Based Equivalence (TBE), a lightweight approach that identifies equivalent quantum mutants by transpiling original and mutated circuits under the same configuration and comparing their resulting OpenQASM code. We evaluate TBE on 348,299 surviving mutants, 92,011 of which are equivalent; TBE identifies 29,536 of them (32.1%) as equivalent while achieving 100% precision and 82% accuracy.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper proposes Transpiler-Based Equivalence (TBE) to identify equivalent mutants in quantum circuits: original and mutated circuits are transpiled under identical configurations and their OpenQASM outputs compared for syntactic identity. On a set of 348,299 surviving mutants (92,011 labeled equivalent), TBE flags 29,536 as equivalent, reporting 100% precision and 82% accuracy.

Significance. If the ground-truth labels prove independent, TBE supplies a lightweight, scalable filter that could reduce wasted effort on equivalent mutants in quantum mutation testing. The evaluation scale (hundreds of thousands of mutants) is a concrete strength.

major comments (2)
  1. [Abstract / §4] Abstract and §4 (Evaluation): the headline metrics (100% precision, 82% accuracy on 29,536/92,011 equivalents) rest on the assumption that the 92,011 ground-truth labels were produced by a procedure independent of TBE, of the same transpilation configuration, and of the 'surviving tests' criterion. No description of the prior study's labeling method is supplied, so the reported precision cannot yet be treated as an independent validation.
  2. [§4] §4: the accuracy figure of 82% is reported without error bars, without breakdown by circuit family or depth, and without analysis of false-negative cases (equivalent mutants missed by TBE). These omissions make it impossible to assess whether the 32.1% detection rate is robust or sensitive to the chosen transpiler settings.
minor comments (2)
  1. [Abstract] Abstract: state the exact transpiler (Qiskit version, optimization level, basis gates) used for the comparison; this detail is required for reproducibility.
  2. [§3] §3: clarify whether TBE treats two circuits as equivalent only when their OpenQASM strings match exactly, or whether it normalizes for trivial differences (e.g., qubit ordering, gate decomposition).

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback and for highlighting the evaluation scale as a strength. We address each major comment below with proposed revisions to enhance clarity.

read point-by-point responses

  1. Referee: [Abstract / §4] Abstract and §4 (Evaluation): the headline metrics (100% precision, 82% accuracy on 29,536/92,011 equivalents) rest on the assumption that the 92,011 ground-truth labels were produced by a procedure independent of TBE, of the same transpilation configuration, and of the 'surviving tests' criterion. No description of the prior study's labeling method is supplied, so the reported precision cannot yet be treated as an independent validation.

    Authors: We agree that a description of the prior study's labeling procedure is required to substantiate independence. The 92,011 equivalent labels derive from the referenced prior work, which determined equivalence via exhaustive simulation-based semantic checking independent of transpilation. In the revised manuscript we will add a concise summary of this method to §4, allowing the precision claim to be evaluated as an independent validation. revision: yes

  2. Referee: [§4] §4: the accuracy figure of 82% is reported without error bars, without breakdown by circuit family or depth, and without analysis of false-negative cases (equivalent mutants missed by TBE). These omissions make it impossible to assess whether the 32.1% detection rate is robust or sensitive to the chosen transpiler settings.

    Authors: We acknowledge the benefit of additional robustness analysis. Error bars are not applicable, as TBE performs deterministic syntactic comparison with no stochastic component. We will incorporate a breakdown by circuit family and depth into the revised §4, together with an explicit discussion of the false-negative cases (the 62,475 equivalent mutants not flagged by TBE) and their relation to the fixed transpiler configuration chosen to match the prior study. This will allow readers to assess sensitivity of the 32.1% detection rate. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical method evaluated against externally labeled data

full rationale

The paper introduces TBE as a practical heuristic (transpile both circuits identically then compare OpenQASM text) and reports its precision/accuracy on a set of 92,011 mutants already labeled equivalent by a prior study. No equations, no fitted parameters, no derivation chain, and no claim that the labels were produced by TBE itself. The evaluation is therefore a standard external-benchmark comparison; the 100 % precision figure is not forced by construction. Minor self-citation of the prior mutant-generation work, if present, is not load-bearing for the reported metrics.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The method rests on a domain assumption about transpilation revealing equivalence. No free parameters or invented entities are introduced.

axioms (1)
  • domain assumption Two quantum circuits that produce identical OpenQASM output after transpilation under the same configuration are semantically equivalent.
    This is the core mechanism of TBE; if false, the identification of equivalents would not hold.

pith-pipeline@v0.9.1-grok · 5711 in / 1276 out tokens · 41673 ms · 2026-06-26T04:37:34.038065+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

16 extracted references · 12 canonical work pages · 1 internal anchor

  1. [1]

    Mutation Testing in Practice: Insights From Open-Source Software Developers

    A. B. S ´anchez, J. A. Parejo, S. Segura, A. Dur ´an, and M. Papadakis. “Mutation Testing in Practice: Insights From Open-Source Software Developers”. In:IEEE Transactions on Software Engineering50.5 (2024), pp. 1130–1143.DOI: 10.1109/TSE.2024.3377378

    work page doi:10.1109/tse.2024.3377378 2024

  2. [2]

    Mutation Testing of Quantum Programs: A Case Study With Qiskit

    D. Fortunato, J. Campos, and R. Abreu. “Mutation Testing of Quantum Programs: A Case Study With Qiskit”. In:IEEE Transactions on Quantum Engineering3 (2022), pp. 1–17.DOI: 10.1109/TQE.2022. 3195061

    work page doi:10.1109/tqe.2022 2022

  3. [3]

    Thinking like a developer? comparing the attention of humans with neural models of code,

    E. Mendiluze, S. Ali, P. Arcaini, and T. Yue. “Muskit: A Mutation Analysis Tool for Quantum Software Testing”. In:IEEE/ACM Inter- national Conference on Automated Software Engineering (ASE). 2021, pp. 1266–1270.DOI: 10.1109/ASE51524.2021.9678563

    work page doi:10.1109/ase51524.2021.9678563 2021

  4. [4]

    Two notions of correctness and their relation to testing

    T. A. Budd and D. Angluin. “Two notions of correctness and their relation to testing”. In:Acta Inf.18.1 (Mar. 1982), 31–45.ISSN: 0001- 5903.DOI: 10 . 1007 / BF00625279.URL: https : / / doi . org / 10 . 1007 / BF00625279

    1982

  5. [5]

    Trivial Compiler Equivalence: A Large Scale Empirical Study of a Simple, Fast and Ef- fective Equivalent Mutant Detection Technique

    M. Papadakis, Y . Jia, M. Harman, and Y . Le Traon. “Trivial Compiler Equivalence: A Large Scale Empirical Study of a Simple, Fast and Ef- fective Equivalent Mutant Detection Technique”. In:2015 IEEE/ACM 37th IEEE International Conference on Software Engineering. V ol. 1. 2015, pp. 936–946.DOI: 10.1109/ICSE.2015.103

    work page doi:10.1109/icse.2015.103 2015

  6. [7]

    MQT Bench: Bench- marking Software and Design Automation Tools for Quantum Com- puting

    N. Quetschlich, L. Burgholzer, and R. Wille. “MQT Bench: Bench- marking Software and Design Automation Tools for Quantum Com- puting”. In:Quantum7 (2023), p. 1062.DOI: 10.22331/q-2023-07- 20-1062

    work page doi:10.22331/q-2023-07- 2023

  7. [8]

    Quantum computing with Qiskit

    A. Javadi-Abhari et al.Quantum computing with Qiskit. 2024.DOI: 10.48550/arXiv.2405.08810. arXiv: 2405.08810

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2405.08810 2024

  8. [9]

    Miranskyy, J

    A. Miranskyy, J. Campos, A. Mjeda, L. Zhang, and I. G. R. de Guzm ´an. On the Feasibility of Quantum Unit Testing. 2025. arXiv: 2507.17235 [cs.SE].URL: https://arxiv.org/abs/2507.17235

    arXiv 2025

  9. [10]

    Guidelines for conducting and reporting case study research in software engineering.Empirical Software Engineering, 14(2):131–164, 2009

    M. D. Stefano, D. D. Nucci, F. Palomba, and A. D. Lucia. “An empirical study into the effects of transpilation on quantum circuit smells”. In:Empirical Softw. Engg.29.3 (May 2024).ISSN: 1382- 3256.DOI: 10.1007/s10664- 024- 10461- 9.URL: https://doi.org/10. 1007/s10664-024-10461-9

    work page doi:10.1007/s10664- 2024

  10. [11]

    Does data sampling improve deep learning-based vulnerability detection? yeas! and nays!

    Q. Chen, R. C ˆamara, J. Campos, A. Souto, and I. Ahmed. “The Smelly Eight: An Empirical Study on the Prevalence of Code Smells in Quantum Computing”. In:2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE). 2023, pp. 358–370.DOI: 10.1109/ICSE48619.2023.00041

    work page doi:10.1109/icse48619.2023.00041 2023

  11. [12]

    Still just personal assistants? - A multiple case study of generative AI adoption in software organizations

    J. Yao and M. Shepperd. “The impact of using biased performance metrics on software defect prediction research”. In:Information and Software Technology139 (2021), p. 106664.DOI: 10.1016/j.infsof. 2021.106664

    work page doi:10.1016/j.infsof 2021

  12. [13]

    Mutation-based test generation for quantum programs with multi-objective search

    X. Wang, T. Yu, P. Arcaini, T. Yue, and S. Ali. “Mutation-based test generation for quantum programs with multi-objective search”. In: Proceedings of the Genetic and Evolutionary Computation Conference. GECCO ’22. 2022, 1345–1353.DOI: 10.1145/3512290.3528869

    work page doi:10.1145/3512290.3528869 2022

  13. [14]

    Development of a Tool for Finding Equivalent Mutants in Quantum Program: A Perspective to Measure the Quality of Quantum Software

    A. Kumar, P. Kwatra, and S. Garhwal. “Development of a Tool for Finding Equivalent Mutants in Quantum Program: A Perspective to Measure the Quality of Quantum Software”. In: (2023).DOI: 10.21203/ rs.3.rs-2250025/v2.URL: https://doi.org/10.21203/rs.3.rs-2250025/v2

    work page doi:10.21203/rs.3.rs-2250025/v2 2023

  14. [15]

    QGMR: A New Quantum Mutation Testing Operator

    S. Shah, S. Godboley, and P. R. Krishna. “QGMR: A New Quantum Mutation Testing Operator”. In:2026 IEEE International Conference on Software Analysis, Evolution and Reengineering - Companion (SANER-C). 2026, pp. 373–376.DOI: 10.1109/SANER-C67878.2026. 00056

    work page doi:10.1109/saner-c67878.2026 2026

  15. [16]

    Andrews and P

    E. Andrews and P. Mishra.Efficient Mutation Testing of Quantum Machine Learning Models. 2026. arXiv: 2605.00107[quant-ph]. URL: https://arxiv.org/abs/2605.00107

    Pith/arXiv arXiv 2026

  16. [17]

    Robust Mutation Analysis of Quantum Programs Under Noise

    S. Fortz, E. M. Usandizaga, S. Ali, P. Arcaini, and M. R. Mousavi. “Robust Mutation Analysis of Quantum Programs Under Noise”. In: (2026). arXiv: 2605.13279[cs.SE].URL: https://arxiv.org/abs/2605. 13279

    Pith/arXiv arXiv 2026