{"id":"d86d2401-60b9-47c0-9081-0ba43399ec33","arxiv_id":"2507.17235","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A large empirical study finds that the Inverse test, which reverses the expected circuit and checks for the all-zero state, detects quantum circuit mutations with fewer shots than statistical or Swap tests.","lead":"This paper compares ways to unit test quantum software, including a new 'Inverse test' that reverses the expected circuit and checks that all qubits return to zero. On 1.8 million mutated quantum circuits, the Inverse test and the classical Statevector check detected small faults with fewer measurements than statistical tests.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"RQ2's shot cap is anchored to the Inverse test's own theoretical budget, which is provably below the Swap test's required budget for every non-identical mutant, biasing the recall and shot-count comparison.","rationale":"The paper's central claim is plausible: the Inverse test's per-shot detection probability is 1-f versus (1-f)/2 for the Swap test, so the qualitative ranking (Inverse needs fewer shots) follows from the construction and is not in doubt. The load-bearing soft spot is the experimental protocol, not the noise assumption highlighted by the reader. Section IV-C1b caps every test at twice the Inverse test's theoretical shot count from Equation (8). For any non-identical pure states, this cap is strictly below the Swap test's own requirement for the same false-negative probability, so the recall comparison in Table III and the censored medians in Figure 8 are biased. The headline quantitative claim of 60-66% fewer shots could overstate the advantage, and the 'keeping false negatives low' clause is unfair as designed. A re-run with per-test budgets would settle whether the magnitude is real. This does not invalidate the paper's main direction, so the reader's CONDITIONAL verdict remains appropriate; the concern adds a new condition to the RQ2 interpretation rather than changing the overall verdict.","tokens_in":28673,"tokens_out":22801,"duration_ms":255336,"concrete_test":"Re-run RQ2 on the same mutant corpus giving each test its own theoretically sufficient budget: for the Swap test use S_max = ceil(ln(0.05)/ln((1+f)/2)) instead of the Inverse-anchored cap, and for statistical tests extend the two-pass search to a common fixed budget (e.g., 10^5 shots) without the Equation (8) cap. Recompute Table III recall and Figure 8 medians. If the Swap test's recall rises to approximately 0.95 and the median shot ratio falls toward 2, the reported recall advantage and the '60-66% fewer shots' magnitude are artifacts of the asymmetric cap; if the gap persists, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Section IV-C1b, the shot budget is capped at twice the theoretical limit given by Equation (8) with Pe = 0.05, and Equation (8) is the Inverse test's shot-count formula from Section III-B4. For a mutant with fidelity f between the actual and expected pure states, the Inverse test's false-negative probability in S shots is f^S, so its required shots are N_inv = ln(Pe)/ln(f). The Swap test's ancilla remains 0 with probability (1+f)/2 per shot, so its required shots are N_swap = ln(Pe)/ln((1+f)/2). Because ((1+f)/2)^2 > f for every f < 1, N_swap > 2*N_inv. Hence the cap of 2*N_inv is strictly smaller than the number of shots the Swap test needs to reach the same Pe. The same cap is also applied to statistical tests, whose shot requirements are unrelated to Equation (8). Consequently, the Table III recall gap (Inverse 0.904 vs Swap 0.824 on Random Circuits; 0.975 vs 0.958 on MQT Bench) and the censored shot-count medians in Figure 8 partly reflect an asymmetric, Inverse-anchored budget rather than intrinsic test performance. This directly weakens the RQ2 claim that the Inverse test 'keeps false negatives low' relative to the Swap test, because the Swap test is not allowed its own sufficient budget.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper formalizes quantum unit tests for four families—statistical tests, the Swap test, the Statevector test, and a new Inverse test—and compares them both theoretically and empirically. The theoretical part derives a shot-count estimate for the Inverse test from the Quantum Chernoff Bound, and the empirical part applies mutation operators to 10,000 random circuits and 85 MQT Bench circuits, yielding 1,796,880 mutated circuits that are tested with nine methods under a shared shot cap. The paper reports that the Inverse test performs best at distinguishing different quantum states (RQ1) and requires 60--66% fewer shots than the Swap test (RQ2), with the Statevector test being exact but not scalable.","tokens_in":28988,"tokens_out":9137,"duration_ms":92192,"significance":"If the central claims hold, the paper is a valuable contribution to quantum software testing: it gives a clean formal definition of a quantum unit test, introduces a novel reversible-circuit-based Inverse test, derives a parameter-free shot-count prediction from the Quantum Chernoff Bound, and evaluates on an unusually large corpus (1.8M mutated circuits, 100 repetitions per experiment). The paper is also careful to state its fault-tolerant/noiseless execution assumption. However, the headline empirical claims are currently weakened by two technical problems: an incorrect derivation in the Case 1 analysis of Section III-B4, and an asymmetric shot budget in Section IV-C1b that favors the Inverse test over the Swap test. Because RQ1 and RQ2 depend on exactly those comparisons, the conclusions need to be revisited with a fair budgeting scheme.","major_comments":[{"comment":"The analysis of Case 1 states that for a diagonal sigma with sigma_11 = 0, Tr(rho sigma^{1-s}) = 1. This is false for s < 1: the (1,1) entry of sigma^{1-s} is 0^{1-s} = 0, so the trace is 0, with the usual convention 0^0 = 1 applying only at s = 1. The correct minimum is 0, which gives xi_QCB = +infinity and an error probability of 0, not Pe ~ exp(-N * 0) = 1. The displayed equation is also internally inconsistent with the following sentence claiming that a single shot suffices: if the error probability were 1, no number of shots would detect the difference. The qualitative conclusion (one shot detects orthogonal states) is correct, but the derivation and the displayed formula need to be corrected.","section":"Section III-B4, Case 1 (Eq. (5))"},{"comment":"The shot budget is capped at twice the theoretical limit given by Equation (8), which is the Inverse test's own shot-count formula, and this same cap is applied to all tests. For a mutant with fidelity f between the actual and expected pure states, the Inverse test's per-shot false-negative probability is f, while the Swap test's is (1+f)/2. Since ((1+f)/2)^2 > f for every f < 1, the number of shots the Swap test needs to reach the same Pe = 0.05 is strictly greater than twice the Inverse test's requirement. The cap of 2N_inv therefore denies the Swap test a budget sufficient for its own target error rate, while granting the Inverse test its target budget. Consequently, the recall differences in Table III (Inverse 0.904 vs Swap 0.824 on Random Circuits; 0.975 vs 0.958 on MQT Bench) and the median shot counts in Figure 8 partly reflect this asymmetric, Inverse-anchored budget rather than intrinsic test performance. This directly weakens the RQ2 claim that the Inverse test 'keeps false negatives low' relative to the Swap test. The comparison should give each test its own theoretical budget for a common error probability, or explicitly analyze the censored results.","section":"Section IV-C1b and Table III / RQ2"}],"minor_comments":[{"comment":"The stated total number of experiments, 1,585,665,000 = (1,751,850 + 10,000) pairs x 9 tests x 100 repetitions, does not match the reported total of 1,796,880 mutated circuits (1,751,850 random plus 45,030 MQT Bench). Please correct the count and clarify whether the deterministic Statevector test is included in the 100 repetitions.","section":"Section IV-C1c"},{"comment":"The displayed derivative of f(sigma_11, s) = ln(sigma_11^{1-s}) is garbled; the correct derivative with respect to s is -ln(sigma_11), which is never zero for sigma_11 in (0,1). The final conclusion that the minimum occurs at s = 0 is correct, but the intermediate expression should be fixed.","section":"Section III-B4, Case 2"},{"comment":"The Multinomial test rows appear to contain formatting or entry errors, such as totals of 2000 in a 200-repetition experiment and duplicated TP/TN values. Please verify the table entries.","section":"Section I, Table I"},{"comment":"The global caps (10^4 shots for Random Circuits and 10^6 for MQT Bench) are orders of magnitude below some theoretical shot counts shown in Figure 5; since Figure 8 excludes outliers and false negatives, the censoring mechanism and its effect on the reported medians should be described more explicitly.","section":"Section IV-C1b and Figure 8"}],"recommendation":"major_revision","confidential_remarks":"The paper fits the scope of the journal and the empirical effort is substantial. My main concern is the asymmetric shot budget in Section IV-C1b, which is load-bearing for both RQ1 and RQ2; this is fixable by re-running or re-analyzing with fair per-test budgets. The incorrect Case 1 derivation in Section III-B4 should also be corrected. I do not see grounds for rejection, but the current evidence does not justify the headline claims as stated."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper's real contributions are the formal packaging of quantum unit tests, the introduction of the Inverse test (uncompute and check), and an enormous empirical comparison — 1.8M mutants, nine tests, ≈1.5B repetitions. The complexity analysis and the explicit assumption of fault-tolerant execution are also solid. That part is genuinely useful for the quantum software engineering community.\n\nBut the headline shot-efficiency result has a load-bearing flaw. In Section IV-C1b, every test's shot budget is capped at twice the Inverse test's theoretical requirement from Equation (8) with Pe = 0.05. For any non-identical pure state, the Swap test needs strictly more than 2×N_inv shots to reach the same false-negative rate: N_swap = ln(Pe)/ln((1+f)/2) > 2 ln(Pe)/ln(f) = 2 N_inv for all f < 1. So the Swap test is systematically given fewer shots than its own sufficient budget. That inflates the reported recall gap (0.904 vs 0.824; 0.975 vs 0.958) and censors the shot-count medians in Figure 8. The Inverse test may still be the most shot-efficient test, but this comparison does not establish it. The paper's own RQ2 claim about \"keeping false negatives low\" is exactly what this bias undercuts.\n\nThere is also a genuine algebraic error in Section III-B4 (Case 1). The text claims Tr(ρσ^(1-s)) = 1 for orthogonal diagonal σ with σ₁₁ = 0, then concludes a single shot suffices. The trace is 0 for s < 1, and the equation as written gives Pe = 1, contradicting the conclusion. This needs correcting, though it is not the main source of the empirical bias.\n\nTwo smaller but real issues: the main study has no negative cases, so the claimed \"no false positives\" property is only demonstrated in the small motivational example, and no code or data is linked, which makes the 9.3 CPU-years of experiments hard to verify.\n\nWho is this for: researchers working on quantum software testing and mutation analysis. It deserves a serious referee, but the referee should demand a fair shot budget per test (or a common cap large enough for all), a corrected derivation, a negative-case experiment, and an artifact link. With those changes, the empirical ranking could be credible. Right now, the central comparison is not.\n\nRecommendation: send to peer review, but expect major revision before publication.","headline":"A serious, large-scale study of quantum unit tests whose headline shot-efficiency result is partly biased by an Inverse-anchored shot cap; worth refereeing but needs major revision.","tokens_in":29535,"tokens_out":5281,"would_cite":false,"duration_ms":58199,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A new quantum unit test that appends the inverse of the expected state to a circuit distinguishes faulty from fault-free circuits with fewer measurements than the standard Swap test.","keywords":["quantum unit testing","Inverse test","Swap test","Statevector test","Quantum Chernoff Bound","mutation testing","quantum software engineering","fault-tolerant quantum computing"],"falsifier":"Run the Inverse test on a correct (unmutated) 5-qubit circuit on a noisy device or in a simulator with a realistic depolarizing error model (e.g., $10^{-3}$ per gate) for 10,000 shots; if a non-zero bitstring is observed with probability above the nominal $P_e = 0.05$, then the zero-false-positive guarantee fails on that hardware, and if the shots needed to detect a fixed one-degree rotation error then exceed the Swap test's requirement, the ranking reverses.","tokens_in":28489,"feed_emoji":"⚛️","tokens_out":8970,"duration_ms":83985,"temperature":0.7,"pith_summary":"This paper argues that quantum unit testing is feasible and that a newly proposed Inverse test is the most shot-efficient executable test among the alternatives. The test works by appending the conjugate transpose of the expected state to the circuit under test: if the circuit is correct, the final measurement is always the all-zero bitstring, and any other outcome is proof of a fault. The authors evaluate nine tests on 1.8 million mutated quantum circuits and report that the Inverse test needs 60–66% fewer shots than the Swap test and orders of magnitude fewer than statistical tests, while never producing false positives on noiseless hardware. If these results carry to fault-tolerant machines, quantum software developers gain a cheap, deterministic-style unit oracle.","feed_headline":"Inverse test catches quantum bugs with 60-66% fewer shots","feed_subtitle":"A reversed circuit checks for faults in one pattern, beating the Swap test on 1.8 million mutated quantum circuits.","key_machinery":"The load-bearing object is the Inverse-test circuit: prepare the input state, run the program under test, append the conjugate transpose of the expected-state preparation, and measure; the all-zero bitstring is the unique pass signature. Its efficiency is guaranteed by the Quantum Chernoff Bound, which for this test reduces to a shot-count formula $N \\sim \\ln(P_e)/\\ln(\\min_{0\\le s\\le1}\\operatorname{Tr}(\\rho^s\\sigma^{1-s}))$, and in the diagonal case to $N = \\max(\\lceil\\ln(P_e)/\\ln(\\sigma_{11})\\rceil, 1)$. This formula both predicts the exponential growth in shots as the faulty state approaches the correct one and gives practitioners a concrete stopping rule, which the empirical study confirms is slightly conservative.","core_discovery":"The central discovery is that a reversibility check makes a practically usable unit test: encode the expected behaviour into the test circuit itself, and a single measurement pattern (all zeros) means 'pass', while any deviation is a guaranteed fail. In the authors' formulation the Inverse test prepares the input state, applies the program under test, then applies the conjugate transpose of the operation that would produce the expected state; on a correct circuit this collapses the output to the zero-state with probability one. Using the Quantum Chernoff Bound the paper derives a shot-count rule $N \\sim \\ln(P_e)/\\ln(\\min_{0\\le s\\le1}\\operatorname{Tr}(\\rho^s\\sigma^{1-s}))$, which for a diagonal faulty state becomes $\\lceil\\ln(P_e)/\\ln(\\sigma_{11})\\rceil$ and grows exponentially as the faulty state approaches the correct one. Empirically, over 1,796,880 mutated circuits, the Inverse test was rank-1 in 69%–78% of cases with median shot counts of 2–3, versus 5–9 for the Swap test and roughly 600–1900 for statistical tests; its recall was 0.904 on random circuits and 0.975 on benchmark circuits, with zero false positives by construction.","pith_inferences":["Editorial extension: if the shot-count advantage persists on fault-tolerant devices, quantum test suites could be embedded in continuous-integration pipelines with per-subroutine inverse checks, making quantum software engineering resemble classical unit testing more closely than current sampling-based oracles allow.","Editorial extension: the inverse check could serve as a mid-circuit assertion to localize faults after a subroutine, though the paper only treats it as an end-of-circuit unit test; the extra depth and measurement disruption are the natural costs to investigate.","Editorial extension: Equation (8) is derived for diagonal faulty states; a numerical per-circuit evaluation for non-diagonal and mixed states would produce adaptive shot budgets, a direct generalization the paper leaves open.","Editorial caution: since the Inverse test's circuit is roughly twice as deep as the statistical tests', on near-term noisy hardware gate errors could create false positives and overturn the shot-count ranking; comparing the tests under a realistic noise model is the immediate next step."],"forward_implications":["If the Inverse test is adopted on fault-tolerant hardware, verifying a quantum subroutine could cost 60–66% fewer shots than with the Swap test, directly reducing quantum compute time.","Because it produces no false positives on noiseless execution, a passing Inverse test gives a developer certainty that the circuit matches the expected state, eliminating the flaky-test behaviour that statistical tests exhibit.","The Quantum Chernoff Bound formula supplies a rule for choosing the number of shots, and the observed 3.3% missed-fault rate at a 5% error budget suggests the formula is safe, if slightly conservative.","The formal Arrange–Act–Assert definition of a quantum unit test gives a reusable template: any oracle that recognizes the expected state can be wrapped in the same harness.","For subroutines whose expected states are known, testing small instances (e.g., 1–5 qubits) with a deterministic oracle like the Statevector or Inverse test can cover the main execution paths without full-state simulation."],"supporting_citations":[{"why":"Introduces the symmetrization circuit that the paper uses as the Swap test baseline.","marker":"[11]"},{"why":"Derives the Swap test measurement probability that yields its zero-false-positive property.","marker":"[12]"},{"why":"Provides the prior mutation corpus and the 100,000-shot convention that this study extends and corrects.","marker":"[13]"},{"why":"Supplies the Quantum Chernoff Bound used to derive the shot-count formula for the Inverse test.","marker":"[14]"},{"why":"Establishes the symmetric quantum hypothesis-testing lower bound that grounds the Chernoff analysis.","marker":"[15]"},{"why":"Supplies the quantum benchmark circuits used as test subjects in the empirical evaluation.","marker":"[24]"},{"why":"Proposes quantum test case circuits, the conceptual precursor that this work formalizes as a unit test.","marker":"[42]"},{"why":"Defines the gate replacement, deletion, and insertion mutation operators used to create faulty circuits.","marker":"[68]"},{"why":"Demonstrates reversing a quantum circuit to recover correct output, the principle behind the Inverse test.","marker":"[30]"}],"fun_headline_variants":["Inverse test: 2-3 shots vs 600+ for statistical quantum tests","Quantum unit testing: Inverse test cuts shots by 60-66%","Reversibility check finds faults with zero false positives","Inverse test beats Swap test on 1.8M mutated circuits","Inverse test: rank-1 in 69-78% of cases with 2-3 shots"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"All performance claims assume a fault-tolerant quantum computer or an ideal noise-free simulator; if real hardware noise makes the inverse circuit's extra depth create spurious failures, the zero-false-positive guarantee and the shot-count advantage break down.","fun_headline_variants_meta":{"raw":{"variants":["Inverse test: 2-3 shots vs 600+ for statistical quantum tests","Quantum unit testing: Inverse test cuts shots by 60-66%","Reversibility check finds faults with zero false positives","Inverse test beats Swap test on 1.8M mutated circuits","Inverse test: rank-1 in 69-78% of cases with 2-3 shots"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000412,"raw_usage":{"total_tokens":2157,"prompt_tokens":997,"completion_tokens":1160,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":613,"completion_tokens_details":{"reasoning_tokens":1058}},"tokens_in":613,"tokens_out":1160,"duration_ms":10006,"temperature":1.0,"reasoning_tokens":1058,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T14:54:39.460023+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the Inverse test on a correct (unmutated) 5-qubit circuit on a noisy device or in a simulator with a realistic depolarizing error model (e.g., $10^{-3}$ per gate) for 10,000 shots; if a non-zero bitstring is observed with probability above the nominal $P_e = 0.05$, then the zero-false-positive guarantee fails on that hardware, and if the shots needed to detect a fixed one-degree rotation error then exceed the Swap test's requirement, the ranking reverses.","supporting_citations":[{"cited_title":"MQT Bench: Bench- marking software and design automation tools for quantum computing,","cited_arxiv_id":null,"evidence_quote":"Supplies the quantum benchmark circuits used as test subjects in the empirical evaluation."},{"cited_title":"Automatic generation of test circuits for the verification of quantum deterministic algorithms,","cited_arxiv_id":null,"evidence_quote":"Proposes quantum test case circuits, the conceptual precursor that this work formalizes as a unit test."}],"review_version":1}