REVIEW 4 major objections 5 minor 1 cited by
CertiQ: A Mostly-automated Verification of a Realistic Quantum Compiler
T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read A mostly-automated framework verifies 26 of Qiskit's 30 compiler passes and catches three real bugs.
desk verdict Promising push-button verification for Qiskit passes with real bug findings, but the missing artifact and unproven synthesizer/wrapper equivalence keep the headline claim from being fully substantiated. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The quantum circuit calculus is the mechanism that carries the argument. It represents a circuit as a symbolic expression $app(C,Q)$ that applies $C$ to a register of symbolic qubits, with rewriting rules—adjacent CNOT cancellation, SWAP argument exchange, gate commutation, and similar local identities—whose soundness is proven once against the matrix semantics in an interactive proof assistant. When a pass is written in CertiQ, the synthesizer expands branches, abstracts loops through provided templates, and turns the symbolic execution into proof obligations of the form $P_1 \wedge P_2 \wedge \neg G$; an SMT solver checks unsatisfiability, so a satisfying assignment would be a counterexample. A verified library of transformation functions and forward-simulation-based conversion functions supplies the reusable lemmas that keep most verifications within seconds.
What would settle it
Feed a randomly generated Qiskit circuit through one of the 26 CertiQ-generated passes and compare input and output circuits with a full state-vector simulator; any unitary mismatch would show the generated executable diverges from the verified symbolics, refuting the claim.
Extended reading notes
Core claim
CertiQ's central claim is that real-world quantum compiler passes can be verified mostly automatically by reducing each pass to a semantic-preservation check over symbolic circuits and encoding that check as an SMT satisfiability problem. The framework contributes a quantum circuit calculus whose symbolic execution and gate rewriting rules allow adjacent gates to be cancelled, commuted, or swapped without constructing unitary matrices, plus a verified library of high-level transformations and representation conversions that make proofs modular. Using these components, the authors re-implement and verify 26 of the 30 passes in Qiskit version 0.192, integrate the generated code into Qiskit through a wrapper that converts the circuit's directed-acyclic-graph representation to the textual intermediate representation and back, and report that every generated pass passes Qiskit's regression tests. The verification process also identifies three bugs in the original Qiskit implementation, including a routing pass (lookahead_swap) that can fail to terminate on a 16-qubit coupling map.
Load-bearing premise
The guarantee is only as strong as the assumption that the executable Python code CertiQ generates and wires into Qiskit behaves exactly like the symbolic circuit program that was verified, and that every user-declared external function specification is faithful.
Editorial extensions
If this is right
- Qiskit's transformation passes can be checked for semantic preservation and termination without constructing full unitary matrices, making verification practical on circuits of any size.
- Verified passes can be dropped into Qiskit directly because CertiQ generates both the proof and the executable code from one source, and the generated code passes Qiskit's own regression tests.
- The verified library of circuit transformations and representation conversions can be reused to verify future passes with little more than loop and external-function annotations.
- Because three real bugs were found—including a non-terminating routing pass—automatic verification of this kind can act as a practical bug-finding tool for quantum compilers.
Reading between the lines
- The same recipe should transfer to other quantum compilers whose passes manipulate circuit intermediate representations, since the circuit calculus and the simulation-diagram criterion are not tied to Qiskit's specific data structures.
- The non-termination bug suggests that monotonic-function termination checks, as CertiQ uses, are worth adding to routing passes generally, especially as hardware coupling maps grow and greedy heuristics get more complicated.
- CertiQ's explicit refusal to convert Bloch-sphere representations beyond one qubit implies that any future verified optimizer wanting multi-qubit phase information will need a representation that tracks global phases, not just the Bloch projection.
- The framework's guarantee is compositional rather than absolute: each external function is trusted by specification, so integrating CertiQ with more powerful third-party libraries will require those specifications to be verified or narrowed.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents CertiQ, a framework for mostly-automated verification of quantum compiler passes in Qiskit. Compiler passes are written in a Python-like annotated language; a synthesizer produces both a symbolic representation used to generate SMT-based verification conditions and executable Python code compatible with Qiskit. The framework introduces a quantum circuit calculus with rewriting rules, a verified library of circuit transformations and data conversions, and a verifier that encodes proof obligations into Z3 formulas. The authors report re-implementing and verifying 26 of 30 Qiskit passes, detecting three bugs in Qiskit, and confirming that the generated passes pass Qiskit's regression tests without noticeable performance loss.
Significance. If the central claim holds, CertiQ would be a landmark step toward practical verification of real-world quantum compilers: it would be the first mostly-automated framework to verify realistic Qiskit passes, with an SMT-based equivalence check that avoids exponential denotational reasoning and a reusable verified library. The paper also provides falsifiable, concrete results: 26 verified passes, three reported bugs, and regression-test passage. Strengths include an honest identification of external functions as part of the trusted computing base, a forward-simulation framework for data conversions, and a demonstrated use of the framework to find a non-termination bug in lookahead_swap. However, the machine-checked proofs of the calculus are not shipped, the synthesizer's correctness is not established, and several appendix references and figures are missing, so the claimed end-to-end guarantee cannot currently be checked from the manuscript alone.
major comments (4)
- [§6.1–6.2] The correctness of the synthesizer is load-bearing but unproven. The paper claims that CertiQ-generated code can be integrated into Qiskit and that the passes are verified, yet the verification applies to the symbolic representation, not the executable Python code produced by the synthesizer or the DAG-to-OpenQASM-to-DAG wrapper. The paper gives only an operational description of the synthesizer and wrapper and no refinement theorem, no formal semantics for the CertiQ source language, and no argument that both generated artifacts preserve the source program's semantics. A discrepancy between the verified symbolic program and the executed code would invalidate the central claim that the Qiskit passes are verified. The authors should either prove the synthesizer and wrapper correct, or explicitly place them in the trusted computing base and weaken the corresponding claims.
- [§4, 'Soundness proofs' paragraph] The foundation of the verification chain is not independently checkable. The text states that the symbolic execution rules, rewriting rules, and shortcut lemmas 'are treated as axioms' and that their soundness was proven in Coq, but no Coq developments, proof artifacts, or even detailed proof sketches are provided, and the referenced Appendix appears as 'Appendix??'. Since every pass verification and every library-function verification inherits these axioms, the entire claim of 'verified' passes rests on uncheckable assumptions in the preprint. The authors should release the Coq proofs or, at a minimum, provide a complete account of the axioms and their soundness arguments in the appendix.
- [§6.1, loop templates] The three loop templates are claimed to pre-define loop invariants, but the paper does not prove that the verification conditions generated from a template correctly capture the semantics of the generated loop, including base cases and termination. In the iterate_all_gates example, the assertions constrain i but there is no explicit proof obligation for the base case that the empty circuit is equivalent to the first zero gates, and the relationship between the template's formal assertions and the actual behavior of the synthesized loop in Qiskit is not established. This gap is part of the synthesizer-correctness issue and is load-bearing for any pass whose proof relies on these templates.
- [§5, 'Verified quantum data conversions'] The forward-simulation argument for data conversions is not fully presented. The paper refers to a simulation diagram that is missing ('as shown in??' and 'Fig. 7' are absent from the text), and the definition of the conversion relation and the proof that the listed conversions satisfy the simulation conditions are only sketched. Since the claimed detection of a severe bug in Qiskit's representation conversion depends on this argument, the missing diagram and formal details prevent the reader from verifying the conversion-soundness claim.
minor comments (5)
- [§6.1] In the description of iterate_all_gates, the text says 'the new_circ after this iteration must be equivalent to the first i 1 gates of circ', which appears to be a typo for 'i+1 gates'.
- [§8.3] The sentence 'we find a counter-example circuit on coupling map of the IBM 16 qubit device, where the lookahead_swap pass does not terminate on' is grammatically incomplete and should be rephrased.
- [Various] Multiple references to 'Appendix??' and 'as shown in??' appear throughout the paper (e.g., §4 and §5); these missing references and figures should be resolved before publication.
- [§9] The evaluation would benefit from concrete experimental details: how regression tests were run, what 'no noticeable performance loss' means operationally, and what measurements support the claim about verification times finishing 'within seconds' or 'within one minute'.
- [§2.1 and §8.1] The notation for gates and the matrix representations in Table 1 contains minor typesetting issues (e.g., 'm1g−−−→' in Fig. 8) that should be corrected.
Circularity Check
No circularity: verification reduces to external calculus-soundness trust, not to its own outputs.
full rationale
The claimed derivation chain is: annotated CertiQ source -> synthesizer produces symbolic VCs and executable Python -> verifier checks VCs using quantum-circuit-calculus rules and verified-library specs -> pass is extracted and run in Qiskit. I can find no step where a conclusion is equivalent to its input by construction or where a fitted parameter is renamed as a prediction. The calculus rules are independent lemmas about matrix semantics: the paper says "The symbolic execution, rewriting rules, and the shortcut lemmas are treated as axioms in CertiQ and are proven using the denotational semantics with the Coq proof assistant." Using such rules to verify passes is a normal axiom/lemma application, not a self-definitional reduction. Real gaps exist but are not circularity: the Coq soundness proofs and shortcut lemmas are referenced but not supplied (the text points to "Appendix??"), and the synthesizer/wrapper equivalence between the symbolic program and the executable Python is not given a refinement theorem (Section 6.2). Those are trust/soundness gaps. Self-citations to the authors' prior work (e.g., [18], [19], [26]) appear only in background and are not load-bearing for the verification claims. External checks—Qiskit regression tests and bugs confirmed by the Qiskit team—provide independent validation. Since no specific equation or renamed fitting can be quoted that reduces the verification claim to its assumptions, the circularity score is 0.
Assumptions & free parameters
assumptions (5)
- domain assumption The quantum circuit calculus rewriting rules (SWAP, cancellation, commutation, bridging) are sound with respect to denotational semantics.
- ad hoc to paper The CertiQ synthesizer preserves the semantics of the source program when generating both the symbolic representation and the executable Python code.
- ad hoc to paper The three pre-defined loop templates (iterate_all_gates, while_gate_remaining, collect_runs) correctly encode the loop semantics, including base cases and invariants.
- domain assumption Python lists and other trusted library components are correctly implemented, and external functions satisfy their user-specified contracts.
- domain assumption The Z3 SMT solver is sound for the theory of arrays and uninterpreted functions used in the verification conditions.
Cite this review
Pith. "Pith review of CertiQ: A Mostly-automated Verification of a Realistic Quantum Compiler." pith.science (2026). https://pith.science/paper/422YCUPE
@misc{pith2026190808963,
author = {Pith},
title = {Pith review of: CertiQ: A Mostly-automated Verification of a Realistic Quantum Compiler},
year = {2026},
howpublished = {\url{https://pith.science/paper/422YCUPE}},
note = {Machine review of arXiv:1908.08963}
}
read the original abstract
We present CertiQ, a verification framework for writing and verifying compiler passes of Qiskit, the most widely-used quantum compiler. To our knowledge, CertiQ is the first effort enabling the verification of real-world quantum compiler passes in a mostly-automated manner. Compiler passes written in the CertiQ interface with annotations can be used to generate verification conditions, as well as the executable code that can be integrated into Qiskit. CertiQ introduces the quantum circuit calculus to enable the efficient checking of equivalence of quantum circuits by encoding such a checking procedure into an SMT problem. CertiQ also provides a verified library of widely-used data structures, transformation functions for circuits, and conversion functions for different quantum data representations. This verified library not only enables modular verification but also sheds light on future quantum compiler design. We have re-implemented and verified 26 (out of 30) Qiskit compiler passes in CertiQ, during which three bugs are detected in the Qiskit implementation. Our verified compiler pass implementations passed all of Qiskit's regression tests without showing noticeable performance loss.
Figures
Figures from the paper (6 more)
Forward citations
Cited by 1 Pith paper
-
An experience-based classification of quantum bugs in quantum software
A largely experience-based catalog of 14 quantum software bugs proposes an intersection-aware classification and finds no one debugging strategy maps cleanly onto bug classes.
Reference graph
Works this paper leans on
-
[1]
Towards Large-scale Functional Verification of Universal Quantum Circuits
Matthew Amy. Towards Large-scale Functional Verification of Universal Quantum Circuits. Electronic Proceedings in Theoretical Computer Science, 287:1–21, jan 2019
work page 2019
-
[2]
Matthew Amy, Martin Roetteler, and Krysta M. Svore. Ver- ified Compilation of Space-Efficient Reversible Circuits. In Computer Aided Verification, pages 3–21. International Con- ference on Computer Aided Verification, 2017
work page 2017
-
[3]
F Bloch. Nuclear Induction.Phys. Rev., 70(7-8):460–474, oct 1946
work page 1946
-
[4]
Juan Carlos Garcia-Escartin and Pedro Chamorro-Posada. Equivalent Quantum Circuits. Technical report, Universidad de Valladolid, Dpto. Teoria de la Senal e Ing., 2011
work page 2011
-
[5]
Available electronically athttp: //coq.inria.fr/doc
The Coq Development Team.The Coq Reference Manual, version 8.4, August 2012. Available electronically athttp: //coq.inria.fr/doc
work page 2012
-
[6]
Open quantum assembly language.arXiv preprint arXiv:1707.03429, 2017
Andrew W Cross, Lev S Bishop, John A Smolin, and Jay M Gambetta. Open quantum assembly language.arXiv preprint arXiv:1707.03429, 2017
arXiv 2017
-
[7]
Leonardo DeMoura andNikolaj Bjørner. Z3: An efficient SMT Solver. InInternational Conference on Tools and Algorithms for the Construction and Analysis of Systems, volume 4963 LNCS, pages 337–340, 2008
work page 2008
-
[8]
Cambridge University Press, 2009
Philippe Gille and Tamas Szamuely.Central Simple Algebras and Galois Cohomology. Cambridge University Press, 2009
work page 2009
Show all 30 references
-
[9]
Verified Optimization in a Quantum Intermediate Representation.arxiv, apr 2019
Kesha Hietala, Robert Rand, Shih-Han Hung, Xiaodi Wu, and Michael Hicks. Verified Optimization in a Quantum Intermediate Representation.arxiv, apr 2019. 11
2019
-
[10]
LLVM: A Compilation Framework for Lifelong Program Analysis & Transformation
Chris Lattner and Vikram Adve. LLVM: A Compilation Framework for Lifelong Program Analysis & Transformation. Technical report, UIUC, 2003
2003
-
[11]
Formal verification of a realistic compiler.Com- munications of the ACM, 52(7):107–115, 2009
Xavier Leroy. Formal verification of a realistic compiler.Com- munications of the ACM, 52(7):107–115, 2009
2009
-
[12]
Q| SI?: A quantum programming environment
Shusen Liu, Xin Wang, Li Zhou, Ji Guan, Yinan Li, Yang He, Runyao Duan, and Mingsheng Ying. Q| SI?: A quantum programming environment. InLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intel- ligence and Lecture Notes in Bioinformatics), volu...
2018
-
[13]
Lopes, David Menendez, Santosh Nagarakatte, and John Regehr
Nuno P. Lopes, David Menendez, Santosh Nagarakatte, and John Regehr. Provably correct peephole optimizations with alive. InProceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’15, page 22–32, New York, NY, USA, 2015. Associati...
2015
-
[14]
Forward and backward simulations
Nancy Lynch and Frits Vaandrager. Forward and backward simulations. Information and Computation, 121(2):214–233, 1995
1995
-
[15]
Quantum Circuit Simplification and Level Compaction.IEEE Transactions on Computer-Aided Design of Integrated Cir- cuits and Systems, 27(3):436–444, mar 2008
D Maslov, G W Dueck, D M Miller, and C Negrevergne. Quantum Circuit Simplification and Level Compaction.IEEE Transactions on Computer-Aided Design of Integrated Cir- cuits and Systems, 27(3):436–444, mar 2008
2008
-
[16]
McKay, Thomas Alexander, Luciano Bello, Michael J
David C. McKay, Thomas Alexander, Luciano Bello, Michael J. Biercuk, Lev Bishop, Jiayin Chen, Jerry M. Chow, AntonioD. Córcoles, DanielEgger, StefanFilipp, JuanGomez, Michael Hush, Ali Javadi-Abhari, Diego Moreda, Paul Na- tion, Brent Paulovicks, Erick Winston, Christopher J. ...
2018
-
[18]
Baker, Ali Javadi-Abhari, Fred- eric T
Prakash Murali, Jonathan M. Baker, Ali Javadi-Abhari, Fred- eric T. Chong, and Margaret Martonosi. Noise-Adaptive Compiler Mappings for Noisy Intermediate-Scale Quantum Computers. pages 1015–1029. Association for Computing Machinery (ACM), 2019
2019
-
[19]
Mckay, Margaret Martonosi, and Ali Javadi-Abhari
Prakash Murali, David C. Mckay, Margaret Martonosi, and Ali Javadi-Abhari. Software mitigation of crosstalk on noisy intermediate-scale quantum computers. In Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operati...
2020
-
[20]
Scaling symbolic evaluation for automated verification of systems code with serval
Luke Nelson, JamesBornholt, Ronghui Gu, AndrewBaumann, Emina Torlak, and Xi Wang. Scaling symbolic evaluation for automated verification of systems code with serval. InPro- ceedings of the 27th ACM Symposium on Operating Systems Principles, SOSP ’19, pages 225–242, New York, NY, USA,
-
[21]
Hyperkernel: Push-button verification of an os kernel
Luke Nelson, Helgi Sigurbjarnarson, Kaiyuan Zhang, Dy- lan Johnson, James Bornholt, Emina Torlak, and Xi Wang. Hyperkernel: Push-button verification of an os kernel. In Proceedings of the 26th Symposium on Operating Systems Principles, SOSP ’17, pages 252–269, New York, NY, USA,
-
[22]
Nielsen and Isaac L
Michael A. Nielsen and Isaac L. Chuang.Quantum Compu- tation and Quantum Information: 10th Anniversary Edition. Cambridge University Press, New York, NY, USA, 10th edi- tion, 2011
2011
-
[23]
Available at https://github.com/ Qiskit/qiskit-terra/issues/1871
Qiskit bug report, 2018. Available at https://github.com/ Qiskit/qiskit-terra/issues/1871
2018
-
[24]
ReQWIRE: Reasoning about Reversible Quantum Circuits
Robert Rand, Jennifer Paykin, Dong-Ho Lee, and Steve Zdancewic. ReQWIRE: Reasoning about Reversible Quantum Circuits. arXiv e-prints, page arXiv:1901.10118, Jan 2019
1901 arXiv
-
[25]
QWIRE Practice: Formal Verification of Quantum Circuits in Coq
Robert Rand, Jennifer Paykin, and Steve Zdancewic. QWIRE Practice: Formal Verification of Quantum Circuits in Coq. Electronic Proceedings in Theoretical Computer Science, 2018
2018
-
[26]
Schuster, Henry Hoffmann, and Frederic T
Yunong Shi, Nelson Leung, Pranav Gokhale, Zane Rossi, David I. Schuster, Henry Hoffmann, and Frederic T. Chong. Optimized Compilation of Aggregated Instructions for Real- istic Quantum Computers. InASPLOS, Providence, Rhode Island, 2019
2019
-
[27]
Push-Button Verification of File Systems via Crash Refinement
Helgi Sigurbjarnarson, James Bornholt, Emina Torlak, and Xi Wang. Push-Button Verification of File Systems via Crash Refinement. In the Proceedings of the 12th USENIX Sym- posium on Operating Systems Design and Implementation (OSDI, 2016
2016
-
[28]
Verified translation between low-level quantum languages
Kartik Singhal, Robert Rand, and Michael Hicks. Verified translation between low-level quantum languages. The First International Workshop on Programming Languages for Quan- tum Computing, jan 2020
2020
-
[29]
Available at https: //github.com/Qiskit/qiskit-terra/issues
Qiskit Terra Github issue page, 2018. Available at https: //github.com/Qiskit/qiskit-terra/issues
2018
-
[30]
Viamontes, Igor L
George F. Viamontes, Igor L. Markov, and John P. Hayes. Checking equivalence of quantum circuits and states. In Georges G E Gielen, editor,ICCAD. IEEE, 2007
2007
-
[31]
Floyd–hoare logic for quantum programs
Mingsheng Ying. Floyd–hoare logic for quantum programs. ACM Transactions on Programming Languages and Systems, 33(6):1–49, dec 2011. 12
2011
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.