REVIEW 3 major objections 6 minor 20 references
Forensics of Error Rates of Quantum Hardware
T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read A user can rank a quantum backend's qubit links by error rate just by counting which links its transpiler picks, with agreement within two bins for up to 83.5% of links.
desk verdict Plausible side-channel for backend qubit quality, but the evidence is too thin and the topology confound is untested. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the transpiler's fidelity metric $E = (1-E_{g_1})^{n_1}(1-E_{g_2})^{n_2}\cdots(1-E_{g_N})^{n_N}$, the product over all executed gates of one minus each gate's error rate; the transpiler chooses among lowest-gate-count mappings the one with the largest $E$. This turns each decision into a biased sample of low-error links. The paper's ranking mechanism is then the frequency with which each physical qubit link appears in the extracted topologies of many transpiled circuits, converted into six equal bins of 24 links and compared with the same binning applied to the vendor's published two-qubit (ECR) error rates. The mechanism carries the argument because selection frequency is observable from ordinary transpiled output, while the error rates themselves are the hidden quantity being estimated.
What would settle it
Transpile the same suite of circuits on a backend while forcing the mapping cost function to ignore error rates and minimize only gate count and depth; if link-selection frequencies still match the published error bins at the reported rates, the agreement is an artifact of topology rather than a side-channel on error rates, whereas a drop in agreement would confirm the proposed mechanism.
Extended reading notes
Core claim
The paper's central claim is that the relative error rates of a quantum backend's physical qubit links can be recovered from the transpiled circuits returned to the user, without direct access to calibration data. The transpiler is assumed to minimize aggregate gate error through a fidelity metric $E=(1-E_{g_1})^{n_1}(1-E_{g_2})^{n_2}\cdots(1-E_{g_N})^{n_N}$, so the physical links it selects most often across many circuits are the least error-prone ones. Ranking 144 links into six equal bins of 24 by selection frequency, and binning the same links by published ECR (echoed cross-resonance) two-qubit error rates, the authors find that the two rankings agree within two bin ranks for roughly 83.5% of links on one 127-qubit backend and 80% on another. The paper therefore claims that link-selection frequency in ordinary transpiled programs is a usable, coarse side-channel for the backend's error-rate structure.
Load-bearing premise
The method's load-bearing premise is that the transpiler's link choices are driven mainly by the links' error rates; if gate count, circuit depth, or swap minimization dominate routing, link-selection frequency will reflect circuit topology rather than hardware quality.
Editorial extensions
If this is right
- A user who submits several large random circuits to a cloud backend can reconstruct a coarse quality ranking of that backend's qubit links using only the returned transpiled circuits.
- A suspicious mapping that avoids the link bins most favored by the transpiler on other runs is evidence that the vendor did not allocate the highest-fidelity qubits.
- The side-channel works even when the provider hides calibration data or hardware identity, since the signal is carried by the transpiled program itself.
- If the same method is run repeatedly, changes in link-selection frequencies would indicate re-calibration, hardware drift, or a change in the vendor's allocation policy.
Reading between the lines
- I infer that this frequency-based ranking could be automated as a per-job verification service: a client submits probe circuits, computes bins, and compares the vendor-chosen mapping against the client's own ranking before paying.
- The observed agreement likely depends on the circuit generation procedure; five random 100-qubit circuits are one sample, and circuits concentrated on a small region of the coupling map should make only local links visible, which is a testable boundary condition.
- Because the mapping cost function is set by the software stack, the frequency signal mixes hardware quality with transpiler policy; a change in compiler version could shift the bins even with unchanged hardware.
- A finer-grained use of the same data would be to look not at aggregate link counts but at pairwise ordering of links within a bin, which the paper's equal-width bins intentionally throw away.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a forensic method to estimate relative two-qubit (ECR) error rates of qubit links on IBM quantum backends from transpiled circuits alone. The authors extract the physical qubit topology from transpiled QASM using the technique of [8], count how frequently each physical link is selected across five random 100-qubit circuits transpiled on IBM Sherbrooke and IBM Brisbane, and group the 144 links into six equal-sized bins by selection frequency. They compare this bin ranking against a bin ranking derived from publicly reported ECR error rates, reporting that for 80% (Brisbane) and 83.5% (Sherbrooke) of links the predicted bin differs by at most two bins from the ECR-based bin. The paper concludes that link-selection frequency is a usable side-channel for coarse backend error-rate ranking.
Significance. If the central modeling premise holds, the method would provide a non-invasive, user-side forensic tool for validating whether a cloud quantum provider transpiled and executed a circuit on high-fidelity qubit links, which matters for third-party quantum cloud security and trust. The approach uses only the transpiled circuit artifact, which is a realistic deliverable in the threat model, and it builds on an existing topology-extraction method. However, the paper's significance is conditional: the current empirical support is thin (five circuits, no statistical error bars, no baseline comparison), and the central inference that selection frequency is driven by ECR error rates rather than by routing topology is not tested by any control experiment. Strengths of the paper are its clearly stated threat model and the explicit use of an external public benchmark for comparison, which avoids circular derivation of the ranking.
major comments (3)
- [Section III-A, Eq. (1)] The central modeling premise is untested. The paper states that the transpiler first finds mappings with the lowest gate count, circuit depth, and number of SWAP gates, and only then chooses among them using the fidelity metric E in Eq. (1). If routing passes such as Sabre minimize SWAP count rather than ECR-weighted path cost, then link-selection frequency will be dominated by graph-topology bottlenecks (e.g., central heavy-hex edges) rather than by hardware error rates. The reported 80-83.5% agreement could then arise from spatial correlation between topology and ECR, or from the fact that the same backend error data supplied to the transpiler are later used as the ground truth for comparison. The paper provides no control experiment, such as transpiling with permuted or synthetic error rates, or comparing the frequency ranking against a topology-based centrality ranking, to isolate the causal role of ECR rates. This is load-bearing because the entire forensic claim rests on selection frequency being a proxy for link error rate.
- [Section III-B] The statistical support for the headline claim is insufficient. Only five random 100-qubit circuits are used, with no repeated transpilation runs, no reporting of variance across circuits or across transpiler random seeds, and no confidence intervals on the reported percentages. With 144 links divided into 6 bins of 24 links each, the agreement metric counting a difference of up to 2 bins as success has a high chance baseline: for two independent uniform random bin assignments, the probability of a bin difference of at most 2 is about 67%. The reported 80-83.5% is therefore not far above chance, and the paper does not compare against a null model or report per-bin confusion statistics. The claim that the method is 'valid' requires a baseline and an error analysis.
- [Section III-B] The temporal alignment between the transpiled circuits and the public ECR snapshot is not given. The paper does not state the dates on which the IBM Sherbrooke and IBM Brisbane backends were transpiled, the date of the ECR data used as ground truth, or the calibration schedule of the backends. Since IBM error rates drift over time, a mismatch of even a few days could materially change the ground-truth bin ranking, especially for links near bin boundaries. The absence of these dates makes the quantitative agreement figures impossible to reproduce or interpret.
minor comments (6)
- [Section III-B] The paper does not describe the random circuit generation parameters (depth, gate types, random seed) or the transpiler optimization level used, which prevents reproduction of the experimental setup.
- [Section II-A4] The definition of T2 time is grammatically incomplete: 'measures the time it takes to collapse from the superposition state to either |0⟩ or|1⟩' should describe the decay of coherence rather than a collapse event.
- [Fig. 2 and Fig. 3] The figures showing ECR error information and frequency-based rankings are not referenced in the body text with specific discussion; the captions also lack units and color-bar legends, making it hard to read the actual error values.
- [Abstract and Section I] The abstract and introduction state that public error information 'may not be accurate' while the method uses that same public information as ground truth for validation; this tension should be discussed explicitly.
- [References] Reference [1] is a bare URL without a title or access date, and several references are arXiv preprints without version or DOI information; a consistent citation format is needed.
- [Section I] The contributions list contains a formatting artifact: item (i) starts with '(i)Fingerprinting' and the text says '(b) Creating trust' after '(i)', suggesting a missing item (a); the list should be cleaned up.
Circularity Check
No circularity: the frequency-based bin ranking is derived from transpiler behavior, not from the public ECR data used as ground truth.
full rationale
The paper's derivation chain is: (1) the Qiskit transpiler chooses logical-to-physical mappings that minimize aggregate error using the fidelity metric E of Section III-A; (2) therefore links with lower ECR error are used more often across many circuits; (3) the paper counts link-selection frequency in five random 100-qubit circuits and bins the 144 links by that frequency; (4) it bins the same links by publicly available ECR error rates; (5) it reports the agreement. Step (3) does not use the public ECR data to construct the frequency bins, and no parameter is fitted to the benchmark; the binning is a fixed six-bin equal-size split. The public ECR data is the mechanism that makes the transpiler favor better links, so using it as the comparison ground truth is a standard side-channel validation, not a logical circularity. The dependence on the authors' prior topology-extraction work [8] is a methodological dependency, but it is not a circular reduction: the 80-83.5% agreement with the external IBM calibration data provides an independent empirical check of the whole pipeline, and the paper does not invoke any self-cited uniqueness theorem or ansatz to force the result. The main weaknesses (topology may dominate routing; only five circuits; public data may not equal executed hardware) are empirical validity risks, not circularity.
Assumptions & free parameters
free parameters (2)
- Number of bins and bin size =
6 bins, 24 links each
- Number of random circuits =
5
assumptions (3)
- domain assumption Transpiler mapping and routing choices minimize aggregate error according to the fidelity metric E.
- domain assumption Public ECR error rates reflect backend quality during the transpilation period.
- domain assumption Physical topology extraction from [8] correctly recovers the backend coupling map.
Cite this review
Pith. "Pith review of Forensics of Error Rates of Quantum Hardware." pith.science (2026). https://pith.science/paper/K5LLE3R7
@misc{pith2026250511706,
author = {Pith},
title = {Pith review of: Forensics of Error Rates of Quantum Hardware},
year = {2026},
howpublished = {\url{https://pith.science/paper/K5LLE3R7}},
note = {Machine review of arXiv:2505.11706}
}
read the original abstract
There has been a rise in third-party cloud providers offering quantum hardware as a service to improve performance at lower cost. Although these providers provide flexibility to the users to choose from several qubit technologies, quantum hardware, and coupling maps; the actual execution of the program is not clearly visible to the customer. The success of the user program, in addition to various other metadata such as cost, performance, & number of iterations to converge, depends on the error rate of the backend used. Moreover, the third-party provider and/or tools (e.g., hardware allocator and mapper) may hold insider/outsider adversarial agents to conserve resources and maximize profit by running the quantum circuits on error-prone hardware. Thus it is important to gain visibility of the backend from various perspectives of the computing process e.g., execution, transpilation and outcomes. In this paper, we estimate the error rate of the backend from the original and transpiled circuit. For the forensics, we exploit the fact that qubit mapping and routing steps of the transpilation process select qubits and qubit pairs with less single qubit and two-qubit gate errors to minimize overall error accumulation, thereby, giving us clues about the error rates of the various parts of the backend. We ranked qubit links into bins based on ECR error rates publicly available, and compared it to the rankings derived from our investigation of the relative frequency of a qubit link being chosen by the transpiler. For upto 83.5% of the qubit links in IBM Sherbrooke and 80% in IBM Brisbane, 127 qubit IBM backends, we are able to assign a bin rank which has a difference upto 2 with the bin rank assigned on the basis of actual error rate information.
Figures
Reference graph
Works this paper leans on
-
[8]
Quantum PUF for Security and Trust in Quantum Computing,
K. Phalak, A. A. -. Saki, M. Alam, R. O. Topaloglu and S. Ghosh, "Quantum PUF for Security and Trust in Quantum Computing," in IEEE Journal on Emerging and Selected Topics in Circuits and Systems, vol. 11, no. 2, pp. 333-342, June 2021, doi: 10.1109/JETCAS.2021.3077024
arXiv 2021
-
[1]
https://quantum-computing.ibm.com/services/resources/docs/resources /manage/systems/queue
-
[3]
Potential of quantum computing for drug discovery,
Y. Cao, J. Romero, and A. Aspuru-Guzik, “Potential of quantum computing for drug discovery,” IBM Journal of Research and Development, vol. 62, no. 6, pp. 6–1, 2018
2018
-
[4]
Detecting Fraudulent Services on Quantum Cloud Platforms via Dynamic Fingerprinting
Wu, Jindi, Tianjie Hu, and Qun Li. "Detecting Fraudulent Services on Quantum Cloud Platforms via Dynamic Fingerprinting." arXiv preprint arXiv:2408.11203 (2024)
arXiv 2024
-
[5]
Chuanqi Xu, Ferhat Erata, and Jakub Szefer. 2023. Exploration of Power Side-Channel Vulnerabilities in Quantum Computer Controllers. In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security (CCS ’23). Association for Computing Machinery, New York, NY, USA, 579–593. https://doi.org/10.1145/3576915.3623118
arXiv 2023
-
[6]
Q-ID: Lightweight Quantum Network Server Identification through Fingerprinting
Wu, Jindi, Tianjie Hu, and Qun Li. "Q-ID: Lightweight Quantum Network Server Identification through Fingerprinting." IEEE Network (2024)
work page 2024
-
[7]
Fast fingerprinting of cloud-based nisq quantum computers
Smith, Kaitlin N., Joshua Viszlai, Lennart Maximilian Seifert, Jonathan M. Baker, Jakub Szefer, and Frederic T. Chong. "Fast fingerprinting of cloud-based nisq quantum computers." In 2023 IEEE International Symposium on Hardware Oriented Security and Trust (HOST), pp. 1-12. IEEE, 2023
work page 2023
-
[9]
Forensics of Transpiled Quantum Circuits
Roy, Rupshali, Archisman Ghosh, and Swaroop Ghosh. "Forensics of Transpiled Quantum Circuits." arXiv preprint arXiv:2412.18939 (2024)
work page Pith review arXiv 2024
Show all 20 references
-
[10]
Karpuzcu
Salonik Resch and Ulya R. Karpuzcu. 2021. Benchmarking Quantum Computers and the Impact of Quantum Noise. ACM Comput. Surv. 54, 7, Article 142 (September 2022), 35 pages. https://doi.org/10.1145/3464420
2021 doi
-
[11]
Dueck, Rolf Drechsler: RevLib: An Online Resource for Reversible Functions and Reversible Circuits
Robert Wille, Daniel Große, Lisa Teuber, Gerhard W. Dueck, Rolf Drechsler: RevLib: An Online Resource for Reversible Functions and Reversible Circuits. Int'l Symp. on Multi-Valued Logic, 2008
2008
-
[12]
The Quantum Imitation Game: Reverse Engineering of Quantum Machine Learning Models
Ghosh, Archisman, and Swaroop Ghosh. "The Quantum Imitation Game: Reverse Engineering of Quantum Machine Learning Models." arXiv preprint arXiv:2407.07237 (2024)
2024 arXiv
-
[13]
Advanced digital forensics and anti-digital forensics for IoT systems: Techniques, limitations and recommendations
Yaacoub, Jean-Paul A., Hassan N. Noura, Ola Salman, and Ali Chehab. "Advanced digital forensics and anti-digital forensics for IoT systems: Techniques, limitations and recommendations." Internet of Things 19 (2022): 100544
2022
-
[14]
Watermarking of Quantum Circuits
Roy, Rupshali, and Swaroop Ghosh. "Watermarking of Quantum Circuits." arXiv preprint arXiv:2409.01484 (2024)
2024 arXiv
-
[15]
A Quantum Computer Trusted Execution Environment,
T. Trochatos, C. Xu, S. Deshpande, Y. Lu, Y. Ding and J. Szefer, "A Quantum Computer Trusted Execution Environment," in IEEE Computer Architecture Letters, vol. 22, no. 2, pp. 177-180, July-Dec. 2023, doi: 10.1109/LCA.2023.3325852
2023
-
[16]
Ghosh, A., Kundu, D., Chatterjee, A., and Ghosh, S. (2024). Guardians of the Quantum GAN. https://arxiv.org/abs/2404.16156
2024 arXiv
-
[17]
Upadhyay, S., & Ghosh, S. (2024). SHARE: Secure Hardware Allocation and Resource Efficiency in Quantum Systems. https://arxiv.org/abs/2405.00863
2024 arXiv
-
[18]
M., & Venter, H
Karie, N. M., & Venter, H. S. (2015). Taxonomy of challenges for digital forensics. Journal of Forensic Sciences, 60(4), 885–893
2015
-
[19]
Sayakkara, A., Le-Khac, N.-A., & Scanlon, M. (2019). A survey of electromagnetic side-channel attacks and discussion on their case-progressing potential for digital forensics. Digital Investigation, 29, 43–54
2019
-
[20]
Rekhis, S., & Boudriga, N. (2011). Logic-based approach for digital forensic investigation in communication networks. Computers & Security, 30(6–7), 376–396
2011
-
[21]
Thongkamwitoon, T. (2014). Digital forensic techniques for the reverse engineering of image acquisition chains. Imperial College London
2014
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.