REVIEW 4 major objections 6 minor 1 cited by
Empowering Large Scale Quantum Circuit Development: Effective Simulation of Sycamore Circuits
T0 review · 4 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read The paper claims the Quantum Rings SDK can classically simulate 53-qubit Sycamore random circuit sampling circuits on 32 GB hardware, reaching an average linear XEB score of 0.678 that the authors call the highest observed for these…
desk verdict Benchmark of a proprietary simulator; headline XEB is not independently grounded for the largest circuits. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the linear cross-entropy benchmark, defined in the paper as $\mathcal{F}_{\mathrm{XEB}} = N \left(\frac{1}{k} \sum_{i=1}^{k} p(x_i)\right) - 1$, which measures how strongly the sampled bitstring probabilities correlate with the ideal circuit probabilities; it approaches 1 for noiseless sampling and 0 for uniform sampling. The authors use this score as a proxy for fidelity, following the standard relation $\mathcal{F}_{\mathrm{XEB}} \approx (1-\epsilon)^{\#\text{gates}}$. The simulator converts the published QASM circuit descriptions into amplitudes, samples bitstrings, and the samples are checked against the Porter-Thomas distribution as a secondary validation.
What would settle it
Take the $n=51$ and $n=53$ circuits, compute reference amplitudes with an independent exact or high-precision approximate method such as tensor-network contraction, then recompute the linear XEB of the SDK samples against that reference; if the resulting scores fall well below 0.678, the paper's central fidelity claim fails, while if they remain high, the self-comparison concern is resolved.
Extended reading notes
Core claim
On the paper's own terms, the discovery is that a universal quantum simulator can reproduce the output distribution of the 53-qubit Sycamore random circuit sampling circuits with high fidelity under memory constraints typical of a developer machine. The authors compute the linear XEB score of equation (2) for circuits of $n=12$ through $n=53$ qubits, using the original experiment's published amplitude files as the ideal reference where available and the SDK's own amplitudes for $n=51$ and $n=53$. They report an average $\mathcal{F}_{\mathrm{XEB}} = 0.678$, a value of $0.622$ for the largest circuit at 2.5 million samples, and execution times they describe as reasonable. They conclude that this is the highest fidelity observed to date for these circuits.
Load-bearing premise
The unstated load-bearing premise is that for the 51- and 53-qubit circuits, where the published ideal amplitude files are missing, the simulator's own amplitudes can stand in as the 'ideal' reference for the XEB calculation; if that premise is false, the average score and the 'highest fidelity' claim weaken because the average includes those largest circuits.
Editorial extensions
If this is right
- Simulating 53-qubit random circuit sampling circuits on 32 GB hardware means large-scale circuit debugging no longer requires a supercomputer or specialized hardware access.
- Cross-entropy benchmarking can serve as a routine validation metric for classical simulators, not just for physical quantum processors.
- The reported average score provides a new classical baseline for the 14-cycle Sycamore circuits that future simulation claims would need to beat.
- If the execution times hold, developers can test commercial quantum algorithms at near-ideal fidelity before fault-tolerant quantum hardware exists.
Reading between the lines
- The reported average depends partly on self-referential XEB for the 51- and 53-qubit circuits, where the simulator's own amplitudes serve as the ideal reference; an independent amplitude reference could move the headline 0.678, so the 'highest fidelity' claim should be read with that caveat.
- The paper's own citation of work on limitations of linear cross-entropy suggests a testable extension: deliberately inject depolarizing noise into the simulator and check whether $\mathcal{F}_{\mathrm{XEB}}$ decays as predicted by the error model.
- The same protocol could be pushed to the 60-qubit, 24-cycle random circuit sampling circuits from a later experiment; the scalability limits of the simulator are not tested by the 53-qubit dataset alone.
- A direct comparison against an independent tensor-network-based reference computation for the largest circuits would convert the self-comparison into an external benchmark.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper reports classical simulation of Google's random-circuit-sampling circuits from the Dryad dataset using the Quantum Rings SDK on machines with only 32 GB of memory. The authors sample 500,000 bitstrings for most circuits and 2,500,000 bitstrings for n=51 and n=53, compute linear cross-entropy benchmarking (XEB) using Eq. (2), and report an average F_XEB of 0.678, which they interpret as strong correlation with ideal quantum simulation and as the highest fidelity observed to date. They also report time-to-first-sample for the largest n=53 circuits.
Significance. If valid, the central claim would be a notable practical demonstration that large random-circuit-sampling circuits can be classically simulated on modest hardware, with useful implications for quantum circuit development and debugging. The paper has genuine strengths: it uses Google's published QASM circuits, makes source code and amplitude files publicly available on GitHub, checks the sample distribution against the Porter-Thomas prediction, and for circuits up to n=50 uses Google's own amplitude files as an external XEB reference. However, the headline average is compromised by the treatment of the largest circuits, and the comparison with prior work is not quantified.
major comments (4)
- [Section III, Figure 4, Eq. (2)] For n=51 and n=53 the reported XEB values are not independently grounded. Equation (2) defines F_XEB relative to an ideal distribution p(x_i), but Figure 4 explicitly marks n=51/53 as the range for which Google's amplitudes are unavailable, while the text reports F_XEB=0.622 at n=53 and includes these circuits in the average 0.678. The only available source of "ideal" amplitudes in this range is the Quantum Rings SDK itself, so both the sampled bitstrings and the reference distribution are outputs of the same approximate simulator; this measures self-consistency, not fidelity to the ideal quantum state. The fact that the reported n=53 value is 0.622 rather than approximately 1 shows that the reference and sampling distributions are not even properly matched, further confirming that the metric is not interpretable. The manuscript nowhere states what reference was used for n=51 and n=53, nor does it provide an independent validation for those circuits.
- [Section III vs. Figure 4] The text and the figure caption contradict each other about whether Google's amplitude files exist for n=51 and n=53. Section III says the 2,500,000-sample count for these circuits was chosen "to ensure that the number of samples matched those in Google's amplitude files," which implies such files exist, while the Figure 4 caption says the dashed segment is the qubits range for which Google's amplitudes are unavailable. This distinction is load-bearing: if the files exist, the XEB should have been computed with that external reference and the figure should show the corresponding blue points; if they do not, the sentence about matching the sample count is unexplained and the circularity concern applies.
- [Section V (Conclusion) and Abstract] The claim that the average XEB of 0.678 "exceeds the XEB values currently reported for the same circuits today" and "represents the highest fidelity observed to date" is unsupported. No table, equation, or citation provides the prior XEB values for the same circuits from references [4-7,9,10,12,15]. Moreover, XEB for a classical simulator is not directly comparable to hardware XEB: a noiseless simulator would yield F_XEB=1 by definition, whereas hardware XEB is suppressed by gate errors. The authors should either provide a quantitative comparison with prior classical simulation results for these exact circuits or remove the superlative claim.
- [Section IV] The manuscript gives no description of the simulation algorithm or its approximation error. It says only that the SDK "output[s] the amplitudes of each measurement" and that circuits were executed in a Python environment. Exact statevector simulation of 53 qubits would require roughly 64 PB of memory, far beyond the 32 GB used, so the simulator must be approximate. The reported average F_XEB of 0.678, which is well below the F_XEB=1 expected from an exact simulator, further indicates a substantial deviation from the ideal distribution. Without knowing whether the method is a tensor-network, low-rank, or heuristic approximation, and without error bounds, the XEB result cannot be interpreted as evidence of "effective simulation," and the experiment cannot be independently reproduced from the text.
minor comments (6)
- [Sections III and IV] The Dryad dataset is dated "June 13, 2022" in Section III and "June 23, 2022" in Section IV for the same reference [7]; please reconcile the dates.
- [Section II] The name "Artu et al." should be "Arute et al."; additionally, the citation given as [5] is Boixo et al., so the attribution of the error-gate XEB argument should be checked.
- [Figures 2 and 3] The captions of Figures 2 and 3 do not identify which particular circuit (n, m, pattern, seed) produced the plotted distribution, so the Porter-Thomas validation is not reproducible.
- [Figure 5] Figure 5 reports time-to-first-sample for p='ABCDCDAB' and m=12-20, while the XEB study uses p='EFGH' and m=14; the relationship between the timing experiment and the XEB experiment should be stated explicitly.
- [Section IV] The phrase "the SDK was used to output the amplitudes of each measurement" is ambiguous about whether the simulator produces the full 2^n amplitude vector or only probabilities for the sampled bitstrings; please clarify.
- [Eq. (2)] In Eq. (2), p(x_i) is never explicitly defined as the ideal probability |<x_i|psi_U>|^2, which is particularly important here because the choice of reference distribution is the central methodological issue of the paper.
Circularity Check
Headline XEB of 0.678 is partly self-referential: for n=51 and n=53, Google amplitudes are unavailable and the SDK's own amplitudes serve as the 'ideal' reference.
-
self definitional
[Section III (The Experiment), Figure 4 caption, and Eq. (2) in Section II]
"Google also provided the amplitudes for most of these circuits, which we used to calculate the 𝐹𝑋𝐸𝐵 for comparison. Except for the circuits with 𝑛 = {51, 53}, we sampled the remaining circuits 500,000 times each. // Figure 4 caption: "The dashed segment is the qubits range for which Google’s amplitudes are unavailable. The red colored line shows the values calculated using the Quantum Rings SDK.""
Equation (2) defines F_XEB = N(1/k sum p(x_i)) - 1, where p(x_i) is the ideal output probability of the sampled bitstring. For n=51 and n=53, Figure 4 states Google amplitudes are unavailable, and the red F_XEB line is 'calculated using the Quantum Rings SDK.' Since the same SDK generated both the sampled bitstrings and the reference amplitudes, p(x_i) in Eq. (2) is the SDK's own approximate distribution, not an independent ideal. The resulting XEB measures self-consistency (roughly the purity of the SDK's output distribution), not fidelity to the exact quantum state.
full rationale
Most of the paper's derivation chain is not circular: for circuits with n ≤ 50, the XEB is computed against Google's provided amplitude files, which is an independent external reference, and the Porter-Thomas distribution comparison (Figure 3) is an external statistical check. The SDK is not fit to the data, and no parameter estimation is used to produce the reported XEB values. However, the central headline result — the average XEB of 0.678 and the claim of 'highest fidelity observed to date' — depends on the n=51 and n=53 data points. For those circuits, the paper's Figure 4 caption explicitly states that Google's amplitudes are unavailable, and the red line is 'calculated using the Quantum Rings SDK.' Because Eq. (2) defines XEB relative to an ideal distribution, substituting the SDK's own amplitudes as the 'ideal' reference turns those two points into a self-comparison. The paper does not provide any alternative independent verification for these largest circuits. Therefore the central claim is partially circular: the independently grounded portion (n ≤ 50) is solid, but the headline average is materially contaminated by self-referential points. This does not mean the entire study is invalid; the smaller-circuit results stand on their own. The circularity score is 6 because the central claim reduces, for its largest and most impressive circuits, to the simulator comparing itself to itself.
Assumptions & free parameters
assumptions (4)
- domain assumption RCS output probabilities follow a Porter-Thomas distribution (Eq. 1)
- domain assumption Linear XEB approximates the fidelity of the sampled distribution (Eq. 4)
- domain assumption Google's amplitude files provide the ideal output probabilities for circuits with n ≤ 50
- ad hoc to paper For n=51 and n=53, the Quantum Rings SDK's own amplitudes can serve as the ideal reference for XEB
Cite this review
Pith. "Pith review of Empowering Large Scale Quantum Circuit Development: Effective Simulation of Sycamore Circuits." pith.science (2026). https://pith.science/paper/4HKSKRF7
@misc{pith2026241112131,
author = {Pith},
title = {Pith review of: Empowering Large Scale Quantum Circuit Development: Effective Simulation of Sycamore Circuits},
year = {2026},
howpublished = {\url{https://pith.science/paper/4HKSKRF7}},
note = {Machine review of arXiv:2411.12131}
}
read the original abstract
Simulating quantum systems using classical computing equipment has been a significant research focus. This work demonstrates that circuits as large and complex as the random circuit sampling (RCS) circuits published as a part of Google's pioneering work [4-7] claiming quantum supremacy can be effectively simulated with high fidelity on classical systems commonly available to developers, using the universal quantum simulator included in the Quantum Rings SDK, making this advancement accessible to everyone. This study achieved an average linear cross-entropy benchmarking (XEB) score of 0.678, indicating a strong correlation with ideal quantum simulation and exceeding the XEB values currently reported for the same circuits today while completing circuit execution in a reasonable timeframe. This capability empowers researchers and developers to build, debug, and execute large-scale quantum circuits ahead of the general availability of low-error rate quantum computers and invent new quantum algorithms or deploy commercial-grade applications.
Figures
Forward citations
Cited by 1 Pith paper
-
Benchmarking Zero-Setup Quantum Circuit Simulators
GPU-accelerated zero-setup simulators show sub-quadratic MPS bond-dimension scaling and up to 1,400× PPS speedups, uniquely reaching fine truncation accuracy on the 127-qubit kicked Ising circuit.
Reference graph
Works this paper leans on
-
[1]
Complexity-Theoretic Foundations of Quantum Supremacy Experiments
Aaronson, Scott, and Lijie Chen. 2016. “Complexity-Theoretic Foundations of Quantum Supremacy Experiments.” ArXiv:1612.05903 [Quant-Ph]. https://arxiv.org/abs/1612.05903
arXiv 2016
-
[2]
Sampling Problems on a Quantum Computer
Mansky, Maximilian Balthasar, Jonas Nüßlein, David Bucher, Daniëlle Schuman, Sebastian Zielinski, and Claudia Linnhoff - Popien. 2023. “Sampling Problems on a Quantum Computer.” ArXiv.org. September 17, 2023. https://doi.org/10.1109/QCE57702.2023.00062
arXiv 2023
-
[3]
Sampling Random Quantum Circuits: A Pedestrian’s Guide
Mullane, Sean. n.d. “Sampling Random Quantum Circuits: A Pedestrian’s Guide.” Accessed July 8, 2024. https://arxiv.org/pdf/2007.07872
arXiv 2024
-
[4]
Quantum Supremacy Using a Programmable Superconducting Processor
Arute, Frank, Kunal Arya, Ryan Babbush, Dave Bacon, Joseph C. Bardin, Rami Barends, Rupak Biswas, et al. 2019. “Quantum Supremacy Using a Programmable Superconducting Processor.” Nature 574 (7779): 505 –10. https://doi.org/10.1038/s41586-019-1666-5
-
[5]
Characterizing Quantum Supremacy in Near-Term Devices
Boixo, Sergio, Sergei V . Isakov, Vadim N. Smelyanskiy, Ryan Babbush, Nan Ding, Zhang Jiang, Michael J. Bremner, John M. Martinis, and Hartmut Neven. 2018. “Characterizing Quantum Supremacy in Near-Term Devices.” Nature Physics 14 (6): 595–600. https://doi.org/10.1038/s41567-018-0124-x. P a g e | 9
-
[6]
Supplementary Information for ‘Quantum Supremacy Using a Programmable Superconducting Processor.’
Arute, Frank, Kunal Arya, Ryan Babbush, Dave Bacon, Joseph C. Bardin, Rami Barends, Rupak Biswas, et al. 2019. “Supplementary Information for ‘Quantum Supremacy Using a Programmable Superconducting Processor.’” ArXiv.org. October 23, 2019. https://arxiv.org/abs/1910.11333
arXiv 2019
-
[7]
Martinis, John M. et al. (2022). Quantum supremacy using a programmable superconducting processor [Dataset]. Dryad. https://doi.org/10.5061/dryad.k6t1rj8
-
[8]
A Blueprint for Demonstrating Quantum Supremacy with Superconducting Qubits
Neill, C., P. Roushan, K. Kechedzhi, S. Boixo, S. V . Isakov, V . Smelyanskiy, A. Megrant, et al. 2018. “A Blueprint for Demonstrating Quantum Supremacy with Superconducting Qubits.” Science 360 (6385): 195–99. https://doi.org/10.1126/science.aao4309
Show all 21 references
- [9]
-
[10]
Simulating the Sycamore Quantum Supremacy Circuits
Pan, Feng, and Pan Zhang. 2021. “Simulating the Sycamore Quantum Supremacy Circuits.” ArXiv:2103.03074 [Physics, Physics:quant-Ph], March. https://arxiv.org/abs/2103.03074
2021 arXiv
-
[11]
Closing the ‘Quantum Supremacy’ Gap
Liu, Yong (Alexander), Xin (Lucy) Liu, Fang (Nancy) Li, Haohuan Fu, Yuling Yang, Jiawei Song, Pengpeng Zhao, et al. 2021. “Closing the ‘Quantum Supremacy’ Gap.” Proceedings of the International Conference for High Performance Computing, Networking, St orage and Analysis, Novem...
2021
-
[12]
The Computational Power of Random Quantum Circuits in Arbitrary Geometries
DeCross, Matthew, Reza Haghshenas, Minzhao Liu, Enrico Rinaldi, Johnnie Gray, Y uri Alexeev, Charles H. Baldwin, et al. 2024. “The Computational Power of Random Quantum Circuits in Arbitrary Geometries.” ArXiv.org. June 21, 2024. https://doi.org/10.48550/arXiv.2406.02501
-
[13]
Kalai, Gil, Yosef Rinott, and Tomer Shoham
-
[14]
On the Complexity and Verification of Quantum Random Circuit Sampling
Bouland, Adam, Bill Fefferman, Chinmay Nirkhe, and Umesh Vazirani. 2018. “On the Complexity and Verification of Quantum Random Circuit Sampling.” Nature Physics 15 (2): 159–63. https://doi.org/10.1038/s41567-018-0318-2
2018 doi
-
[15]
Quantum Computational Advantage via 60-Qubit 24-Cycle Random Circuit Sampling
Zhu, Qing -Ling, Sirui Cao, Fusheng Chen, Ming-Cheng Chen, X H Chen, Tung -Hsun Chung, Hui Deng, et al. 2021. “Quantum Computational Advantage via 60-Qubit 24-Cycle Random Circuit Sampling.” ArXiv (Cornell University), September. https://doi.org/10.48550/arxiv.2109.03494
-
[16]
Limitations of Linear Cross-Entropy as a Measure for Quantum Advantage
Gao, Xun, Marcin Kalinowski, Chi -Ning Chou, Mikhail D. Lukin, Boaz Barak, and Soonwon Choi. 2024. “Limitations of Linear Cross-Entropy as a Measure for Quantum Advantage.” PRX Quantum 5 (1): 010334. https://doi.org/10.1103/PRXQuantum.5.010334
2024 doi
-
[17]
The Sol Supercomputer at Arizona State University
Jennewein, Douglas M, Johnathan Lee, Chris Kurtz, William Dizon, Ian Shaeffer, Alan Chapman, Alejandro Chiquete, et al. 2023. “The Sol Supercomputer at Arizona State University.” Practice and Experience in Advanced Research Computing, July. https://doi.org/10.1145/3569951.3597573
2023
-
[18]
Morvan, A., Villalonga, B., Mi, X. et al. Phase transitions in random circuit sampling. Nature 634, 328–333 (2024). https://doi.org/10.1038/s41586-024-07998-6
2024 doi
-
[19]
The amplitudes are available at: https://github.com/Quantum-Rings/quantum- rings-rcs-benchmarking/tree/main/amplitudes P a g e | 10
The source code used in this study is available at the following repository: https://github.com/Quantum-Rings/quantum- rings-rcs-benchmarking. The amplitudes are available at: https://github.com/Quantum-Rings/quantum- rings-rcs-benchmarking/tree/main/amplitudes P a g e | 10
-
[20]
License keys can be obtained by registering at https://www.quantumrings.com
The Quantum Rings SDK can be installed by following the instructions at: https://quantumrings.com/doc/index.html. License keys can be obtained by registering at https://www.quantumrings.com. (free for academic and non-commercial use)
- [2023]
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.