REVIEW 3 major objections 6 minor 32 references
GPU zero-setup simulators beat CPU by up to 1,400× on 127-qubit Pauli-path simulation and give sub-quadratic MPS scaling, with clear crossover rules for when CPU is still better.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · grok-4.5
2026-07-14 14:50 UTC pith:ZTQ442WC
load-bearing objection Solid empirical systems paper with public code and useful regime maps; the 1,400× / “only GPU” claim is true under the stated hardware but is load-bearing on asymmetric CPU provisioning the authors themselves flag. the 3 major comments →
Benchmarking Zero-Setup Quantum Circuit Simulators
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
On shared circuit families, BlueQubit’s GPU backends are the fastest zero-setup option once problem size amortizes launch overhead: MPS scales sub-quadratically with bond dimension (T ∝ χ^1.49) versus near-quadratic CPU scaling, producing a growing speedup S(χ) ∝ χ^0.55; on the 127-qubit kicked-Ising PPS task they alone reach δ below 10^−5 and achieve up to ~1,400× speedup at δ = 2.5×10^−5, traversing the non-monotonic accuracy peak that coarser CPU runs never leave.
What carries the argument
A standardized cross-platform protocol that sweeps bond dimension χ, qubit count, depth, and PPS truncation δ on identical circuits, decomposes MPS into build versus sampling phases, and reports both wall-clock scaling exponents and accuracy against a tensor-network reference, exposing hardware crossovers and the non-monotonic PPS error trajectory that isolated single-backend numbers hide.
Load-bearing premise
The claim that only GPU reaches the fine truncation and high-accuracy regimes rests on the specific hardware and software limits of the CPU baselines tested (few cores, 16 GB laptop memory, and a platform API floor on δ), which the paper itself notes are not fundamental limits of CPU Pauli-path simulation.
What would settle it
Re-run the same 127-qubit kicked-Ising PPS sweep at δ ≤ 10^−5 on a large-memory multi-core CPU cluster with an unconstrained truncation parameter; if wall-clock times and accuracy match or beat the reported GPU numbers, the “only GPU reaches fine δ” claim fails under realistic CPU provisioning.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript presents a systematic, reproducible benchmarking study of zero-setup quantum circuit simulators, focusing on GPU-accelerated approximate methods (MPS and PPS) and comparing BlueQubit (CPU/GPU) against AWS Braket SV1, Quantum Rings, PPS-Qiskit, and PauliPropagation.jl under shared circuit families and protocols. For MPS (BlueQubit CPU vs GPU on the same Quimb stack), the authors report sub-quadratic bond-dimension scaling on GPU (T∝χ^1.49±0.06) versus near-quadratic CPU scaling, with a growing speedup S(χ)∝χ^0.55, plus a QFT control showing GPU slowdown at low entanglement. For PPS on IBM’s 127-qubit kicked Ising circuit, they report up to ~1,400× GPU speedup at δ=2.5×10^-5 and that only the GPU backend reaches the fine-δ accuracy-recovery regime below 10^-5. A public GitHub repository with code and configurations is provided.
Significance. If the reported measurements and fits hold under the stated configurations, the paper fills a real documentation gap: practitioners lack controlled, end-to-end comparisons of hosted and self-contained approximate simulators across χ, n, d, and δ. Strengths include shared circuit definitions, N=5 means, power-law fits with standard errors, build/sampling phase decomposition, cross-implementation agreement of PPS estimates where ranges overlap, explicit COI disclosure, and a public benchmarking repository with pinned dependency manifests. The QFT low-entanglement control and the non-monotonic PPS error trajectory (with external O_exact from Begušić/Gray/Chan and Tindall et al.) are particularly useful for backend selection and for interpreting accuracy vs truncation. The work is primarily empirical and comparative rather than theoretically novel, but the standardized protocol and released artifacts are reusable contributions to the simulation-benchmarking literature.
major comments (3)
- [Abstract / Section VI-A / VII-E] Abstract, §VI-A, and Figs. 10–12: The load-bearing claim that GPUs deliver ~1,400× speedup and are “the only backends that reach accuracy regimes below δ=10^-5” is true only under the evaluated provisioning. BlueQubit PPS-CPU is limited by the small1 2-core configuration and an API that rejects δ<10^-5 (software cap, not OOM); PPS-Qiskit and PauliPropagation.jl hit a 16 GB laptop memory ceiling (§III, §VI-A). The paper correctly notes these are not fundamental to CPU PPS (§VII-E), yet the abstract and strongest results still use absolute “only” language. Please rephrase the abstract, key-results paragraph, and §VI conclusions so that “only among the commodity/self-contained configurations tested” (or equivalent) is adjacent to the claim, and so that the practical implication about unlocking the non-monotonic recovery regime is explicitly conditional on those caps.
- [Abstract / Section III-B / Section V] §III-B and §V: The MPS section is a same-implementation CPU-vs-GPU infrastructure comparison (both Quimb-based), not a multi-simulator comparison; no other evaluated zero-setup backend exposes configurable MPS. The abstract and introduction currently list AWS Braket and Quantum Rings alongside BlueQubit before stating MPS results, which can be read as implying a broader MPS cross-platform study. Please make the comparison type (infrastructure vs end-to-end platform) explicit in the abstract and at the start of §V, consistent with the careful wording already present in §III-B and §VII-A.
- [Section V-A / VII-F] §V-A, Eqs. (3)–(5) and Fig. 3: The sub-quadratic GPU exponent and the S(χ)∝χ^0.55 growing-advantage claim are fitted over χ∈[32,1536], a range the authors note has not yet entered the SVD-dominated O(χ^3) regime. The extrapolation to χ=5000 (11.7 h vs 119.2 h) is labeled illustrative, but it is still used to support implications for utility-scale bond dimensions in §VII-F. Please either (i) restrict quantitative extrapolation claims to the measured range and treat χ=5000 only as a qualitative illustration, or (ii) add a short sensitivity discussion of how the projected speedup would change if the effective exponent rises toward 2–3 beyond χ~1536.
minor comments (6)
- [Section III-A / Figures 1–2] §III-A: The small1 o small2 transition at 29 qubits is explained, but Figures 1–2 would benefit from an explicit annotation or caption note at that point so readers do not misread the dip as a simulator artifact.
- [Section VI-B / Figure 11] §VI-B / Fig. 11: State the numerical value and provenance of O_exact=0.2955 in the figure caption as well as the main text, so the error panel is self-contained.
- [Section VII-D] §VII-D: Cost is flagged as omitted; a brief order-of-magnitude cost note for the discontinued Braket SV1 runs (or a pointer that relative cost tracks runtime under time-based billing) would help readers without requiring a full multi-vendor pricing study.
- [Throughout] Typographical consistency: “A WS Braket” appears with a space in several places (e.g., abstract-adjacent text and §III-B); standardize to “AWS Braket.”
- [Index Terms / Introduction] Index terms and introduction mention “performance modeling”; the body mainly reports empirical power-law fits. Consider either a short explicit performance model subsection or softening that term to “scaling characterization.”
- [Footnotes / Conclusion] Repository URL is given twice with slightly different emphasis (footnotes 1–2); a single canonical citation plus a short reproducibility checklist (how to regenerate each figure) would improve reuse.
Circularity Check
Empirical benchmarking paper: wall-clock and accuracy results are measured against external references, not derived circularly from fitted inputs or self-citation chains.
full rationale
This is a cross-platform performance study, not a first-principles derivation. Central claims (state-vector runtimes, MPS T∝χ^α exponents, PPS speedups up to ~1,400×, and non-monotonic accuracy vs δ) are empirical wall-clock and error measurements on fixed circuit families (Qiskit QV/QFT, IBM kicked Ising, peaked circuits). Accuracy is scored against external tensor-network references O_exact = 0.2955 from Begušić/Gray/Chan and Tindall et al., not against quantities fitted from the same runs. Power-law fits (e.g., T_CPU ∝ χ^{2.03±0.11}, T_GPU ∝ χ^{1.49±0.06}) characterize measured data and are explicitly caveated when extrapolated; they are not re-labeled as independent predictions. Self-citations (Gharibyan et al. [29] with overlapping author Mullath on non-monotonic PPS convergence; [25] for the marginal-attack rule) supply interpretive context or a reconstruction heuristic and do not force the speedup or ranking results. Conflict of interest (BlueQubit affiliation) is disclosed and is not circularity under the stated criteria. No step reduces a claimed prediction to its inputs by construction. Score 0 is the honest finding.
Axiom & Free-Parameter Ledger
free parameters (6)
- GPU MPS scaling exponent α_G =
1.49 ± 0.06
- CPU MPS scaling exponent α_C =
2.03 ± 0.11
- BlueQubit CPU core provisioning (small1/small2) =
2 or 20 cores
- Local SDK host memory ceiling =
16 GB
- PPS truncation threshold sweep δ =
δ ∈ [2.89×10^-6, 10^-2]
- O_exact reference for ⟨Z_62⟩ =
0.2955
axioms (6)
- domain assumption Standard MPS gate cost model: contraction O(χ²) and SVD O(χ³) per two-qubit gate, with expected large-χ dominance of SVD.
- domain assumption PPS term updates are independent and map to SIMT parallelism, so speedup should grow with retained Pauli count N_P.
- domain assumption Wall-clock / API-reported runtime with N=5 means is a valid primary performance metric for zero-setup backends.
- domain assumption Non-monotonic PPS convergence framework of Gharibyan et al. applies to the kicked Ising instance studied.
- ad hoc to paper Power-law fits T ∝ χ^α over χ∈[32,1536] are meaningful for extrapolation and backend selection.
- standard math Standard linear algebra / floating-point arithmetic and published library primitives (Quimb, cuQuantum, qsim) behave as documented.
read the original abstract
Practitioners increasingly rely on hosted simulation environments, but their performance characteristics remain poorly documented. We present a systematic benchmarking study of GPU-accelerated approximate quantum simulation across two widely used methods: matrix product states (MPS) and Pauli path simulation (PPS), comparing BlueQubit (a hosted tool that handles hardware provisioning, simulator configuration, and job orchestration) against AWS Braket, Quantum Rings, PPS-Qiskit, and PauliPropagation.jl. For MPS, we find that GPU runtime yields sub-quadratic scaling with bond dimension, with a growing advantage over CPU at increasing scale. For Pauli path simulation on IBM's 127-qubit kicked Ising benchmark, GPUs deliver up to $1{,}400\times$ speedup at fine truncation thresholds ($\delta = 2.5 \times 10^{-5}$, 27.6M Pauli terms), and are the only backends that reach accuracy regimes below $\delta = 10^{-5}$, which remained inaccessible to the commodity CPU-based implementations and self-contained SDKs evaluated here. We also provide a reproducible characterization of these simulators across regimes, including tradeoffs that isolated evaluations do not show. To support transparency and reuse, we provide a public GitHub repository containing all benchmarking code and configurations.
Figures
Reference graph
Works this paper leans on
-
[1]
Quantum computing in the NISQ era and beyond,
J. Preskill, “Quantum computing in the NISQ era and beyond,”Quan- tum, vol. 2, p. 79, 2018
2018
-
[2]
Amazon braket developer guide,
Amazon Web Services, “Amazon braket developer guide,” https://docs. aws.amazon.com/braket/, 2024, accessed: 2026-04-04
2024
-
[3]
Quantum rings sdk documentation,
Quantum Rings, “Quantum rings sdk documentation,” https:// quantumrings.com/docs, 2024, accessed: 2026-04-04
2024
-
[4]
pauli-prop: A Rust-accelerated Python package for Pauli propagation,
Qiskit Community, “pauli-prop: A Rust-accelerated Python package for Pauli propagation,” https://github.com/Qiskit/pauli-prop, 2025, accessed: 2026-04-04
2025
-
[5]
PauliPropagation.jl: A Julia library for Pauli propaga- tion simulation of quantum circuits,
M. S. Rudolph, T. Jones, Y . Teng, S. Y . Chang, A. Angrisani, and Z. Holmes, “PauliPropagation.jl: A Julia library for Pauli propaga- tion simulation of quantum circuits,” https://github.com/MSRudolph/ PauliPropagation.jl, 2025, accessed: 2026-04-04. Companion paper: arXiv:2505.21606
Pith/arXiv arXiv 2025
-
[6]
Benchmarking simulated and physical quantum processing units using quantum and hybrid algorithms,
M. Kordzanganeh, M. Buchberger, B. Kyriacou, M. Povolotskii, W. Fis- cher, A. Kurkin, W. Somogyi, A. Sagingalieva, M. Pflitsch, and A. Mel- nikov, “Benchmarking simulated and physical quantum processing units using quantum and hybrid algorithms,”Advanced Quantum Technolo- gies, vol. 6, no. 8, p. 2300043, 2023
2023
-
[7]
Intel Quantum Simulator: A cloud-ready high-performance simulator of quantum circuits,
G. G. Guerreschi, J. Hogaboam, F. Baruffa, and N. P. D. Sawaya, “Intel Quantum Simulator: A cloud-ready high-performance simulator of quantum circuits,”Quantum Science and Technology, vol. 5, p. 034007, 2020
2020
-
[8]
Efficient classical simulation of slightly entangled quantum computations,
G. Vidal, “Efficient classical simulation of slightly entangled quantum computations,”Physical Review Letters, vol. 91, no. 14, p. 147902, 2003
2003
-
[9]
Harnessing CUDA-Q’s MPS for tensor network simulations of large-scale quantum circuits,
G. Schieffer, S. Markidis, and I. Peng, “Harnessing CUDA-Q’s MPS for tensor network simulations of large-scale quantum circuits,” 2025. [Online]. Available: https://arxiv.org/abs/2501.15939
Pith/arXiv arXiv 2025
-
[10]
Fast classical simulation of evidence for the utility of quantum computing before fault tolerance,
T. Begu ˇsi´c and G. K.-L. Chan, “Fast classical simulation of evidence for the utility of quantum computing before fault tolerance,”arXiv preprint arXiv:2306.16372, 2023
Pith/arXiv arXiv 2023
-
[11]
A polynomial-time classical algorithm for noisy random circuit sampling,
D. Aharonov, X. Gao, Z. Landau, Y . Liu, and U. Vazirani, “A polynomial-time classical algorithm for noisy random circuit sampling,” inProceedings of the 55th ACM Symposium on Theory of Computing (STOC), 2023, pp. 945–957
2023
-
[12]
Advanced large-scale quantum simulation techniques in cuQuantum SDK v25.11,
T. Lubowe, B. Kloss, D. Lykov, T. Jones, and D. Lowell, “Advanced large-scale quantum simulation techniques in cuQuantum SDK v25.11,” https://developer.nvidia.com/blog/ advanced-large-scale-quantum-simulation-techniques-in-cuquantum-sdk-v25-11/, Dec. 2025, nVIDIA Technical Blog. 3github.com/arulrhikm/mps-pps-zero-setup-benchmarks
2025
-
[13]
A. Chatterjee, S. Rappaport, A. Giri, S. Johri, T. Proctor, D. E. Bernal Neira, P. Sathe, and T. Lubinski, “A comprehensive cross-model framework for benchmarking the performance of quantum Hamilto- nian simulations,”IEEE Transactions on Quantum Engineering, 2025, preprint: arXiv:2409.06919
Pith/arXiv arXiv 2025
-
[14]
Empowering large scale quantum circuit development: Effective simulation of Sycamore circuits,
V . Kasirajan, T. Battelle, and B. Wold, “Empowering large scale quantum circuit development: Effective simulation of Sycamore circuits,”arXiv preprint arXiv:2411.12131, 2024
Pith/arXiv arXiv 2024
-
[15]
cuQuantum SDK: A high-performance library for accelerating quantum science,
H. Bayraktar, A. Charara, D. Clark, S. Cohen, T. Costa, Y .-L. L. Fang, Y . Gao, J. Guan, J. Gunnels, A. Haidar, A. Hehn, M. Hohnerbach, M. Jones, T. Lubowe, D. Lyakh, S. Morino, P. Springer, S. Stanwyck, I. Terentyev, S. Varadhan, J. Wong, and T. Yamaguchi, “cuQuantum SDK: A high-performance library for accelerating quantum science,” in2023 IEEE Internat...
2023
-
[16]
Implementation of tensor network simulation TN-Sim under NWQ-Sim,
A. C. Hoyt, J. S. Bersson, S. Garner, C. Liu, and A. Li, “Implementation of tensor network simulation TN-Sim under NWQ-Sim,”arXiv preprint arXiv:2601.04422, 2026, includes runtime breakdown separating tensor contraction and SVD at fixed bond dimension. Conference version: 2025 IEEE High Performance Extreme Computing Conference (HPEC), DOI: 10.1109/HPEC676...
-
[17]
quimb: A python package for quantum information and many- body calculations,
J. Gray, “quimb: A python package for quantum information and many- body calculations,”Journal of Open Source Software, vol. 3, no. 29, p. 819, 2018. [Online]. Available: https://doi.org/10.21105/joss.00819
-
[18]
qsim: A fast quantum circuit simulator,
Quantum AI Team, “qsim: A fast quantum circuit simulator,” Google,
-
[19]
Available: https://github.com/quantumlib/qsim
[Online]. Available: https://github.com/quantumlib/qsim
-
[20]
Qiskit: An open-source framework for quantum com- puting,
IBM Quantum, “Qiskit: An open-source framework for quantum com- puting,” https://qiskit.org, 2023, accessed: 2026-04-04
2023
-
[21]
Amazon braket Hybrid Jobs,
Amazon Web Services, “Amazon braket Hybrid Jobs,” https://docs.aws. amazon.com/braket/latest/developerguide/braket-jobs.html, 2024, ac- cessed: 2026-07-07. Supports user-provided container images and GPU- backed instances for hybrid quantum-classical workloads
2024
-
[22]
Validating quantum computers using randomized model circuits,
A. W. Cross, L. S. Bishop, S. Sheldon, P. D. Nation, and J. M. Gambetta, “Validating quantum computers using randomized model circuits,”Physical Review A, vol. 100, no. 3, p. 032328, 2019
2019
-
[23]
Quantum computing with Qiskit,
A. Javadi-Abhari, M. Treinish, K. Krsulich, C. J. Wood, J. Lishman, J. Gacon, S. Martiel, P. D. Nation, L. S. Bishop, A. W. Cross, B. R. Johnson, and J. M. Gambetta, “Quantum computing with Qiskit,” 2024
2024
-
[24]
Evidence for the utility of quantum computing before fault tolerance,
Y . Kim, A. Eddins, S. Anand, K. X. Wei, E. van den Berg, S. Rosenblatt, H. Nayfeh, Y . Wu, M. Zaletel, K. Temme, and A. Kandala, “Evidence for the utility of quantum computing before fault tolerance,”Nature, vol. 618, pp. 500–505, 2023
2023
-
[25]
On verifiable quantum advantage with peaked circuit sampling,
S. Aaronson and Y . Zhang, “On verifiable quantum advantage with peaked circuit sampling,” 2024. [Online]. Available: https: //arxiv.org/abs/2404.14493
Pith/arXiv arXiv 2024
-
[26]
Heuristic quantum advantage with peaked circuits,
H. Gharibyan, M. Z. Mullath, N. E. Sherman, V . P. Su, H. Tepanyan, and Y . Zhang, “Heuristic quantum advantage with peaked circuits,”
-
[27]
Available: https://arxiv.org/abs/2510.25838
[Online]. Available: https://arxiv.org/abs/2510.25838
-
[28]
The density-matrix renormalization group in the age of matrix product states,
U. Schollw ¨ock, “The density-matrix renormalization group in the age of matrix product states,”Annals of Physics, vol. 326, no. 1, pp. 96–192, 2011
2011
-
[29]
T. Begu ˇsi´c, J. Gray, and G. K.-L. Chan, “Fast and converged classical simulations of evidence for the utility of quantum computing before fault tolerance,”Science Advances, vol. 10, no. 3, p. eadk4321, 2024, preprint: arXiv:2308.05077
Pith/arXiv arXiv 2024
-
[30]
Efficient tensor network simulation of IBM’s Eagle kicked Ising experiment,
J. Tindall, M. Fishman, E. M. Stoudenmire, and D. Sels, “Efficient tensor network simulation of IBM’s Eagle kicked Ising experiment,” PRX Quantum, vol. 5, p. 010308, 2024
2024
-
[31]
A practical guide to using Pauli path simulators for utility-scale quantum experiments,
H. Gharibyan, S. Hariprakash, M. Z. Mullath, and V . P. Su, “A practical guide to using Pauli path simulators for utility-scale quantum experiments,” 2025
2025
-
[32]
MQT Bench: Bench- marking software and design automation tools for quantum computing,
N. Quetschlich, L. Burgholzer, and R. Wille, “MQT Bench: Bench- marking software and design automation tools for quantum computing,” Quantum, vol. 7, p. 1062, 2023
2023
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.