REVIEW 3 major objections 5 minor 1 cited by
A quantum annealer, used as ground truth, shows a tensor-network method's error stays flat where theory predicted a drop.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
Using QA hardware as a proxy ground truth, the authors find BP-TNS correlation error does not decrease from L=3 to L=4 cubic dimer lattices, contrary to the prediction in Ref [10].
T0 review reviewed 2026-08-05 challenge →
load-bearing objection A useful, honest reverse-benchmarking paper that probably kills a specific BP-TNS scaling prediction, but the key L=4 estimate leans on an untested QA-error-size-constancy assumption. the 3 major comments →
Evaluating classical simulations with a quantum processor
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
Core claim
The authors generate an approximate ground truth with a quantum annealing processor and use it to estimate the BP-TNS error relative to a matrix-product-state ground truth via a quadrature subtraction: the squared BP-TNS-versus-QA error minus the squared QA-versus-MPS error, under assumptions of statistical independence between the two error sources and constancy of QA error with system size. Applying this estimator at L=4 (N=128), beyond the reach of practical MPS ground truths, they find no evidence that the median BP-TNS correlation error decreases from L=3 to L=4 over the ensembles tested. For bimodal spin glasses the estimated error continues to rise; for uniform spin glasses it is flat
What carries the argument
The central object is the estimator of Eq. (5), which computes the BP-TNS error relative to an MPS ground truth as the quadrature difference between BP-TNS error relative to a quantum annealer and the annealer's own error relative to MPS. It relies on two assumptions: that QA and BP-TNS two-body correlation errors are statistically independent, and that QA error is roughly constant from L=3 to L=4. This estimator is what allows the paper to push BP-TNS error measurements to system sizes where MPS ground truth would require bond dimension around 3000.
Load-bearing premise
The load-bearing premise is that the quantum processor's own error stays the same from L=3 to L=4, but that constancy was only demonstrated on square lattices, not on the cubic dimer lattices with bimodal couplings used here; if QA error grows with system size, the subtraction understates the tensor-network error and could mask the predicted drop.
What would settle it
Run the same quench on a handful of L=4 cubic dimer instances and compute an MPS ground truth at large enough bond dimension for convergence; if the directly measured BP-TNS error decreases from L=3 to L=4, or if QA error measured this way rises, the paper's conclusion fails.
If this is right
- Loop corrections in BP-TNS should not be assumed to generically improve estimates; they increase error for bimodal couplings and longer quenches.
- Increasing bond dimension in BP-TNS offers diminishing returns in loop-rich lattices, and measurement time appears exponential over the range studied rather than the asymptotic χ^7.
- A quantum processor can act as a ground-truth source for benchmarking classical algorithms in regimes where classical ground truth is impractical.
- The predicted asymptotic regime for BP-TNS error does not govern at N=128, so scaling predictions for these algorithms need to be treated with caution or tested at even larger sizes.
Where Pith is reading between the lines
- If QA error actually grows with system size in cubic dimer lattices, the quadrature subtraction would understate BP-TNS error at L=4 and could mask a genuine decrease; this is testable with high-bond-dimension MPS on a small set of L=4 instances.
- The dimer-perfect-correlation approximation used to expand correlation measurements is valid on average (mean 0.991) but has outliers down to 0.7 in bimodal inputs, which could bias bimodal error estimates.
- The same benchmarking protocol could be applied to other approximate classical methods, such as PEPS variants or neural quantum states, wherever their errors are statistically independent of the processor's.
- The flat uniform-ensemble behavior at ta=20 ns may indicate that the fixed-correlation-length asymptotic regime sets in only at larger N, or that loop removal is offset by other sources of error.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes to use a quantum annealing (QA) processor as an approximate ground truth for evaluating the accuracy of belief-propagation-gauged tensor-network state (BP-TNS) simulations of quench dynamics in three-dimensional disordered Ising models. Since an MPS ground truth is impractical at the target system size (L=4, N=128), the authors introduce an indirect estimator, Eq. (5), that subtracts the QA error in quadrature from the BP-TNS-vs-QA error, relying on statistical independence of the two error sources and constancy of the QA error with system size. The estimator is validated at L=3 (Fig. 3). Applying it at L=4, the authors find that the estimated BP-TNS error does not decrease from L=3 to L=4, and for bimodal inputs it increases, contradicting the asymptotic-scaling prediction of Ref. [10].
Significance. If the result holds, it is a valuable cautionary result for the tensor-network community: asymptotic scaling predictions for BP-TNS in three dimensions may not govern at N=128, and QA hardware can serve as a benchmarking ground truth in regimes where classical ground truth is inaccessible. The paper's strengths include the empirical validation of the indirect estimator at L=3, the use of bootstrap confidence intervals, and the explicit statement of assumptions. The central claim, however, rests on assumptions that are only partially tested, most importantly the size-independence of the QA error on the cubic dimer geometry.
major comments (3)
- [Section III, Eq. (5), Fig. 4] The L=4 values of the indirect estimator are computed by setting \hat\epsilon^QA_MPS equal to the L=3 median value of \epsilon^QA_MPS. The constancy of QA error with system size is cited only to square-lattice data in Ref. [4] (Fig. S25), not to cubic dimer lattices or bimodal couplings. If the QA error grows with system size on this geometry, the quadrature subtraction in Eq. (5) inflates \hat\epsilon^TNS_MPS at L=4 and could mask the predicted decrease. Please provide a sensitivity analysis: report the critical value of \epsilon^QA_MPS(L=4) at which the observed flat or increasing trend would become a decrease, and present the available \epsilon^QA_MPS versus L data for the cubic dimer geometry. Without this, the central disconfirmation claim is not fully supported.
- [Section III, Fig. 4] The conclusion 'There is no evidence of \hat\epsilon^TNS_MPS decreasing from L=3 to L=4 ... This disagrees with the predictions of Ref. [10]' treats a null result as a disconfirmation. A null result only contradicts a prediction if the confidence interval excludes the predicted decrease. The predicted magnitude is not quoted, and no confidence interval for the L=3-to-L=4 difference is given. Please quantify the change (e.g., difference with CI) and compare it explicitly with the Ref. [10] prediction, or soften the claim to 'no evidence' without asserting disagreement.
- [Supplementary VII] The dimer-perfect-correlation approximation is applied to bimodal inputs, where the authors note ground-truth dimer correlations as low as 0.7; the mean value 0.991 is quoted only for L=3 at ta=20ns. Since the central claim compares L=3 and L=4 across ta up to 40ns, a size- or time-dependent bias in this approximation could propagate into \hat\epsilon^TNS_MPS. Please report mean and minimum dimer correlations for all ensembles used (including L=4 if available), or compute exact correlation sets for a subset of instances to bound the bias.
minor comments (5)
- [Supplementary VII] Typo: 'significantly effect estimates' should be 'significantly affect estimates'.
- [Fig. 2 caption] Typo: 'Ast a increases' should be 'As t_a increases'.
- [Section III] The notation \epsilon^TNS_MPS and \hat\epsilon^TNS_MPS is easy to confuse; consider renaming the indirect estimator (e.g., \tilde\epsilon^TNS_MPS) or adding a short glossary.
- [Section II] The term 'biclique' is used without definition; please define it for readers not familiar with D-Wave embedding terminology.
- [Eq. (3)] State explicitly that the sums in the normalized l2 norm run over all pairs i<j and that the denominator uses ground-truth correlations; this is clear but could be stated for precision.
Circularity Check
No circular reduction: the L=3-to-L=4 comparison is measured; the QA-error constancy assumption is a stated limitation, not a built-in conclusion.
full rationale
The central claim is a measured scaling comparison, not a fitted or definitional identity. Equation (5) is an estimator for BP-TNS error against MPS, combining measured epsilon_TNS_QA with an assumed hat-epsilon_QA_MPS taken from the largest available MPS-validated size (L=3). The estimator is validated at L=3 in Fig. 3 against direct MPS ground truth. The L=4 value is then computed from newly measured epsilon_TNS_QA data, so the observed flatness/increase in Fig. 4 is not forced by the estimator; it depends on the actual measured values. The paper explicitly flags the key assumption: 'The second assumption is that epsilon_QA_MPS is approximately constant as a function of system size in QA,' and cites Ref. [4] for square-lattice evidence. This is an extrapolation to cubic dimer/bimodal ensembles and a genuine robustness limitation, but it is not a circular equivalence: a larger QA error at L=4 would inflate the estimate, yet that is a bias scenario, not a logical identity. Self-citations to Ref. [4] are independent published evidence with MPS-ground-truth validation, so they do not constitute load-bearing circularity. The dimer-perfect-correlation approximation in Supplementary VII is acknowledged and is a modeling simplification, not a redefinition of the target. No step in the derivation reduces to its own inputs: the prediction attributed to Ref. [10] is refuted by measurement, and Eq. (5) is not used to define the observed trend into existence. Score 2 reflects the presence of minor self-citation and the unverified size-independence assumption, which are correctness risks, not circular steps.
Axiom & Free-Parameter Ledger
free parameters (2)
- energy scale α =
0.4 to 1.0 (chosen by hand depending on ta and coupling distribution)
- BP-TNS bond dimension χ =
8 (cubic dimer), 16 (diamond)
axioms (5)
- domain assumption QA two-body correlation errors are statistically independent of BP-TNS errors (correlation coefficient 0.07 for uniform at ta=20ns)
- domain assumption QA error ϵ_QA_MPS is approximately constant with system size
- standard math MPS-TDVP with high bond dimension provides a valid ground truth at accessible sizes
- domain assumption Trotterized evolution to s=0.6 is equivalent to full quench s=1.0 within error
- ad hoc to paper Two spins in a dimer are perfectly correlated when expanding the measured correlation set
Cite this review
Pith. "Pith review of Evaluating classical simulations with a quantum processor." pith.science (2026). https://pith.science/paper/RQVEMR65
@misc{pith2026250815759,
author = {Pith},
title = {Pith review of: Evaluating classical simulations with a quantum processor},
year = {2026},
howpublished = {\url{https://pith.science/paper/RQVEMR65}},
note = {Machine review of arXiv:2508.15759}
}
read the original abstract
As simulations of quantum systems cross the limits of classical computability, both quantum and classical approaches become hard to verify. Scaling predictions are therefore based on local structure and asymptotic assumptions, typically with classical methods being used to evaluate quantum simulators where possible. Here, in contrast, we use a quantum annealing processor to produce a ground truth for evaluating classical tensor-network methods whose scaling has not yet been firmly established. Our observations run contrary to previous scaling predictions, demonstrating the need for caution when extrapolating the accuracy of classical simulations of quantum dynamics. Our results demonstrate that the virtuous cycle of competition between classical and quantum simulations can lend insight in both directions.
Figures
Forward citations
Cited by 1 Pith paper
-
Comment on "Beyond-classical computation in quantum simulation"
With Monte Carlo noise and autocorrelation accounted for, Neural Quantum States achieve lower correlation error than the QPU on the studied 2D spin-glass annealing instances.
Reference graph
Works this paper leans on
-
[1]
Preskill, Quantum computing and the entanglement frontier (2012), arXiv:1203.5813
J. Preskill, Quantum computing and the entanglement frontier (2012), arXiv:1203.5813
Pith/arXiv arXiv 2012
-
[2]
A. J. Daley, I. Bloch, C. Kokail, S. Flannigan, N. Pearson, M. Troyer, and P. Zoller, Practical quantum advantage in quantum simulation, Nature607, 667 (2022)
2022
- [3]
-
[4]
A. D. King, A. Nocera, M. M. Rams, J. Dziarmaga, R. Wiersema,et al., Beyond-classical computation in quantum simulation, Science388, 199 (2025)
work page 2025
-
[5]
T. I. Andersen, N. Astrakhantsev, A. H. Karamlou, J. Berndtsson, J. Motruk,et al., Thermalization and crit- icality on an analogue–digital quantum simulator, Nature 638, 79 (2025)
work page 2025
-
[6]
R. Haghshenas, E. Chertkov, M. Mills, W. Kadow, S.-H. Lin,et al., Digital quantum magnetism at the frontier of classical simulations (2025), arXiv:2503.20870
Pith/arXiv arXiv 2025
-
[7]
A. Berezutskii, M. Liu, A. Acharya, R. Ellerbrock, J. Gray,et al., Tensor networks for quantum comput- ing, Nature Reviews Physics 10.1038/s42254-025-00853-1 (2025)
-
[8]
J. Tindall and D. Sels, Confinement in the Transverse Field Ising Model on the Heavy Hex Lattice, Physical 6 5 10 15 10−2 10−1 100 5 10 15 10−2 10−1 100 5 10 15 10−2 10−1 100 5 10 15 10−2 10−1 100 5 10 15 10−2 10−1 100 5 10 15 10−2 10−1 100 5 10 15 101 102 103 104 lmax = 7 lmax = 4 lmax = 0 χχχ χ εc εc Seconds per two-body measurement (12 cores) εQA MPS ε...
work page 2024
-
[9]
Beguˇ si´ c, J
T. Beguˇ si´ c, J. Gray, and G. K.-L. Chan, Fast and con- verged classical simulations of evidence for the utility of quantum computing before fault tolerance, Science Ad- vances10, eadk4321 (2024)
2024
-
[10]
J. Tindall, A. Mello, M. Fishman, M. Stoudenmire, and D. Sels, Dynamics of disordered quantum systems with two- and three-dimensional tensor networks (2025)
work page 2025
-
[11]
G. Park, J. Gray, and G. K.-L. Chan, Simulating quan- tum dynamics in two-dimensional lattices with tensor network influence functional belief propagation (2025), arXiv:2504.07344 [quant-ph]
arXiv 2025
-
[12]
M. S. Rudolph and J. Tindall, Simulating and Sampling from Quantum Circuits with 2D Tensor Networks (2025), arXiv:2507.11424
arXiv 2025
-
[13]
Boixo, S
S. Boixo, S. V. Isakov, V. N. Smelyanskiy, R. Babbush, N. Ding, Z. Jiang, M. J. Bremner, J. M. Martinis, and H. Neven, Characterizing quantum supremacy in near- term devices, Nature Physics14, 595 (2018)
2018
- [14]
- [15]
-
[16]
G. Vidal, Efficient Classical Simulation of Slightly Entan- gled Quantum Computations, Physical Review Letters 91, 147902 (2003)
work page 2003
-
[17]
Schollw¨ ock, The density-matrix renormalization group in the age of matrix product states, Annals of Physics 326, 96 (2011)
U. Schollw¨ ock, The density-matrix renormalization group in the age of matrix product states, Annals of Physics 326, 96 (2011)
2011
- [18]
-
[19]
Haferkamp, D
J. Haferkamp, D. Hangleiter, J. Eisert, and M. Gluza, Contracting projected entangled pair states is average- case hard, Phys. Rev. Res.2, 013010 (2020)
2020
-
[20]
R. Haghshenas, M. J. O’Rourke, and G. K.-L. Chan, Con- version of projected entangled pair states into a canonical form, Phys. Rev. B100, 054404 (2019)
work page 2019
-
[21]
Evenbly, Gauge fixing, canonical forms, and optimal truncations in tensor networks with closed loops, Phys
G. Evenbly, Gauge fixing, canonical forms, and optimal truncations in tensor networks with closed loops, Phys. Rev. B98, 085155 (2018)
2018
-
[22]
Tindall and M
J. Tindall and M. Fishman, Gauging tensor networks with belief propagation, SciPost Physics15, 222 (2023)
2023
-
[23]
H. C. Jiang, Z. Y. Weng, and T. Xiang, Accurate De- termination of Tensor Network State of Quantum Lat- tice Models in Two Dimensions, Physical Review Letters 101, 090603 (2008)
work page 2008
-
[24]
V. Murg, F. Verstraete, and J. I. Cirac, Variational study of hard-core bosons in a two-dimensional optical lattice 7 using projected entangled pair states, Phys. Rev. A75, 033605 (2007)
work page 2007
-
[25]
Or´ us and G
R. Or´ us and G. Vidal, Simulation of two-dimensional quantum systems on an infinite lattice revisited: Cor- ner transfer matrix for tensor contraction, Phys. Rev. B 80, 094403 (2009)
2009
-
[26]
Z. Y. Xie, H. C. Jiang, Q. N. Chen, Z. Y. Weng, and T. Xiang, Second renormalization of tensor-network states, Phys. Rev. Lett.103, 160601 (2009)
2009
-
[27]
J. Dziarmaga, Time evolution of an infinite projected en- tangled pair state: Neighborhood tensor update, Physical Review B104, 094411 (2021)
work page 2021
-
[28]
J. Haegeman, J. I. Cirac, T. J. Osborne, I. Pi ˚Aorn, H. Verschelde, and F. Verstraete, Time-dependent vari- ational principle for quantum lattices, Physical Review Letters107, 1 (2011)
work page 2011
-
[29]
Haegeman, C
J. Haegeman, C. Lubich, I. Oseledets, B. Vandereycken, and F. Verstraete, Unifying time evolution and optimiza- tion with matrix product states, Physical Review B94, 165116 (2016)
2016
-
[30]
Alkabetz and I
R. Alkabetz and I. Arad, Tensor networks contraction and the belief propagation algorithm, Physical Review Research3, 023073 (2021)
2021
-
[31]
G. Evenbly, N. Pancotti, A. Milsted, J. Gray, and G. K.- L. Chan, Loop Series Expansions for Tensor Networks (2024)
work page 2024
-
[32]
A. D. King, J. Raymond, T. Lanting, R. Harris, A. Zucca, et al., Quantum critical dynamics in a 5,000-qubit pro- grammable spin glass, Nature617, 61 (2023)
work page 2023
-
[33]
M. Bernaschi, I. Gonz´ alez-Adalid Pemart ´ ın, V. Mart ´ ın- Mayor, and G. Parisi, The quantum transition of the two- dimensional Ising spin glass, Nature631, 749 (2024)
work page 2024
-
[34]
Quantum dynamics in frustrated Ising fullerenes
A. Lopez-Bezanilla, W. Bernoudy, K. Boothby, J. Ray- mond, A. Nocera, and A. D. King, Quantum dynam- ics in frustrated Ising fullerenes (2025), arXiv:2505.08994 [quant-ph]
work page internal anchor Pith review Pith/arXiv arXiv 2025
- [35]
- [36]
- [37]
-
[38]
iTensorNetworks.jl (2023)
work page 2023
-
[39]
M. Fishman, S. White, and E. Stoudenmire, The ITen- sor Software Library for Tensor Network Calculations, SciPost Physics Codebases4, 4 (2022). ACKNOWLEDGMENTS The authors are grateful to Joseph Tindall for help- ful discussions. Work at UBC was supported by Natural Sciences and Engineering Research Council of Canada (NSERC) Alliance Quantum Program (Gran...
work page 2022
This paper was first reviewed by deepseek-v4-flash on August 5, 2026.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.