Recognition: 2 theorem links
FTPrimitiveBench: A Benchmark Suite For Logical Computation Under Hardware-Motivated and Biased Noise Models
Pith reviewed 2026-05-08 18:36 UTC · model grok-4.3
The pith
Structured noise causes logical quantum primitives to show distinct performance depending on the noise type, primitive, and decoder.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
FTPrimitiveBench links custom or representative structured noise specifications to generators for surface-code Clifford primitives and shows through simulation that these noise families produce qualitatively different outcomes for logical memory, lattice surgery, transversal Hadamard, and phase gates. The outcomes are shaped by the interplay between the noise model, the primitive, and the decoder, extending conventional memory-only benchmarks to active logical computation and enabling hardware-aware co-design of fault-tolerant protocols.
What carries the argument
FTPrimitiveBench, a standardized suite that connects noise-model specifications to circuit generators for logical primitives so their interaction under biased or non-uniform noise can be simulated reproducibly.
If this is right
- Memory-only benchmarks miss noise interactions that appear once active logical operations such as lattice surgery are included.
- Decoder performance depends on the noise structure, so the same decoder can be optimal or suboptimal depending on the bias or non-uniformity present.
- Hardware-aware co-design becomes feasible once noise models and primitive constructions are linked in a standardized way.
- Comparative studies of different QEC protocols and decoders can be performed reproducibly by using the same noise-primitive interface.
Where Pith is reading between the lines
- Decoders could be selected or trained on the basis of the dominant noise bias measured for a given device or region.
- Different logical primitives might be preferred in different parts of a chip to match local error distributions.
- The same benchmarking approach could be applied to other error-correcting codes to identify which primitives benefit most from hardware-specific noise features.
- Extending the generators to include more complex temporal correlations would allow tests of whether current findings hold for longer computations.
Load-bearing premise
The selected families of structured noise are representative enough of the main heterogeneities and correlations present in real quantum hardware for evaluating logical primitives.
What would settle it
Calibrate the benchmark's noise parameters from measurements on a physical device, run the simulated primitives, then compare the predicted logical error rates against the error rates measured when the same primitives are executed on that device.
Figures
read the original abstract
Fault-tolerant quantum computing requires understanding how error-correcting codes perform on diverse physical hardware. This is typically assessed via noisy stabilizer simulation of logical circuits at HPC scale, combined with a noise model that yields a logical error rate for the relevant code distances and depths. The uniform depolarizing model is the standard baseline, but its homogeneous assumptions fail to capture the heterogeneity, asymmetries, and correlations of real devices, where Pauli, measurement, and spatio-temporal errors are not weakly coupled. Yet these same structured features create opportunities for joint code-hardware co-design, motivating noise models that more faithfully reflect target hardware while remaining tractable to simulate. We introduce FTPrimitiveBench, a systematic benchmarking approach for studying how logical primitives interact with hardware-motivated noise. It supports both custom specifications and representative structured noise families: Pauli bias, measurement bias, and spatial or spatio-temporal non-uniformity -- together with generators for core surface-code Clifford primitives: logical memory, lattice surgery, transversal logical Hadamard, and the logical phase gate via lattice surgery. We find that structured noise affects these primitives in qualitatively distinct ways, with outcomes shaped by the interplay between noise model, primitive, and decoder choice. These results extend memory benchmarks to active logical computation, where the interaction between noise structure and primitive implementation matters. By standardizing the link between noise-model specification and primitive construction, FTPrimitiveBench enables reproducible comparative studies of QEC protocols and decoders, supporting hardware-aware co-design of fault-tolerant architectures. Code: https://github.com/ShuwenKan/FTPrimitiveBench.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript introduces FTPrimitiveBench, a benchmark suite for studying logical primitives (logical memory, lattice surgery, transversal logical Hadamard, and logical phase gate via lattice surgery) under hardware-motivated structured noise models, specifically Pauli bias, measurement bias, and spatial or spatio-temporal non-uniformity. It supplies generators for both the primitives and the noise families, performs noisy stabilizer simulations, and reports qualitative findings that structured noise affects the primitives in distinct ways shaped by the interplay of noise model, primitive implementation, and decoder choice. The work positions this as an extension of memory-only benchmarks to active logical computation and emphasizes standardization and reproducibility via open code.
Significance. If the reported qualitative distinctions hold under the supplied noise families, the benchmark offers a practical, standardized tool for evaluating how realistic noise heterogeneities interact with fault-tolerant primitives, supporting hardware-aware co-design of QEC protocols and decoders. A clear strength is the provision of open code for primitive generators and noise models, which makes the central empirical observations directly reproducible without hidden parameters or circular self-references.
minor comments (3)
- Abstract: the phrase 'qualitative findings from simulations' would benefit from a single sentence specifying the code distances, depths, and number of shots used, to give readers an immediate sense of the simulation scale.
- Section on noise models (likely §3 or §4): the description of how spatial and spatio-temporal non-uniformity is implemented could include a brief pseudocode snippet or explicit probability map example for one primitive, to aid immediate reproducibility.
- Results section: while the qualitative distinctions are the main claim, adding a small table summarizing the relative ordering of logical error rates across the four primitives for each noise family would make the interplay easier to compare at a glance.
Simulated Author's Rebuttal
We thank the referee for their positive assessment of FTPrimitiveBench, including recognition of its utility as a standardized, reproducible tool for evaluating logical primitives under hardware-motivated noise models, and for the recommendation to accept the manuscript.
Circularity Check
No significant circularity; empirical benchmark results are independent
full rationale
The paper introduces FTPrimitiveBench as a new simulation framework for logical primitives under structured noise models and reports qualitative distinctions observed when running those simulations. All central claims are direct outputs of the new benchmark executions rather than reductions of fitted parameters, self-defined quantities, or load-bearing self-citations. The provided open code allows independent reproduction of the reported interplay between noise families, primitives, and decoders. No derivation chain collapses by construction to the paper's own inputs.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption Noisy stabilizer simulation provides a faithful model for evaluating logical error rates of surface-code primitives under the specified noise families.
Reference graph
Works this paper leans on
-
[1]
Scott Aaronson and Daniel Gottesman. 2004. Improved simulation of stabilizer circuits.Physical Review A—Atomic, Molecular, and Optical Physics 70, 5 (2004), 052328
2004
- [2]
-
[3]
Google Quantum AI. 2023. Suppressing quantum errors by scaling a surface code logical qubit.Nature614, 7949 (2023), 676 – 681
2023
-
[4]
Google Quantum AI. 2025. Quantum error correction below the surface code threshold.Nature638, 8052 (2025), 920–926. doi:10.1038/s41586-024- 08449-y
- [5]
-
[6]
Panos Aliferis, Frederico Brito, David P DiVincenzo, John Preskill, Matthias Steffen, and Barbara M Terhal. 2009. Fault-tolerant computing with biased-noise superconducting qubits: a case study.New Journal of Physics11, 1 (2009), 013061
2009
-
[7]
James M Auger, Hussain Anwar, Mercedes Gimeno-Segovia, Thomas M Stace, and Dan E Browne. 2017. Fault-tolerance thresholds for the surface code with fabrication errors.Physical Review A96, 4 (2017), 042316
2017
-
[8]
Ryan Babbush, Jarrod R McClean, Michael Newman, Craig Gidney, Sergio Boixo, and Hartmut Neven. 2021. Focus beyond quadratic speedups for error-corrected quantum advantage.PRX quantum2, 1 (2021), 010103
2021
-
[9]
Beale et al
Stefanie J. Beale et al. 2025. Logical error rates for the surface code under a mixed coherent and stochastic circuit-level noise model inspired by trapped ions.Physical Review Research(2025)
2025
- [10]
-
[11]
Michael E Beverland, Prakash Murali, Matthias Troyer, Krysta M Svore, Torsten Hoefler, Vadym Kliuchnikov, Guang Hao Low, Mathias Soeken, Aarthi Sundaram, and Alexander Vaschillo. 2022. Assessing requirements to scale to practical quantum advantage.arXiv preprint arXiv:2211.07629 (2022)
work page internal anchor Pith review arXiv 2022
-
[12]
Dolev Bluvstein, Simon J Evered, Alexandra A Geim, Sophie H Li, Hengyun Zhou, Tom Manovitz, Sepehr Ebadi, Madelyn Cain, Marcin Kalinowski, Dominik Hangleiter, et al. 2024. Logical quantum processor based on reconfigurable atom arrays.Nature626, 7997 (2024), 58–65
2024
- [13]
-
[14]
Hector Bombin, Chris Dawson, Ryan V Mishmash, Naomi Nickerson, Fernando Pastawski, and Sam Roberts. 2023. Logical blocks for fault-tolerant topological quantum computation.PRX Quantum4, 2 (2023), 020303
2023
-
[15]
J Pablo Bonilla Ataides, David K Tuckett, Stephen D Bartlett, Steven T Flammia, and Benjamin J Brown. 2021. The XZZX surface code.Nature communications12, 1 (2021), 2172
2021
-
[16]
Christopher Chamberland and Earl T Campbell. 2022. Circuit-level protocol and analysis for twist-based lattice surgery.Physical Review Research4, 2 (2022), 023090
2022
-
[17]
Zi-Han Chen, Ming-Cheng Chen, Chao-Yang Lu, and Jian-Wei Pan. 2026. Efficient Magic State Cultivation on RP 2.PRX Quantum7, 1 (2026), 010315. 31
2026
-
[18]
Zi-Han Chen, Ming-Cheng Chen, Chao-Yang Lu, and Jian-Wei Pan. 2026. Transversal logical Clifford gates on the rotated surface code with reconfigurable neutral atom arrays.Physical Review Letters136, 13 (2026), 130601
2026
-
[19]
Iris Cong, Harry Levine, Alexander Keesling, Dolev Bluvstein, Sheng-Tao Wang, and Mikhail D Lukin. 2022. Hardware-efficient, fault-tolerant quantum computation with Rydberg atoms.Physical Review X12, 2 (2022), 021049
2022
-
[20]
Niel de Beaudrap and Dominic Horsman. 2020. The ZX calculus is a language for surface code lattice surgery.Quantum4 (2020), 218
2020
-
[21]
Eric Dennis, Alexei Kitaev, Andrew Landahl, and John Preskill. 2002. Topological quantum memory.J. Math. Phys.43, 9 (2002), 4452–4505
2002
-
[22]
Alexander Erhard, Hendrik Poulsen Nautrup, Michael Meth, Lukas Postler, Roman Stricker, Martin Stadler, Vlad Negnevitsky, Martin Ringbauer, Philipp Schindler, Hans J Briegel, et al. 2021. Entangling logical qubits with lattice surgery.Nature589, 7841 (2021), 220–224
2021
-
[23]
Simon J Evered, Dolev Bluvstein, Marcin Kalinowski, Sepehr Ebadi, Tom Manovitz, Hengyun Zhou, Sophie H Li, Alexandra A Geim, Tout T Wang, Nishad Maskara, et al. 2023. High-fidelity parallel entangling gates on a neutral-atom quantum computer.Nature622, 7982 (2023), 268–272
2023
-
[24]
Austin G Fowler. 2012. Proof of finite surface code threshold for matching.Physical review letters109, 18 (2012), 180502
2012
- [25]
- [26]
-
[27]
Austin G Fowler, Matteo Mariantoni, John M Martinis, and Andrew N Cleland. 2012. Surface codes: Towards practical large-scale quantum computation.Physical Review A—Atomic, Molecular, and Optical Physics86, 3 (2012), 032324
2012
-
[28]
Austin G Fowler, Ashley M Stephens, and Peter Groszkowski. 2009. High-threshold universal quantum computation on the surface code.Physical Review A—Atomic, Molecular, and Optical Physics80, 5 (2009), 052312
2009
- [29]
- [30]
-
[31]
Michael R Geller and Zhongyuan Zhou. 2013. Efficient error models for fault-tolerant architectures and the Pauli twirling approximation.arXiv preprint arXiv:1305.2021(2013)
work page Pith review arXiv 2013
-
[32]
Lukas Gerster, Fernando Martínez-García, Pavel Hrmo, Martin W van Mourik, Benjamin Wilhelm, Davide Vodola, Markus Müller, Rainer Blatt, Philipp Schindler, and Thomas Monz. 2022. Experimental Bayesian calibration of trapped-ion entangling operations.PRX quantum3, 2 (2022), 020350
2022
-
[33]
Spiro Gicev, Lloyd C. L. Hollenberg, and Muhammad Usman. 2024. Quantum computer error structure probed by quantum error correction syndrome measurements.Phys. Rev. Res.6 (Dec 2024), 043249. Issue 4. doi:10.1103/PhysRevResearch.6.043249
-
[34]
Craig Gidney. [n. d.]. Crumble. https://algassert.com/crumble. Accessed: 2026-04-30
2026
-
[35]
Craig Gidney. 2021. Stim: a fast stabilizer circuit simulator.Quantum5 (2021), 497
2021
-
[36]
Craig Gidney. 2024. Inplace Access to the Surface Code Y Basis.Quantum8 (April 2024), 1310. doi:10.22331/q-2024-04-08-1310
-
[37]
Craig Gidney. 2025. How to factor 2048 bit RSA integers with less than a million noisy qubits.arXiv preprint arXiv:2505.15917(2025)
work page internal anchor Pith review arXiv 2025
-
[38]
Craig Gidney, Michael Newman, Austin Fowler, and Michael Broughton. 2021. A fault-tolerant honeycomb memory.Quantum5 (2021), 605
2021
-
[39]
Craig Gidney, Noah Shutty, and Cody Jones. 2024. Magic state cultivation: growing T states as cheap as CNOT gates.arXiv preprint arXiv:2409.17595 (2024)
work page internal anchor Pith review arXiv 2024
-
[40]
1997.Stabilizer codes and quantum error correction
Daniel Gottesman. 1997.Stabilizer codes and quantum error correction. California Institute of Technology
1997
- [41]
-
[42]
Adrian Harkness, Shuwen Kan, Chenxu Liu, Meng Wang, John M Martyn, Shifan Xu, Diana Chamaki, Ethan Decker, Ying Mao, Luis F Zuluaga, et al
- [43]
-
[44]
Oscar Higgott. 2022. Pymatching: A python package for decoding quantum codes with minimum-weight perfect matching.ACM Transactions on Quantum Computing3, 3 (2022), 1–16
2022
-
[45]
Oscar Higgott and Craig Gidney. 2025. Sparse blossom: correcting a million errors per core second with minimum-weight matching.Quantum9 (2025), 1600
2025
-
[46]
Oscar Higgott and Craig Gidney. 2025. Sparse Blossom: correcting a million errors per core second with minimum-weight matching.Quantum9 (Jan. 2025), 1600. doi:10.22331/q-2025-01-20-1600
- [47]
-
[48]
Dominic Horsman, Austin G Fowler, Simon Devitt, and Rodney Van Meter. 2012. Surface code quantum computing by lattice surgery.New Journal of Physics14, 12 (2012), 123011
2012
-
[49]
William J Huggins, Tanuj Khattar, Amanda Xu, Matthew Harrigan, Christopher Kang, Guang Hao Low, Austin Fowler, Nicholas C Rubin, and Ryan Babbush. 2025. The FLuid Allocation of Surface code Qubits (FLASQ) cost model for early fault-tolerant quantum algorithms.arXiv preprint arXiv:2511.08508(2025)
- [50]
-
[51]
Amara Katabarwa and Michael R Geller. 2015. Logical error rate in the Pauli twirling approximation.Scientific reports5, 1 (2015), 14670. 32
2015
- [52]
-
[53]
A Yu Kitaev. 2003. Fault-tolerant quantum computation by anyons.Annals of physics303, 1 (2003), 2–30
2003
-
[54]
Paul V Klimov, Julian Kelly, Zijun Chen, Matthew Neeley, Anthony Megrant, Brian Burkett, Rami Barends, Kunal Arya, Ben Chiaro, Yu Chen, et al
-
[55]
Fluctuations of energy-relaxation times in superconducting qubits.Physical review letters121, 9 (2018), 090502
2018
-
[56]
Sebastian Krinner, Nathan Lacroix, Ants Remm, Agustin Di Paolo, Elie Genois, Catherine Leroux, Christoph Hellings, Stefania Lazar, Francois Swiadek, Johannes Herrmann, et al. 2022. Realizing repeated quantum error correction in a distance-three surface code.Nature605, 7911 (2022), 669–674
2022
- [57]
-
[58]
Raphaël Lescanne, Marius Villiers, Théau Peronnin, Alain Sarlette, Matthieu Delbecq, Benjamin Huard, Takis Kontos, Mazyar Mirrahimi, and Zaki Leghtas. 2020. Exponential suppression of bit-flips in a qubit encoded in an oscillator.Nature Physics16, 5 (2020), 509–513
2020
-
[59]
Daniel Litinski. 2019. A game of surface codes: Large-scale quantum computing with lattice surgery.Quantum3 (2019), 128
2019
-
[60]
Daniel Litinski. 2019. Magic state distillation: Not as costly as you think.Quantum3 (2019), 205
2019
-
[61]
Áron Márton and János K Asbóth. 2025. Optimal number of stabilizer measurement rounds in an idling surface code patch.Quantum9 (2025), 1767
2025
-
[62]
Áron Márton, Luis Colmenarez, Lukas Bödeker, and Markus Müller. 2025. Lattice surgery-based logical state teleportation via noisy links.Physical Review Research7, 3 (2025), 033238
2025
-
[63]
Satvik Maurya, Joshua Viszlai, Nithin Raveendran, Poulami Das, and Swamit Tannu. 2025. decoder-bench: Benchmarking Decoders for Quantum Error Correction. In2025 IEEE International Symposium on Workload Characterization (IISWC). IEEE, 286–295
2025
-
[64]
Matt McEwen, Lara Faoro, Kunal Arya, Andrew Dunsworth, Trent Huang, Seon Kim, Brian Burkett, Austin Fowler, Frank Arute, Joseph C Bardin, et al. 2022. Resolving catastrophic error bursts from cosmic rays in large arrays of superconducting qubits.Nature Physics18, 1 (2022), 107–111
2022
-
[65]
Steven A Moses, Charles H Baldwin, Michael S Allman, R Ancona, L Ascarrunz, C Barnes, J Bartolotta, B Bjork, P Blanchard, M Bohn, et al. 2023. A race-track trapped-ion quantum processor.Physical Review X13, 4 (2023), 041052
2023
- [66]
-
[67]
Christophe Piveteau, Christopher T Chubb, and Joseph M Renes. 2024. Tensor-network decoding beyond 2d.PRX Quantum5, 4 (2024), 040303
2024
-
[68]
Harald Putterman, Kyungjoo Noh, Connor T Hann, Gregory S MacCabe, Shahriar Aghaeimeibodi, Rishi N Patel, Menyoung Lee, William M Jones, Hesam Moradinejad, Roberto Rodriguez, et al. 2025. Hardware-efficient quantum error correction via concatenated bosonic qubits.Nature638, 8052 (2025), 927–934
2025
-
[69]
Quantum Computing Stack Exchange. [n. d.]. When calculating code thresholds, why do we need to take a number of rounds that scales with distance? https://quantumcomputing.stackexchange.com/questions/24048. Accessed: 2026-05-05
2026
-
[70]
2022.LDPC: Python tools for low density parity check codes
Joschka Roffe. 2022.LDPC: Python tools for low density parity check codes. https://pypi.org/project/ldpc/
2022
-
[71]
Joschka Roffe, David R. White, Simon Burton, and Earl Campbell. 2020. Decoding across the quantum low-density parity-check code landscape. Physical Review Research2, 4 (Dec 2020). doi:10.1103/physrevresearch.2.043423
-
[72]
Neil J Ross and Peter Selinger. 2014. Optimal ancilla-free Clifford+ T approximation of z-rotations.arXiv preprint arXiv:1403.2975(2014)
work page Pith review arXiv 2014
-
[73]
Mohamed A Shalby, Renyu Wang, Denis Sedov, and Leonid P Pryadko. 2025. Optimized noise-resilient surface code teleportation interfaces.Physical Review A112, 2 (2025), L020403
2025
-
[74]
Peter W Shor. 1996. Fault-tolerant quantum computation. InProceedings of 37th conference on foundations of computer science. IEEE, 56–65
1996
-
[75]
Volodymyr Sivak, Alexis Morvan, Michael Broughton, Matthew Neeley, Alec Eickbusch, Dmitry Abanin, Amira Abbas, Rajeev Acharya, Laleh Aghababaie Beni, Georg Aigeldinger, et al. 2025. Reinforcement learning control of quantum error correction.arXiv preprint arXiv:2511.08493 (2025)
-
[76]
Volodymyr Sivak, Michael Newman, and Paul Klimov. 2024. Optimization of decoder priors for accurate quantum error correction.Physical Review Letters133, 15 (2024), 150603
2024
-
[77]
Stephen T Spitz, Brian Tarasinski, Carlo WJ Beenakker, and Thomas E O’Brien. 2018. Adaptive Weight Estimator for Quantum Error Correction in a Time-Dependent Environment.Advanced Quantum Technologies1, 1 (2018), 1800012
2018
- [78]
-
[79]
Andrew M Steane. 1996. Error correcting codes in quantum theory.Physical Review Letters77, 5 (1996), 793
1996
-
[80]
Ashley M Stephens. 2014. Fault-tolerant thresholds for quantum error correction with the surface code.Physical Review A89, 2 (2014), 022321
2014
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.