Pith. sign in

REVIEW 3 major objections 6 minor 92 references

Action-Factored Multi-Agent Reinforcement Learning for Scalable Quantum Device Tuning

T0 review · 3 major / 6 minor · reviewed 2026-07-13 · grok-4.5

Pith's one-line read Learning a factored voltage basis online lets multi-agent RL tune quantum-dot arrays of unseen size in nearly constant steps.

desk verdict Solid methods paper: online action factorization plus role-shared MARL gives real sim scaling for QD tuning; hardware gap is the only load-bearing soft spot. read the letter →

arxiv 2607.09422 v1 pith:CS224CXY submitted 2026-07-10 cs.LG cond-mat.mes-hall

classification cs.LGcond-mat.mes-hall
keywords multi-agentreinforcementlearningquantumdotarraysaction-spacefactorizationgatevirtualizationzero-shotgeneralizationdevicetuningparametersharingchargestabilitydiagrams
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Tuning arrays of electrostatically defined quantum dots is hard because every gate voltage capacitively disturbs its neighbors, so local controllers fight each other. This paper claims that the right fix is not better coordination among coupled agents, but online learning of a virtual action basis that approximately diagonalizes the gate-to-dot capacitance matrix. Under that factorization the joint control problem becomes a sum of nearly independent local objectives, so agents of the same physical type can share a single policy and act from local charge-stability diagrams alone. The resulting system, QADAPT, is trained on four-dot devices and then deployed without retraining on arrays from two to eight dots, reaching target charge occupations and tunnel couplings with roughly constant time steps (near-linear measurement cost). If the claim holds, calibration time need no longer grow combinatorially with processor size.

What carries the argument

Adaptive action-space factorization: a lightweight CNN plus Kalman filter builds a time-dependent cross-capacitance matrix Φ_t that maps physical plunger voltages into virtual voltages whose local quadratic objectives become nearly separable.

What would settle it

Train once on a four-dot simulator, then measure the number of charge-stability-diagram acquisitions needed to reach a fixed tuning score on real two-, six- and eight-dot devices; if steps rise sharply with size or the policy fails to transfer, the central scaling claim is false.

Watch

Extended reading notes

Core claim

QADAPT shows that an online, Kalman-refined estimate of the local gate-to-dot capacitance matrix can reparameterize the joint voltage action space into approximately independent virtual controls. With that factorization, role-shared decentralized PPO policies trained only on four-dot systems zero-shot transfer to larger and smaller arrays while keeping the number of convergence steps nearly constant.

Load-bearing premise

The online capacitance estimate must stay accurate enough that each virtual voltage update mainly affects only its own dot; if real-device drift, hysteresis or closed-regime diagrams break that estimate, the claimed decoupling and size-independent scaling fail.

Editorial extensions

If this is right

  • Calibration of larger quantum-dot processors can reuse the same shared policies without combinatorial retraining.
  • Measurement budgets for reaching target charge and tunnel-coupling regimes scale roughly linearly rather than combinatorially with array length.
  • The same factorization-plus-role-sharing pattern applies, with only input/output changes, to simultaneous-drive tuning of superconducting qubits.
  • Centralized world-model or critic architectures become unnecessary once the physical action basis itself is made approximately separable.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the online factorization remains stable under mild hysteresis, the same modular agents could absorb slow parameter drift without full re-optimization.
  • The approach suggests a general design rule for other multi-parameter quantum control problems: first learn a physically grounded change of coordinates, then share policies across identical roles.
  • Hardware validation on devices that leave the open-reservoir regime would be the decisive next experiment for practical adoption.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces QADAPT, a cooperative multi-agent RL framework for tuning electrostatically defined quantum-dot arrays. It learns an online factored action basis by estimating local gate-to-dot capacitances from charge stability diagrams with a CNN plus Kalman filter, then trains role-shared (plunger/barrier) actor-critic policies in the virtualized coordinates under a Dec-POMDP formulation reduced to a contextual bandit (γ=0). In QArray simulations the method zero-shot transfers from a 4-dot training array to 2–8-dot systems, keeps roughly constant steps-to-target while CSD acquisitions scale near-linearly in N, and outperforms black-box optimizers and several MARL/model-based baselines under a fixed measurement budget. Appendix A supplies local-linearization mismatch and preconditioning bounds; Appendix G sketches an analogous factorization for superconducting-qubit all-XY tune-up.

Significance. If the simulation results transfer, the work addresses a genuine bottleneck: combinatorial growth of joint voltage spaces and cross-talk-induced non-stationarity that currently limit autonomous tuning beyond few-dot devices. The combination of physically motivated online virtualization, role-based parameter sharing, and CTDE is a clear, modular design that is more scalable than centralized world models or joint-action critics. Strengths that should be credited include: (i) explicit mismatch/contraction analysis under local linearization (Appendix A, Thm. A.4, Prop. A.10); (ii) systematic ablations isolating virtualization, Kalman filtering, and multi-agent structure (Table 1); (iii) size sweeps and multiple external baselines (Figs. 2–3); (iv) an honest Limitations section; and (v) released code. These make the paper a useful contribution to quantum-device automation and to structure-aware MARL even if hardware validation remains future work.

major comments (3)
  1. [§3.2, Appendix A, Fig. 2, §5.3] The central scaling claim rests on Φ_t remaining a stable preconditioner so that C_t Φ_t^{-1}≈I and the joint quadratic objective becomes approximately separable (Appendix A, Thm. A.4 / Prop. A.10; §3.2). Fig. 2a–c and the IPPO / no-Kalman ablations (Table 1) support this only under QArray open-regime dynamics with relatively stationary couplings (Appendix B). §5.3 correctly flags hysteresis, drift, time-varying offsets, and isolated-regime CSD features as risks, but the manuscript never stress-tests them. Without at least simulated non-stationarity (e.g., process-noise sweeps, mid-episode capacitance jumps, or hysteresis), the zero-shot O(N) measurement claim and the hardware-facing abstract conclusion are under-secured. Please add such stress tests or materially scope the claim to stationary open-regime simulators.
  2. [§3.1, §3.4, Appendix B Table 3] Rewards are dense, immediate, and shaped from distance to a known target voltage configuration v* (and lever-arm normalization), with explicit zero/half-reward radii in Appendix B Table 3; the γ=0 bandit reduction (§3.4) relies on this. That is acceptable for controlled sim benchmarks, but it means agents are not discovering target charge/tunnel regimes from CSD-derived scores alone. For the claim of a scalable route to processor calibration, the manuscript should state clearly what replaces the oracle distance reward on hardware (classifier scores, physical cost functions, etc.) and, ideally, show that performance is not brittle when the reward is replaced by a noisier CSD-based proxy. As written, transferability of the learned policies beyond oracle-shaped sim is unclear.
  3. [Table 1, Fig. 3, §4] Several baselines that fail hard may not be decisive as currently reported. MADDPG and FACMAC show near-zero convergence after 500k environment steps with noted gradient pathologies (Table 1); without a hyperparameter-sensitivity or architecture-matched continuous-control setup, it is hard to attribute failure solely to the absence of action factorization versus optimization difficulty. DreamerV3 is omitted at 8 dots for compute reasons (Fig. 3). Please either strengthen these baselines (or document search budgets) or frame the comparison more carefully as “under a shared measurement budget and default continuous-control settings,” so the superiority claim remains proportionate.
minor comments (6)
  1. [Fig. 1] Fig. 1 caption and panel labels are dense; a short legend distinguishing physical vs virtual voltages (v_P vs u_P / vPx) would help readers who skip §3.2.
  2. [§3.2, Appendix A] Notation for virtual voltages switches among u_p, vPx, and Δu/Δv across §3.2, Appendix A, and the figure; unify symbols.
  3. [Table 1] Table 1 reports three convergence radii (2%/5%/10%) in a single cell; a multi-column layout would make effect sizes easier to read.
  4. [Appendix G, Figs. 6–7] The superconducting extension (Appendix G) is interesting but lightly evaluated relative to the main claims; either mark it clearly as a proof-of-concept or add a short quantitative comparison to a non-factored baseline there.
  5. [§1, §5.3] Minor prose: “plunger gates predominantly control… and barrier gates modulate” (§1) is clear, but “open’ regime” / ‘isolated’ regime (§5.3) should use consistent quotation style; check arXiv line breaks that split math (e.g., v^t_i).
  6. [§3.2, §5.1] State the CSD resolution and whether f_virt is frozen or fine-tuned when transferring across array sizes; this affects reproducibility of the zero-shot protocol.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: empirical MARL evaluation with standard preconditioning theory; claims are not forced by definition or self-citation.

full rationale

QADAPT’s load-bearing claims (zero-shot size transfer, near-constant steps, outperformance under a fixed CSD budget) are empirical results from training shared PPO policies on a 4-dot QArray environment and evaluating against external baselines (DreamerV3, Bayesian optimization, L-BFGS, Nelder–Mead, random search, IPPO, MAPPO, MADDPG, FACMAC) and ablations (Table 1, Figs. 2–3). Appendix A derives that if Φ_t approximates the local response Jacobian so C_t Φ_t^{-1} ≈ I, the one-step quadratic objective becomes approximately separable (Thm. A.4, Prop. A.10); that is a standard linearization/preconditioning argument, not a tautology that forces the reported convergence rates. Rewards are shaped from known simulated targets—normal for controlled RL benchmarks—and do not make the scaling claim true by construction. Self-citations (e.g. QArray) supply the simulator, not a uniqueness theorem that forbids alternatives. No fitted parameter is renamed a prediction; no ansatz is smuggled in as external fact. Limitations §5.3 openly condition success on stable online Φ under real non-idealities. Score 0.

Assumptions & free parameters 5 free parameters · 6 assumptions · 2 invented entities

The central claim rests on a capacitive linear-response model of cross-talk, a Dec-POMDP/bandit training reduction, and simulator physics (open regime, randomized couplings) rather than on new fundamental constants. Free parameters are standard RL/filter/architecture knobs and reward-shaping distances; invented entities are algorithmic constructs (QADAPT, virtual action basis Φ) with independent handles only insofar as they can be tested on hardware.

free parameters (5)
  • PPO clip ε, entropy coeff, LR, batch/minibatch sizes
    Training stability knobs (Table 4) that affect reported convergence rates; not derived from device physics.
  • Kalman process/measurement noise (implicit)
    Controls how aggressively Φ updates; authors note raising process noise as a drift mitigation, trading steps-to-convergence.
  • Reward shaping distances (zero/half reward radii, barrier reward radius)
    Table 3 distances define dense per-agent rewards; change the optimization landscape agents see.
  • Convergence radii 2%/5%/10% of tuning range
    Success definition in Table 1; rates depend on these thresholds.
  • f_virt CNN architecture and CSD resolution/scan size
    Determines quality of capacitance observations feeding Φ; ablations show backbone choice matters less than virtualization itself.
assumptions (6)
  • domain assumption Local device response is well-approximated by a Lipschitz Jacobian so one-step quadratic virtualization analysis applies (Assumption A.1, Lemma A.2).
    Underpins Appendix A contraction/separability claims used to justify action factorization.
  • domain assumption Cross-talk is capturable by a (time-varying) capacitive network estimated from local CSDs up to second-nearest neighbors.
    Core physics model of §3.2 and Fig. 1; fails if non-capacitive or strongly nonlinear effects dominate.
  • ad hoc to paper Immediate dense rewards make γ=0 (contextual bandit) sufficient; long-horizon credit assignment is unnecessary.
    §3.4 design choice supported by ablation but may fail under hysteresis/isolated-regime dynamics noted in Limitations.
  • domain assumption Devices operate in an open regime with reservoir charge exchange so steady-state CSD features remain informative.
    Stated limitation §5.3; isolated arrays change CSD structure.
  • domain assumption Plunger vs barrier role symmetry justifies two shared policy parameter sets independent of array size.
    Enables zero-shot agent addition; authors note fabrication may blur roles.
  • standard math Standard Dec-POMDP / CTDE multi-agent RL formalism and PPO clipped surrogate.
    Background RL machinery (§3.1, §3.4).
invented entities (2)
  • QADAPT framework (action-factored role-shared MARL for QD tuning)
    purpose: Name the modular pipeline combining online virtualization with shared decentralized policies.
    Algorithmic system, not a physical entity; evidence is simulation performance.
  • Online virtual action basis Φ_t from CNN+Kalman capacitance estimates
    purpose: Reparameterize plunger voltages to reduce cross-agent interference and precondition local control.
    Constructed online from measurements; falsifiable on hardware via residual cross-talk and tuning success, but not yet independently validated outside this paper’s sims.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Action-Factored Multi-Agent Reinforcement Learning for Scalable Quantum Device Tuning." pith.science (2026). https://pith.science/paper/CS224CXY

@misc{pith2026260709422,
  author       = {Pith},
  title        = {Pith review of: Action-Factored Multi-Agent Reinforcement Learning for Scalable Quantum Device Tuning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CS224CXY}},
  note         = {Machine review of arXiv:2607.09422}
}
read the original abstract

Cooperative multi-agent reinforcement learning is well suited to problems with large parameter spaces and exploitable local structure, such as the tuning of electrostatically-defined quantum-dot arrays. However, if parameter cross-talk is strong, a non-stationary environment from the perspective of any individual agent can destabilize learning - the same effect that plagues manual tuning of such systems. We propose using a factored representation of the action space, learned online, to decouple agents and minimize their interference. Our framework, QADAPT, uses this factorization to efficiently learn shared policies based on local measurements and rewards. With this modular strategy, we achieve zero-shot generalization to unseen quantum device sizes and maintain an approximately constant number of convergence steps to reach target regimes. This work provides a scalable route toward the rapid calibration of large-scale quantum processors.

Figures

Figures reproduced from arXiv: 2607.09422 by the authors.

Figure 1
Figure 1. Algorithmic Flow of QADAPT (a) Barrier gates (Bx) interdigitate plunger gates (Px), and primarily control dot-dot tunnel couplings and charge occupations, respectively. All gates contribute in defining the confinement potential necessary to form quantum dots (d1...dD). Gate cross-talk can be approximated by a capacitive network, which in general extends beyond nearest neighbors, greatly complicating array tuning. Ch… view at source ↗
Figure 2
Figure 2. Gate Virtualization and Scaling Performance of QADAPT. a) Kalman filter’s posterior uncertainty (variance) σ against absolute estimation error µˆerror for nearest neighbor gate-to-dot cross-capacitances. Each marker represents one time step in a 4-dot system (3 CSD measurements), averaged over 1000 episodes. The cross-capacitance has a fixed ground truth value of 0.7. A top-right to bottom-left trajectory demonstrat… view at source ↗
Figure 3
Figure 3. Domain-specific Benchmarks with Array Size. As the length of the quantum dot array increases from 2 to 8 (moving left to right), QADAPT maintains rapid convergence, out-performing all benchmark methods, including: model-based reinforcement learning (DreamerV3), Bayesian optimisation using Gaussian processes, the gradient free Nelder-Mead optimiser, the gradient-based Limited-memory Broyden-Fletcher-Goldfarb-Shanno a… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Average per-agent reward for QADAPT trained on a 4-dot array, in addition to each version [PITH_FULL_IMAGE:figures/full_fig_p024_4.png]
Figure 5
Figure 5. Figure 5: Trained agent behavior sampled over 100 episodes for a 4-dot system. The four colors in [PITH_FULL_IMAGE:figures/full_fig_p025_5.png]
Figure 6
Figure 6. Figure 6: All-XY Sequence Calibrated using QADAPT. After random initialization (top), QADAPT reliably tunes superconducting qubit parameters (bare qubit frequency, drive frequency, amplitude, phase, and DRAG coefficient) to near-optimal values in 20 steps (bottom). All violin pl…
Figure 7
Figure 7. Figure 7: Convergence of Superconducting Qubit Tuning. QADAPT generalizes well to the superconducting setting, also scaling zero-shot to qubit numbers of 2, 6 and 8 using a policy trained on 4 qubits. The score is defined as the normalized overlap with the target excited state p…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

92 extracted references · 14 linked inside Pith

  1. [1]

    Sutton and Andrew G

    Richard S. Sutton and Andrew G. Barto.Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA, 2nd edition, 2018

  2. [2]

    Human-level control through deep reinforcement learning

    V olodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. Human-level control through deep reinforcement learning. nature, 518(7540):529–533, 2015

  3. [3]

    Continuous control with deep reinforcement learning, September 15 2020

    Timothy Paul Lillicrap, Jonathan James Hunt, Alexander Pritzel, Nicolas Manfred Otto Heess, Tom Erez, Yuval Tassa, David Silver, and Daniel Pieter Wierstra. Continuous control with deep reinforcement learning, September 15 2020. US Patent 10,776,692

  4. [4]

    Experimental deep reinforcement learning for error-robust gate-set design on a superconducting quantum computer

    Yuval Baum, Mirko Amico, Sean Howell, Michael Hush, Maggie Liuzzi, Pranav Mundada, Thomas Merkh, Andre RR Carvalho, and Michael J Biercuk. Experimental deep reinforcement learning for error-robust gate-set design on a superconducting quantum computer. PRX quantum, 2(4):040324, 2021

  5. [5]

    Deep reinforcement learning quantum control on ibmq platforms and qiskit pulse

    Rudy Semola, Lorenzo Moro, Davide Bacciu, and Enrico Prati. Deep reinforcement learning quantum control on ibmq platforms and qiskit pulse. In 2022 ieee international conference on quantum computing and engineering (qce), pages 759–762. IEEE, 2022

  6. [6]

    Realizing a deep reinforcement learning agent for real-time quantum feedback

    Kevin Reuer, Jonas Landgraf, Thomas Fösel, James O’Sullivan, Liberto Beltrán, Abdulkadir Akin, Gra- ham J Norris, Ants Remm, Michael Kerschbaum, Jean-Claude Besse, et al. Realizing a deep reinforcement learning agent for real-time quantum feedback. Nature Communications, 14(1):7138, 2023. 10

  7. [7]

    Quantum feedback control with a transformer neural network architecture

    Pranav Vaidhyanathan, Florian Marquardt, Mark T Mitchison, and Natalia Ares. Quantum feedback control with a transformer neural network architecture. Physical Review Research, 8(1):L012043, 2026

  8. [8]

    Deep reinforcement learning for efficient measurement of quantum devices

    V Nguyen, SB Orbell, Dominic T Lennon, Hyungil Moon, Florian Vigneau, Leon C Camenzind, Liuqi Yu, Dominik M Zumbühl, G Andrew D Briggs, Michael A Osborne, et al. Deep reinforcement learning for efficient measurement of quantum devices. npj Quantum Information, 7(1):100, 2021

Show all 92 references
  1. [9]

    Artificial intelligence for quantum computing

    Yuri Alexeev, Marwa H Farag, Taylor L Patti, Mark E Wolf, Natalia Ares, Alán Aspuru-Guzik, Simon C Benjamin, Zhenyu Cai, Shuxiang Cao, Christopher Chamberland, et al. Artificial intelligence for quantum computing. Nature Communications, 16(1):10829, 2025

  2. [10]

    Qcaleval: Benchmarking vision- language models for quantum calibration plot understanding

    Shuxiang Cao, Zijian Zhang, Abhishek Agarwal, Grace Bratrud, Niyaz R Beysengulov, Daniel C Cole, Alejandro Gómez Frieiro, Elena O Glen, Hao Hsu, Gang Huang, et al. Qcaleval: Benchmarking vision- language models for quantum calibration plot understanding. arXiv preprint arXiv:2...

  3. [11]

    Machine learning as an enabler of qubit scalability

    Natalia Ares. Machine learning as an enabler of qubit scalability. Nature Reviews Materials, 6(10):870– 871, 2021

  4. [12]

    Data needs and challenges for quantum dot devices automation

    Justyna P Zwolak, Jacob M Taylor, Reed W Andrews, Jared Benson, Garnett W Bryant, Donovan Buterakos, Anasua Chatterjee, Sankar Das Sarma, Mark A Eriksson, Eliška Greplová, et al. Data needs and challenges for quantum dot devices automation. npj Quantum Information, 10(1):105, 2024

  5. [13]

    Environment model construction toward auto-tuning of quantum dot devices based on model-based reinforcement learning

    Chihiro Kondo, Raisei Mizokuchi, Jun Yoneda, and Tetsuo Kodera. Environment model construction toward auto-tuning of quantum dot devices based on model-based reinforcement learning. APL Machine Learning, 3(1), 2025

  6. [14]

    Metasym: A symplectic meta-learning framework for physical intelligence

    Pranav Vaidhyanathan, Aristotelis Papatheodorou, Mark T Mitchison, Natalia Ares, and Ioannis Havoutis. Metasym: A symplectic meta-learning framework for physical intelligence. arXiv preprint arXiv:2502.16667, 2025

  7. [15]

    Meta-learning characteristics and dynamics of quantum systems

    Lucas Schorling, Pranav Vaidhyanathan, Jonas Schuff, Miguel J Carballido, Dominik Zumbühl, Gerard Milburn, Florian Marquardt, Jakob Foerster, Michael A Osborne, and Natalia Ares. Meta-learning characteristics and dynamics of quantum systems. arXiv preprint arXiv:2503.10492, 2025

  8. [16]

    Reinforcement learning for quantum technology

    Marin Bukov and Florian Marquardt. Reinforcement learning for quantum technology. arXiv preprint arXiv:2601.18953, 2026

  9. [17]

    Semiconductor qubits in practice

    Anasua Chatterjee, Paul Stevenson, Silvano De Franceschi, Andrea Morello, Nathalie P de Leon, and Ferdinand Kuemmeth. Semiconductor qubits in practice. Nature Reviews Physics, 3(3):157–177, 2021

  10. [18]

    Spin-qubit control with a milli-kelvin cmos chip

    Samuel K Bartee, Will Gilbert, Kun Zuo, Kushal Das, Tuomo Tanttu, Chih Hwan Yang, Nard Du- moulin Stuyck, Sebastian J Pauka, Rocky Y Su, Wee Han Lim, et al. Spin-qubit control with a milli-kelvin cmos chip. Nature, pages 1–6, 2025

  11. [19]

    Industry-compatible silicon spin-qubit unit cells exceeding 99% fidelity

    Paul Steinacker, Nard Dumoulin Stuyck, Wee Han Lim, Tuomo Tanttu, MengKe Feng, Santiago Serrano, Andreas Nickl, Marco Candido, Jesus D Cifuentes, Ensar Vahapoglu, et al. Industry-compatible silicon spin-qubit unit cells exceeding 99% fidelity. Nature, pages 1–7, 2025

  12. [20]

    Simultaneous operation of an 18-qubit modular array in germanium

    Jurgen J Dijkema, Xin Zhang, Achilleas Bardakas, Daniel Bouman, Alice Cuzzocrea, David van Driel, Davide Girardi, Lucas EA Stehouwer, Giordano Scappucci, Anne-Marije J Zwerver, et al. Simultaneous operation of an 18-qubit modular array in germanium. arXiv preprint arXiv:2604.0...

  13. [21]

    Shared control of a 16 semiconductor quantum dot crossbar array

    Francesco Borsoi, Nico W Hendrickx, Valentin John, Marcel Meyer, Sayr Motz, Floor Van Riggelen, Amir Sammak, Sander L De Snoo, Giordano Scappucci, and Menno Veldhorst. Shared control of a 16 semiconductor quantum dot crossbar array. Nature Nanotechnology, 19(1):21–27, 2024

  14. [22]

    Fully autonomous tuning of a spin qubit

    Jonas Schuff, Miguel J Carballido, Madeleine Kotzagiannidis, Juan Carlos Calvo, Marco Caselli, Jacob Rawling, David L Craig, Barnaby van Straaten, Brandon Severin, Federico Fedele, et al. Fully autonomous tuning of a spin qubit. Nature Electronics, pages 1–10, 2026

  15. [23]

    Machine learning enables completely automatic tuning of a quantum device faster than human experts

    Hyungil Moon, Dominic T Lennon, James Kirkpatrick, Nina M van Esbroeck, Leon C Camenzind, Liuqi Yu, Florian Vigneau, Dominik M Zumbühl, G Andrew D Briggs, Michael A Osborne, et al. Machine learning enables completely automatic tuning of a quantum device faster than human exper...

  16. [24]

    Autotuning of double-dot devices in situ with machine learning

    Justyna P Zwolak, Thomas McJunkin, Sandesh S Kalantre, JP Dodson, ER MacQuarrie, DE Savage, MG Lagally, SN Coppersmith, Mark A Eriksson, and Jacob M Taylor. Autotuning of double-dot devices in situ with machine learning. Physical review applied, 13(3):034075, 2020. 11

  17. [25]

    Rapid autotuning of a sige quantum dot into the single-electron regime with machine learning and rf-reflectometry fpga-based measurements

    Marc-Antoine Roux, Joffrey Rivard, Victor Yon, Alexis Morel, Dominic Leclerc, Claude Rohrbacher, El Bachir Ndiaye, Felice Francesco Tafuri, Brendan Bono, Stefan Kubicek, et al. Rapid autotuning of a sige quantum dot into the single-electron regime with machine learning and rf-...

  18. [26]

    Automated all-rf tuning for spin qubit readout and control

    Cornelius Carlsson, Jaime Saez-Mollejo, Federico Fedele, Stefano Calcaterra, Daniel Chrastina, Giovanni Isella, Georgios Katsaros, and Natalia Ares. Automated all-rf tuning for spin qubit readout and control. arXiv preprint arXiv:2506.10834, 2025

  19. [27]

    Cooperative multi-agent control using deep reinforcement learning

    Jayesh K Gupta, Maxim Egorov, and Mykel Kochenderfer. Cooperative multi-agent control using deep reinforcement learning. In International conference on autonomous agents and multiagent systems, pages 66–83. Springer, 2017

  20. [28]

    Computer-automated tuning of semiconductor double quantum dots into the single-electron regime

    Timothy A Baart, Pieter T Eendebak, Christian Reichl, Werner Wegscheider, and Lieven MK Vandersypen. Computer-automated tuning of semiconductor double quantum dots into the single-electron regime. Applied Physics Letters, 108(21), 2016

  21. [29]

    Experimental online quantum dots charge autotuning using neural networks

    Victor Yon, Bastien Galaup, Claude Rohrbacher, Joffrey Rivard, Alexis Morel, Dominic Leclerc, Clement Godfrin, Ruoyu Li, Stefan Kubicek, Kristiaan De Greve, et al. Experimental online quantum dots charge autotuning using neural networks. Nano Letters, 25(10):3717–3725, 2025

  22. [30]

    Machine learning techniques for state recognition and auto-tuning in quantum dots

    Sandesh S Kalantre, Justyna P Zwolak, Stephen Ragole, Xingyao Wu, Neil M Zimmerman, MD Stewart Jr, and Jacob M Taylor. Machine learning techniques for state recognition and auto-tuning in quantum dots. npj Quantum Information, 5(1):6, 2019

  23. [31]

    Tuning arrays with rays: Physics-informed tuning of quantum dot charge states

    Joshua Ziegler, Florian Luthi, Mick Ramsey, Felix Borjans, Guoji Zheng, and Justyna P Zwolak. Tuning arrays with rays: Physics-informed tuning of quantum dot charge states. Physical Review Applied, 20(3):034067, 2023

  24. [32]

    All-rf-based coarse-tuning algorithm for quantum devices using machine learning

    Barnaby Van Straaten, Federico Fedele, Florian Vigneau, Joseph Hickie, Daniel Jirovec, Andrea Ballabio, Daniel Chrastina, Giovanni Isella, Georgios Katsaros, and Natalia Ares. All-rf-based coarse-tuning algorithm for quantum devices using machine learning. Physical Review Appl...

  25. [33]

    Automated tuning of double quantum dots into specific charge states using neural networks

    Renato Durrer, Benedikt Kratochwil, Jonne V Koski, Andreas J Landig, Christian Reichl, Werner Wegschei- der, Thomas Ihn, and Eliska Greplova. Automated tuning of double quantum dots into specific charge states using neural networks. Physical Review Applied, 13(5):054019, 2020

  26. [34]

    Minia- turizing neural networks for charge state autotuning in quantum dots

    Stefanie Czischek, Victor Yon, Marc-Antoine Genest, Marc-Antoine Roux, Sophie Rochette, Julien Cami- rand Lemyre, Mathieu Moras, Michel Pioro-Ladrière, Dominique Drouin, Yann Beilliard, et al. Minia- turizing neural networks for charge state autotuning in quantum dots. Machine...

  27. [35]

    Quantum device fine-tuning using unsupervised embedding learning

    Nina M van Esbroeck, Dominic T Lennon, Hyungil Moon, Vu Nguyen, Florian Vigneau, Leon C Ca- menzind, Liuqi Yu, Dominik M Zumbühl, G Andrew D Briggs, Dino Sejdinovic, et al. Quantum device fine-tuning using unsupervised embedding learning. New Journal of Physics, 22(9):095003, 2020

  28. [36]

    Cross-architecture tuning of silicon and sige-based quantum devices using machine learning

    Brandon Severin, Dominic T Lennon, Leon C Camenzind, Florian Vigneau, Federico Fedele, Daniel Jirovec, Andrea Ballabio, Daniel Chrastina, Giovanni Isella, Mathieu de Kruijf, et al. Cross-architecture tuning of silicon and sige-based quantum devices using machine learning. Scie...

  29. [37]

    Loading a quantum-dot based “qubyte” register

    Christian V olk, Anne-Marije J Zwerver, Uditendu Mukhopadhyay, Pieter T Eendebak, Cornelis Jacobus van Diepen, Juan Pablo Dehollain, Toivo Hensgens, Takafumi Fujita, Christian Reichl, Werner Wegscheider, et al. Loading a quantum-dot based “qubyte” register. npj Quantum Informa...

  30. [38]

    Cnn-assisted automatic cross-capacitance matrix update for virtual-gate control of quantum dot arrays

    Tatsuo Tsuzuki, Hideaki Yuta, Yui Muto, Arne Ludwig, Andreas Dirk Wieck, Akira Oiwa, and Takafumi Fujita. Cnn-assisted automatic cross-capacitance matrix update for virtual-gate control of quantum dot arrays. Japanese Journal of Applied Physics, 65(1):01SP03, 2026

  31. [39]

    Modular autonomous virtualization system for two-dimensional semiconductor quantum dot arrays

    Anantha S Rao, Donovan Buterakos, Barnaby van Straaten, Valentin John, Cécile X Yu, Stefan D Oost- erhout, Lucas Stehouwer, Giordano Scappucci, Menno Veldhorst, Francesco Borsoi, et al. Modular autonomous virtualization system for two-dimensional semiconductor quantum dot arra...

  32. [40]

    Automatic detection of single-electron regime of quantum dots and definition of virtual gates using u-net and clustering

    Yui Muto, Michael R Zielewski, Motoya Shinozaki, Kosuke Noro, and Tomohiro Otsuka. Automatic detection of single-electron regime of quantum dots and definition of virtual gates using u-net and clustering. arXiv preprint arXiv:2501.05878, 2025. 12

  33. [41]

    Automated charge transition detection in quantum dot charge stability diagrams

    Fabian Hader, Fabian Fuchs, Sarah Fleitmann, Karin Havemann, Benedikt Scherer, Jan V ogelbruch, Lotte Geck, and Stefan Van Waasen. Automated charge transition detection in quantum dot charge stability diagrams. IEEE Transactions on Quantum Engineering, 2025

  34. [42]

    End-to-end analysis of charge stability diagrams with transformers

    Rahul Marchand, Lucas Schorling, Cornelius Carlsson, Jonas Schuff, Barnaby van Straaten, Taylor L Patti, Federico Fedele, Joshua Ziegler, Parth Girdhar, Pranav Vaidhyanathan, et al. End-to-end analysis of charge stability diagrams with transformers. arXiv preprint arXiv:2508.1...

  35. [43]

    Auto- matic virtual voltage extraction of a 2x2 array of quantum dots with machine learning

    Giovanni A Oakes, Jingyu Duan, John JL Morton, Alpha Lee, Charles G Smith, and M Zalba. Auto- matic virtual voltage extraction of a 2x2 array of quantum dots with machine learning. arXiv preprint arXiv:2012.03685, 2020

  36. [44]

    A review of cooperative multi-agent deep reinforcement learning

    Afshin Oroojlooy and Davood Hajinezhad. A review of cooperative multi-agent deep reinforcement learning. Applied Intelligence, 53(11):13677–13722, 2023

  37. [45]

    Multi-agent reinforcement learning: Independent vs

    Ming Tan et al. Multi-agent reinforcement learning: Independent vs. cooperative agents. In Proceedings of the tenth international conference on machine learning, pages 330–337, 1993

  38. [46]

    Value-decomposition networks for cooperative multi-agent learning

    Peter Sunehag, Guy Lever, Audrunas Gruslys, Wojciech Marian Czarnecki, Vinicius Zambaldi, Max Jaderberg, Marc Lanctot, Nicolas Sonnerat, Joel Z Leibo, Karl Tuyls, et al. Value-decomposition networks for cooperative multi-agent learning. arXiv preprint arXiv:1706.05296, 2017

  39. [47]

    Monotonic value function factorisation for deep multi-agent reinforcement learning

    Tabish Rashid, Mikayel Samvelyan, Christian Schroeder De Witt, Gregory Farquhar, Jakob Foerster, and Shimon Whiteson. Monotonic value function factorisation for deep multi-agent reinforcement learning. Journal of Machine Learning Research, 21(178):1–51, 2020

  40. [48]

    Qtran: Learning to factorize with transformation for cooperative multi-agent reinforcement learning

    Kyunghwan Son, Daewoo Kim, Wan Ju Kang, David Earl Hostallero, and Yung Yi. Qtran: Learning to factorize with transformation for cooperative multi-agent reinforcement learning. In International conference on machine learning, pages 5887–5896. PMLR, 2019

  41. [49]

    Deep multi-agent reinforcement learning for decentralized continuous cooperative control

    Christian Schroeder de Witt, Bei Peng, Pierre-Alexandre Kamienny, Philip Torr, Wendelin Böhmer, and Shimon Whiteson. Deep multi-agent reinforcement learning for decentralized continuous cooperative control. arXiv preprint arXiv:2003.06709, 19, 2020

  42. [50]

    Coun- terfactual multi-agent policy gradients

    Jakob Foerster, Gregory Farquhar, Triantafyllos Afouras, Nantas Nardelli, and Shimon Whiteson. Coun- terfactual multi-agent policy gradients. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018

  43. [51]

    Multi-agent actor-critic for mixed cooperative-competitive environments

    Ryan Lowe, Yi I Wu, Aviv Tamar, Jean Harb, OpenAI Pieter Abbeel, and Igor Mordatch. Multi-agent actor-critic for mixed cooperative-competitive environments. Advances in neural information processing systems, 30, 2017

  44. [52]

    Facmac: Factored multi-agent centralised policy gradients

    Bei Peng, Tabish Rashid, Christian Schroeder de Witt, Pierre-Alexandre Kamienny, Philip Torr, Wendelin Böhmer, and Shimon Whiteson. Facmac: Factored multi-agent centralised policy gradients. Advances in neural information processing systems, 34:12208–12221, 2021

  45. [53]

    Learning multiagent communication with backpropagation

    Sainbayar Sukhbaatar, Rob Fergus, et al. Learning multiagent communication with backpropagation. Advances in neural information processing systems, 29, 2016

  46. [54]

    Graph policy gradients for large scale robot control

    Arbaaz Khan, Ekaterina Tolstaya, Alejandro Ribeiro, and Vijay Kumar. Graph policy gradients for large scale robot control. In Conference on robot learning, pages 823–834. PMLR, 2020

  47. [55]

    Scalable multi-agent reinforcement learning through intelligent information aggregation

    Siddharth Nayak, Kenneth Choi, Wenqi Ding, Sydney Dolan, Karthik Gopalakrishnan, and Hamsa Balakrishnan. Scalable multi-agent reinforcement learning through intelligent information aggregation. In International conference on machine learning, pages 25817–25833. PMLR, 2023

  48. [56]

    Multi-agent deep reinforcement learning with extremely noisy observations

    Ozsel Kilinc and Giovanni Montana. Multi-agent deep reinforcement learning with extremely noisy observations. arXiv preprint arXiv:1812.00922, 2018

  49. [57]

    Bridging marl to sarl: An order-independent multi-agent transformer via latent consensus

    Zijian Zhao, Jing Gao, and Sen Li. Bridging marl to sarl: An order-independent multi-agent transformer via latent consensus. arXiv preprint arXiv:2604.13472, 2026

  50. [58]

    Multi- agent reinforcement learning is a sequence modeling problem.Advances in Neural Information Processing Systems, 35:16509–16521, 2022

    Muning Wen, Jakub Kuba, Runji Lin, Weinan Zhang, Ying Wen, Jun Wang, and Yaodong Yang. Multi- agent reinforcement learning is a sequence modeling problem.Advances in Neural Information Processing Systems, 35:16509–16521, 2022

  51. [59]

    Hierarchical consensus-based multi-agent reinforcement learning for multi-robot cooperation tasks

    Pu Feng, Junkang Liang, Size Wang, Xin Yu, Xin Ji, Yiting Chen, Kui Zhang, Rongye Shi, and Wenjun Wu. Hierarchical consensus-based multi-agent reinforcement learning for multi-robot cooperation tasks. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems ...

  52. [60]

    Scaling multi-agent reinforcement learning with selective parameter sharing

    Filippos Christianos, Georgios Papoudakis, Muhammad A Rahman, and Stefano V Albrecht. Scaling multi-agent reinforcement learning with selective parameter sharing. In International Conference on Machine Learning, pages 1989–1998. PMLR, 2021

  53. [61]

    Is independent learning all you need in the starcraft multi-agent challenge? arXiv preprint arXiv:2011.09533, 2020

    Christian Schroeder De Witt, Tarun Gupta, Denys Makoviichuk, Viktor Makoviychuk, Philip HS Torr, Mingfei Sun, and Shimon Whiteson. Is independent learning all you need in the starcraft multi-agent challenge? arXiv preprint arXiv:2011.09533, 2020

  54. [62]

    The sur- prising effectiveness of ppo in cooperative multi-agent games

    Chao Yu, Akash Velu, Eugene Vinitsky, Jiaxuan Gao, Yu Wang, Alexandre Bayen, and Yi Wu. The sur- prising effectiveness of ppo in cooperative multi-agent games. Advances in neural information processing systems, 35:24611–24624, 2022

  55. [63]

    Trust region policy optimisation in multi-agent reinforcement learning

    Jakub Grudzien Kuba, Ruiqing Chen, Muning Wen, Ying Wen, Fanglei Sun, Jun Wang, and Yaodong Yang. Trust region policy optimisation in multi-agent reinforcement learning. arXiv preprint arXiv:2109.11251, 2021

  56. [64]

    Heterogeneous- agent reinforcement learning

    Yifan Zhong, Jakub Grudzien Kuba, Xidong Feng, Siyi Hu, Jiaming Ji, and Yaodong Yang. Heterogeneous- agent reinforcement learning. Journal of Machine Learning Research, 25(32):1–67, 2024

  57. [65]

    Hypermarl: Adaptive hypernetworks for multi-agent rl

    Kale-ab Abebe Tessera, Arrasy Rahman, Amos Storkey, and Stefano V Albrecht. Hypermarl: Adaptive hypernetworks for multi-agent rl. arXiv preprint arXiv:2412.04233, 2024

  58. [66]

    Qarray: A gpu-accelerated constant capacitance model simulator for large quantum dot arrays

    Barnaby van Straaten, Joseph Hickie, Lucas Schorling, Jonas Schuff, Federico Fedele, and Natalia Ares. Qarray: A gpu-accelerated constant capacitance model simulator for large quantum dot arrays. SciPost Physics Codebases, page 035, 2024

  59. [67]

    Codebase release 1.3 for qarray

    Barnaby van Straaten, Joseph Hickie, Lucas Schorling, Jonas Schuff, Federico Fedele, and Natalia Ares. Codebase release 1.3 for qarray. SciPost Physics Codebases, page 035, 2024

  60. [68]

    The complexity of de- centralized control of markov decision processes

    Daniel S Bernstein, Robert Givan, Neil Immerman, and Shlomo Zilberstein. The complexity of de- centralized control of markov decision processes. Mathematics of operations research, 27(4):819–840, 2002

  61. [69]

    Learning to utilize shaping rewards: A new approach of reward shaping

    Yujing Hu, Weixun Wang, Hangtian Jia, Yixiang Wang, Yingfeng Chen, Jianye Hao, Feng Wu, and Changjie Fan. Learning to utilize shaping rewards: A new approach of reward shaping. Advances in Neural Information Processing Systems, 33:15931–15941, 2020

  62. [70]

    Bayesian filtering: From kalman filters to particle filters, and beyond

    Zhe Chen et al. Bayesian filtering: From kalman filters to particle filters, and beyond. Statistics, 182(1):1– 69, 2003

  63. [71]

    Nonlinear bayesian estimation: From kalman filtering to a broader horizon.IEEE/CAA Journal of Automatica Sinica, 5(2):401– 417, 2018

    Huazhen Fang, Ning Tian, Yebin Wang, MengChu Zhou, and Mulugeta A Haile. Nonlinear bayesian estimation: From kalman filtering to a broader horizon.IEEE/CAA Journal of Automatica Sinica, 5(2):401– 417, 2018

  64. [72]

    Actor-critic algorithms

    Vijay Konda and John Tsitsiklis. Actor-critic algorithms. Advances in neural information processing systems, 12, 1999

  65. [73]

    Scaling laws for a multi-agent reinforcement learning model

    Oren Neumann and Claudius Gros. Scaling laws for a multi-agent reinforcement learning model. arXiv preprint arXiv:2210.00849, 2022

  66. [74]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

  67. [75]

    Survey on applications of multi-armed and contextual bandits

    Djallel Bouneffouf, Irina Rish, and Charu Aggarwal. Survey on applications of multi-armed and contextual bandits. In 2020 IEEE congress on evolutionary computation (CEC), pages 1–8. IEEE, 2020

  68. [76]

    Experience-driven networking: A deep reinforcement learning based approach

    Zhiyuan Xu, Jian Tang, Jingsong Meng, Weiyi Zhang, Yanzhi Wang, Chi Harold Liu, and Dejun Yang. Experience-driven networking: A deep reinforcement learning based approach. In IEEE INFOCOM 2018-IEEE conference on computer communications, pages 1871–1879. IEEE, 2018

  69. [77]

    A simplex method for function minimization

    John A Nelder and Roger Mead. A simplex method for function minimization. The Computer Journal, 7(4):308–313, 1965

  70. [78]

    Gaussian Processes for Machine Learning

    Carl Edward Rasmussen and Christopher K I Williams. Gaussian Processes for Machine Learning. MIT Press, 2006

  71. [79]

    Taking the human out of the loop: A review of bayesian optimization

    Bobak Shahriari, Kevin Swersky, Ziyu Wang, Ryan P Adams, and Nando de Freitas. Taking the human out of the loop: A review of bayesian optimization. Proceedings of the IEEE, 104(1):148–175, 2016. 14

  72. [80]

    Updating quasi-newton matrices with limited storage

    Jorge Nocedal. Updating quasi-newton matrices with limited storage. Mathematics of Computation, 35(151):773–782, 1980

  73. [81]

    Mastering diverse domains through world models, 2024

    Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy Lillicrap. Mastering diverse domains through world models, 2024

  74. [82]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778, 2016

  75. [83]

    Long short-term memory

    Sepp Hochreiter and J"urgen Schmidhuber. Long short-term memory. Neural Computation, 9(8):1735– 1780, 1997

  76. [84]

    Attention is all you need.Advances in neural information processing systems, 30, 2017

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.Advances in neural information processing systems, 30, 2017

  77. [85]

    Kaleidoscope: Learnable masks for heterogeneous multi-agent reinforcement learning

    Xinran Li, Ling Pan, and Jun Zhang. Kaleidoscope: Learnable masks for heterogeneous multi-agent reinforcement learning. Advances in Neural Information Processing Systems, 37:22081–22106, 2024

  78. [86]

    Quantum manipulation of two-electron spin states in isolated double quantum dots

    Benoit Bertrand, Hanno Flentje, Shintaro Takada, Michihisa Yamamoto, Seigo Tarucha, Arne Ludwig, Andreas D Wieck, Christopher Bäuerle, and Tristan Meunier. Quantum manipulation of two-electron spin states in isolated double quantum dots. Physical review letters, 115(9):096801, 2015

  79. [87]

    Concentration inequalities and model selection: Ecole d’Eté de Probabilités de Saint-Flour XXXIII-2003

    Pascal Massart. Concentration inequalities and model selection: Ecole d’Eté de Probabilités de Saint-Flour XXXIII-2003. Springer, 2007

  80. [88]

    Mitigating crosstalk errors for simultaneous single-qubit gates on a superconducting quantum processor.arXiv preprint arXiv:2603.11018, 2026

    Jaap J Wesdorp, Eric Hyyppä, Joona Andersson, Janos Adam, Rohit Beriwal, Ville Bergholm, Saga Dahl, Simone Diego Fasciati, Alejandro Gomez Friero, Zheming Gao, et al. Mitigating crosstalk errors for simultaneous single-qubit gates on a superconducting quantum processor.arXiv p...

  81. [89]

    Practical guide for building supercon- ducting quantum devices

    Yvonne Y Gao, M Adriaan Rol, Steven Touzard, and Chen Wang. Practical guide for building supercon- ducting quantum devices. PRX quantum, 2(4):040202, 2021

  82. [90]

    Entanglement and quantum error correction with superconducting qubits

    Matthew Reed. Entanglement and quantum error correction with superconducting qubits. Lulu. com, 2013

  83. [91]

    Charge-insensitive qubit design derived from the cooper pair box

    Jens Koch, Terri M Yu, Jay Gambetta, Andrew A Houck, David I Schuster, Johannes Majer, Alexandre Blais, Michel H Devoret, Steven M Girvin, and Robert J Schoelkopf. Charge-insensitive qubit design derived from the cooper pair box. Physical Review A—Atomic, Molecular, and Optica...

  84. [92]

    world model

    Pierre Guilmin, Adrien Bocquet, Élie Genois, Daniel Weiss, and Ronan Gautier. Dynamiqs: an open-source python library for gpu-accelerated and differentiable simulation of quantum systems. 2025. 15 A Proofs on Virtualization and Filters Notation.Forz∈R d, define∥z∥ 2 := √ z⊤z. ...

Pith tools

Reviewed July 13, 2026 · model on record in the stance chip above.