{"id":"c2e2636b-a8c3-48d9-a043-05228b742a6c","arxiv_id":"2412.04053","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"An RL-trained pulse reads superconducting qubits on IBM hardware with assignment error around 4.6e-3 and completes readout plus reset in 470 to 675 ns, versus 1400 ns for the default protocol.","lead":"Researchers trained a reinforcement learning agent in a simulated environment to design microwave pulses for reading out superconducting qubits. On IBM hardware, the learned pulses reached accuracy comparable to default settings while finishing measurement and resonator reset up to three times faster.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Equation (6)'s fidelity model multiplies the SNR by the decay survival factor inside the erf, so assignment fidelity approaches 1 even when qubit decay is large; the RL reward thus optimizes a surrogate that is not a valid probability model.","rationale":"The paper's central claim has two parts: an empirical speedup demonstrated on IBM hardware, and an RL-based discovery/optimization claim. The hardware data on Kyoto and Brisbane directly support the speedup at comparable fidelity, and that evidence is real. The weakest link is the surrogate that the RL agent optimizes. The reader identified transferability of the Langevin model as the weakest assumption; I found a more specific internal defect inside that surrogate: Eq. (6) is not a valid probability model. Multiplying the SNR by the survival probability inside an erf means that as separation grows, predicted fidelity saturates at 1 regardless of decay, whereas a real mixture of decayed and undecayed trajectories cannot beat the (1+Fq)/2 ceiling in the high-SNR limit. The formula also mishandles initialization fidelity at t=0 and has an identifiability degeneracy between F0 and λ. Since the reward, the robustness landscapes, and the synthetic CLEAR comparison all flow through Eq. (6), the optimality and generality claims are not yet established. The measured speedup is unaffected by this objection, so the verdict stays conditional rather than reject. The concrete next step is to retrain with a corrected fidelity model and check whether the same pulse emerges; if it does, the concern is moot, and if not, the RL result is an artifact of the reward function.","tokens_in":14985,"tokens_out":11746,"duration_ms":130800,"concrete_test":"Analytically evaluate Eq. (6) at S→∞ with Fq=0.5: it predicts F→1, while a Gaussian-mixture model with explicit decay predicts F≤0.75. Then retrain the PPO agent on the Kyoto and Brisbane parameters with a corrected fidelity objective—either a Lindblad master equation or a mixture-state assignment model that treats decay as a probabilistic switch between the |g⟩ and |e⟩ trajectories—and deploy the resulting waveform on the same qubits. If the optimal waveform or the max-fidelity time (264/542 ns) changes materially, the published RL pulse is an artifact of Eq. (6); if the same waveform and hardware timings recur, the concern is non-load-bearing for the empirical speedup.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Appendix B Eq. (6) defines F(t)=0.5[1+erf(F0 × λS(t) × Fq(t))], with Fq=exp(-γ0t - γP∫N) the qubit survival probability. This is not a valid assignment-fidelity model. In the large-separation limit S→∞, the erf argument diverges for any fixed Fq>0, so F→1; but if a fraction 1-Fq of excited-state preparations decay into the ground-state readout channel during the measurement, the high-SNR assignment fidelity is capped at roughly (1+Fq)/2, independent of S. Eq. (6) also gives F=0.5 at t=0 for any F0, although a nonzero initialization fidelity should set a better-than-random floor before signal is acquired, and F0 and λ appear only as a product, so the Appendix B fit is underdetermined. Because the reward in Eq. (3) maximizes max_t F(t), the readout time and waveform found by PPO may be driven by this artifact rather than by the true measurement physics. The live IBM results are real evidence and partly mitigate the concern because the claimed max-fidelity times (264 ns Kyoto, 542 ns Brisbane) occur where Fq is close to 1; however, the simulated robustness landscapes in Fig. 3(d,e) and the CLEAR comparison in Table IV are generated with Eq. (6), so those generality claims inherit the model error.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a reinforcement learning (RL) framework, based on PPO, to optimize the full dispersive readout pulse for superconducting qubits. The agent is trained against a quasiclassical Langevin simulator whose parameters are calibrated to IBM devices, and the resulting waveforms are executed on IBM Kyoto and Brisbane. The authors report assignment errors of (4.6 ± 0.4) × 10^-3 on Kyoto and (7.0 ± 1.0) × 10^-3 on Brisbane, with total readout-and-reset durations of 470 ns and 675 ns, respectively, compared with 1400 ns for the default configuration. They also identify a four-segment analytical form called A4R and claim robustness against ±10% parameter drifts, supported by simulation and by a comparison with the CLEAR protocol.","tokens_in":15298,"tokens_out":4563,"duration_ms":43130,"significance":"If the central speedup claim holds, the work is practically valuable: reducing the measurement-and-reset time while preserving assignment fidelity is directly relevant for quantum error correction and mid-circuit measurement. The paper's strengths are the live hardware measurements with error bars on two IBM devices, the openly stated code and parameters, the short training time, and the reduction of the learned waveform to a simple calibration-ready analytical form. However, the fidelity improvement over the default protocol is not statistically significant in the reported data, and the simulated robustness and CLEAR-comparison results rely on a fidelity model that is not a valid probability model. The claimed generality and state-of-the-art fidelity therefore need additional support before the paper can be accepted.","major_comments":[{"comment":"The fidelity model F(t) = 0.5[1 + erf(F0 × λS(t) × Fq(t))] is not a valid assignment-fidelity probability. Since the erf argument grows without bound as S(t) → ∞, F approaches 1 even for a fixed, non-unit survival factor Fq; physically, a fraction 1 - Fq of excited-state preparations that decay into the ground-state channel should impose a high-SNR assignment error of at least (1 - Fq)/2, independent of S. The model also gives F(0) = 0.5 for any initialization fidelity, and F0 and λ appear only as a product, so the Appendix B fit is underdetermined. Because the reward in Eq. (3) maximizes max_t F(t), the PPO agent may optimize against this artifact rather than against the true measurement physics. The live hardware results at tFmax = 264 ns (Kyoto) and 542 ns (Brisbane) partly mitigate the concern, as Fq is close to unity there, but the simulated robustness landscapes in Fig. 3(d,e) and the CLEAR comparison in Table IV are generated using Eq. (6) and therefore inherit the error. I request a corrected fidelity model—for example, an explicit mixture of ground and excited readout distributions with decay—or a validation of the surrogate against a full master-equation treatment, and a rerun of the affected simulation-based claims with that corrected model.","section":"Appendix B, Eq. (6)"},{"comment":"The claim that the RL waveform achieves 'equal or slightly superior fidelities' is not supported by the quoted uncertainties. On Kyoto, (4.6 ± 0.4) × 10^-3 versus (5.8 ± 0.9) × 10^-3 is within roughly 1.2 combined standard deviations, and on Brisbane the RL value (7.0 ± 1.0) × 10^-3 is identical to the default value to the reported precision. The data therefore establish only that the RL pulse attains comparable fidelity while being substantially faster. The abstract and Section IV.B should be revised to state 'comparable fidelity' rather than 'state-of-the-art performance' or 'slightly superior fidelities,' unless a significance test is provided.","section":"Section IV.B and Table II"},{"comment":"The training environment is a quasiclassical surrogate whose free parameters—λ, F0, γ0, γP, the reward coefficients ki, and the reset penalty factor m—are fitted to the same devices that are used for hardware validation. This is not circular in the sense of using the final hardware result to define the objective, but it creates a risk that the policy is optimized to idiosyncrasies of the self-calibrated model rather than to robust measurement physics. The paper would be strengthened by an explicit transfer experiment to a device with parameters outside the fitted range, or by an analysis of the sensitivity of the measured performance to the fitted parameters. As written, the assertion that the method is 'readily applicable to generic superconducting devices' rests on only two test points and on a surrogate whose validity is questioned by the issues in Eq. (6).","section":"Section III and Appendix B (training environment)"}],"minor_comments":[{"comment":"The phrase 'almost three times faster' is accurate for Kyoto (1400 ns to 470 ns) but for Brisbane the speedup is about 2.1×; please specify per-device factors or state 'two- to threefold.'","section":"Abstract and Section IV.B"},{"comment":"The labels 'RL wfA4R Square' and 'RL wfA4R Square' are ambiguous; please use distinct labels for the learned waveform, the analytical A4R pulse, and the default square pulse, and apply the same labels consistently across all panels.","section":"Fig. 3(a) caption"},{"comment":"The derivation of τ1 is described only as using 'first-order approximations in χt'; please write out the explicit expression that leads to Eq. (8) so the reader can verify the approximation.","section":"Appendix D, after Eq. (8)"},{"comment":"The column heading 'RL/A4R' is undefined; state that these are simulated values for the RL/A4R waveform, and clarify whether the same fidelity metric and integration weights are used as in the hardware measurements.","section":"Table IV caption"},{"comment":"Reference [24] states that source code and parameters are provided, but the availability of the exact trained network weights and the calibration script for A4R is not explicitly confirmed; please state what is included in the repository.","section":"GitHub reproducibility"}],"recommendation":"major_revision","confidential_remarks":"The speedup claim is plausible and the hardware measurements are a real asset, but the fidelity-model flaw in Eq. (6) undermines the simulation-based robustness and CLEAR-comparison claims, and the fidelity improvement over default is not statistically significant. If the authors can replace Eq. (6) with a valid model and rerun the affected simulations, or restrict the claims to the measured speedup with comparable fidelity, the paper may become acceptable. The use of the same device data to fit the surrogate and validate the policy is a further risk that should be addressed in revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Main thing to know: this paper has a real, hardware-validated result. The RL-discovered pulse shortens total readout+reset from 1400 ns to 470–675 ns on IBM Kyoto and Brisbane, with assignment errors statistically indistinguishable from the default. That speedup is the story, and it is solid.\n\nWhat is actually new: prior RL/ML readout work (Refs. 27 and 28) stayed in theses or simulation. Here they train in a Langevin surrogate with device-calibrated parameters, deploy on live cloud hardware, and demonstrate the speedup. The A4R analytical pulse, with unequal reset segments and a simple calibration recipe, is a practical extension of CLEAR. Shipping code and data on GitHub is also a point in their favor.\n\nSoft spots, in proportion. First and most important: Eq. (6) in Appendix B is not a valid assignment-fidelity model. Multiplying the SNR by the qubit survival factor inside the erf means the fidelity goes to 1 as separation grows, even when many excited-state preparations have decayed. The correct cap is roughly (1+Fq)/2. This flaw does not sink the central hardware claim—the measured max-fidelity times (264 ns Kyoto, 542 ns Brisbane) occur where Fq is close to 1—but it does undermine the simulated robustness landscapes and the CLEAR comparison in Table IV, which are generated with Eq. (6). A referee should ask the authors to fix the model or explicitly restrict their claims to the short-time regime. Second, the fidelity improvement over default is within shot noise on both devices; the abstract's \"state-of-the-art performance\" is not supported. They should say comparable fidelity, faster readout. Third, the CLEAR comparison in Appendix E truncates CLEAR's readout segment at max-fidelity time, which is not the standard CLEAR protocol. That makes the 2.5x speedup vs. CLEAR partly apples-to-oranges.\n\nOverall, the engineering claim holds up from live measurements; the theoretical model needs revision and the framing needs tempering. This paper deserves peer review—the speedup is important and the authors are transparent about their methods. A serious referee should focus on the fidelity model, the CLEAR comparison, and the fidelity wording.","headline":"A genuinely useful hardware result—RL-cut readout/reset times by 2-3x on real IBM devices—but the fidelity model in Eq. (6) is not a valid probability model and the 'state-of-the-art' fidelity claim overreaches.","tokens_in":15891,"tokens_out":2553,"would_cite":true,"duration_ms":25894,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A reinforcement-learning agent discovers readout waveforms that complete qubit measurement and resonator reset in under 700 ns, about three times faster than the default configuration, with assignment error as low as 4.6e-3.","keywords":["reinforcement learning","qubit readout","superconducting qubits","dispersive readout","resonator reset","PPO","waveform optimization","active four-tone readout"],"falsifier":"Run the optimized pulse on a third device whose $\\kappa/\\chi$ lies outside the two tested values, or intentionally detune $\\kappa$ or $\\chi$ by more than 10%, and compare the measured single-shot assignment error and the resonator photon population after 470-675 ns against the paper's simulated predictions; a mismatch beyond shot noise would indicate that the surrogate environment, not the reinforcement-learning procedure, is the source of the reported speedup. A second check is to replace Eq. (1) with a full master-equation simulation that includes measurement-induced dephasing and ionization, retrain the agent, and see whether the discovered waveform and its assignment error change.","tokens_in":14740,"feed_emoji":"⚛️","tokens_out":7204,"duration_ms":65810,"temperature":0.7,"pith_summary":"Readout is often the slowest and noisiest step in superconducting quantum computation, and this paper asks whether a machine-learning agent can design the entire readout waveform rather than tuning individual segments. The authors train a model-free reinforcement-learning agent in a quasiclassical Langevin simulator whose parameters are measured from two cloud quantum processors, then transfer the learned pulse to real hardware. They report assignment errors as low as $(4.6 \\pm 0.4)\\times10^{-3}$ with total readout-plus-reset times of 470 ns and 675 ns on the two test devices, roughly three times faster than the devices' default 1400 ns configuration at comparable fidelity. The learned waveforms are stable under $\\pm 10\\%$ drifts in $\\kappa$ and $\\chi$ and collapse onto a simple analytical four-tone form, A4R, that can be calibrated with standard measurements. If these transfers hold, the result matters because faster, high-fidelity readout directly accelerates error correction and mid-circuit measurement, which otherwise idle a quantum processor.","feed_headline":"RL-designed pulse trims qubit readout to 470 ns","feed_subtitle":"On superconducting hardware the learned waveform holds ~99.5% fidelity while cutting readout-plus-reset time nearly threefold.","key_machinery":"The load-bearing machinery is the PPO agent interacting with a quasiclassical Langevin environment. The environment evolves two coherent amplitudes $\\alpha_g(t)$ and $\\alpha_e(t)$ under $\\dot{\\alpha}_{g/e}(t) = -(\\kappa/2 \\mp i\\chi)\\alpha_{g/e}(t) - iA(t)$, converts their separation into a time-dependent assignment fidelity through a Gaussian-noise model with a fitted scale, initialization fidelity, and photon-number-dependent qubit decay rates, and returns a reward that penalizes slow reset, rough or nonzero terminal amplitudes, and photon populations above the default readout's photon number. The discovered structure, A4R, is the named central output: four segments, namely ring-up, steady-state readout, photon depletion, and kickback, each with an amplitude and a duration, with the ring-up and depletion amplitudes set to the hardware maximum and their durations given by closed-form expressions in $\\kappa$.","core_discovery":"The central claim is that a reward function combining assignment fidelity, total reset time, pulse smoothness, and a photon-number cap is enough for a reinforcement-learning agent to rediscover a near-optimal readout protocol: a high-amplitude ring-up that drives the resonator toward steady state quickly, a short calibrated readout segment that reaches maximum separation before steady state, and a two-tone active reset that empties the resonator to below 0.05 photons. The paper argues that the optimal measurement time is not the steady-state time, because qubit decay and measurement-induced transitions accumulate during longer pulses, so the best fidelity occurs at an intermediate time, namely 264 ns on one device. It further claims that the learned waveform can be compressed into eight parameters, four segment amplitudes and four durations, and that this A4R pulse matches the reinforcement-learning pulse's assignment error and speed on hardware while remaining stable under realistic parameter drifts.","pith_inferences":["Beyond the paper, the same training loop could be pointed at nonlinear readout regimes, such as large self-Kerr or near-ionization photon numbers, where no analytical four-tone ansatz exists, turning reinforcement learning from a rediscovery tool into a discovery tool.","Because the reset durations in A4R are essentially set by $\\kappa$, the protocol could be ported to a new device with only a $\\kappa$ measurement, even when $\\chi$ is poorly known.","A natural test would benchmark A4R inside a dynamic quantum circuit with many mid-circuit measurements, predicting overall runtime savings that scale roughly with the number of measurements.","The surrogate's constant-noise, Gaussian-state fidelity formula has not been validated outside the tested parameter window, so the robustness claim should not be extrapolated to much higher photon numbers or vastly different $\\kappa/\\chi$ without revalidation."],"forward_implications":["If the central claim is correct, readout need not wait for resonator steady state: reaching maximum signal-to-noise at an intermediate time shortens the measurement without sacrificing assignment fidelity.","Active two-tone reset can clear tens of photons to below 0.05 in hundreds of nanoseconds, so repeated measurements can be chained with far less idle dead time.","The A4R form means the optimized behavior can be deployed without a neural network on any comparable device, using only a few calibration scans.","The demonstrated stability under $\\pm 10\\%$ drifts in $\\kappa$ and $\\chi$ implies the pulse does not need continuous re-optimization over a typical device-drift timescale.","The same reward structure should apply to other dispersive-readout hardware, because the training environment only needs $\\kappa$, $\\chi$, photon number, and decay rates."],"supporting_citations":[{"why":"Supplies the two cloud quantum devices and the default 1400-ns square-pulse readout baseline that the optimized waveforms are measured against.","marker":"[1]"},{"why":"Defines the CLEAR active ring-up and reset sequence that serves as the comparison baseline and whose two-equal-tone reset A4R generalizes.","marker":"[18]"},{"why":"Provides the coherent Langevin equation that is the core dynamics of the training environment.","marker":"[25]"},{"why":"Supplies the proximal policy optimization algorithm used to train the readout-pulse agent.","marker":"[34]"},{"why":"Supplies the photon-number-dependent qubit-decay model folded into the fidelity formula used for rewards.","marker":"[47]"},{"why":"Provides the simplex optimizer used to calibrate A4R parameters and to reproduce CLEAR for comparison.","marker":"[29]"}],"fun_headline_variants":["RL finds fast qubit readout with 99.5% fidelity","Reinforcement learning slashes qubit readout time 3x","Qubit readout error cut to 0.46% via RL","AI designs 8-parameter pulse for fast qubit readout","RL-optimized readout: 3x faster, 99.5% accurate"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole optimization rests on the fitted quasiclassical surrogate, which uses two independent coherent trajectories, a Gaussian resonator state, and a constant-noise fidelity formula calibrated on the same two devices, transferring faithfully to real hardware; if that surrogate misses measurement-induced effects outside the tested parameter range, the claimed speed and stability may not generalize.","fun_headline_variants_meta":{"raw":{"variants":["RL finds fast qubit readout with 99.5% fidelity","Reinforcement learning slashes qubit readout time 3x","Qubit readout error cut to 0.46% via RL","AI designs 8-parameter pulse for fast qubit readout","RL-optimized readout: 3x faster, 99.5% accurate"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000386,"raw_usage":{"total_tokens":2029,"prompt_tokens":922,"completion_tokens":1107,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":538,"completion_tokens_details":{"reasoning_tokens":1007}},"tokens_in":538,"tokens_out":1107,"duration_ms":8495,"temperature":1.0,"reasoning_tokens":1007,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T21:49:10.859619+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the optimized pulse on a third device whose $\\kappa/\\chi$ lies outside the two tested values, or intentionally detune $\\kappa$ or $\\chi$ by more than 10%, and compare the measured single-shot assignment error and the resonator photon population after 470-675 ns against the paper's simulated predictions; a mismatch beyond shot noise would indicate that the surrogate environment, not the reinforcement-learning procedure, is the source of the reported speedup. A second check is to replace Eq. (1) with a full master-equation simulation that includes measurement-induced dephasing and ionization, retrain the agent, and see whether the discovered waveform and its assignment error change.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the simplex optimizer used to calibrate A4R parameters and to reproduce CLEAR for comparison."}],"review_version":1}