Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Quantum reinforcement learning in dynamic environments

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A hybrid quantum-classical agent that adds a dissipation-based forgetting mechanism to Projective Simulation learns a changed Gridworld reward path faster than a classical agent, reaching 73.2% success versus 61.6% after the switch.

desk verdict A sound dissipation extension of the hybrid QRL agent with one real overclaim: the 'higher average success probability' does not hold at gamma=0.01. read the letter →

arxiv 2507.01691 v1 pith:ZWHU5ZM6 submitted 2025-07-02 quant-ph

Oliver Sefrin , Manuel Radons , Lars Simon , Sabine Wölk This is my paper
classification quant-ph MSC 68Q1268T05 PACS 03.67.Ac03.67.Lx
keywords quantumreinforcementlearninghybridagentamplitudeamplificationcontinualProjectiveSimulationdissipationGridworlddynamicrewardenvironment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether a quantum-accelerated reinforcement-learning agent, whose known speedup was proven for stationary environments, keeps its advantage when the reward structure changes during training. The authors add a dissipation (forgetting) mechanism to the Projective Simulation policy inside the hybrid agent and test it in a Gridworld where the rewarded path moves within each episode and, in the harder scenario, is later replaced by a disjoint path. In simulations with identical hyperparameters, the hybrid agent finds the first reward roughly three times faster than the classical agent, and after the reward-path switch it relearns faster across every tested dissipation setting, reaching an average success probability of 73.2% versus 61.6% for the classical agent at $\gamma=0.05$. The point is not just that one maze was solved: if the mechanism transfers, the quadratic sample-complexity advantage of amplitude amplification would also apply to relearning, which is the bottleneck when a trained agent meets a changed environment.

What carries the argument

The engine is the Grover operator $G = D O_{\mathrm{env}}$ built from the environment's phase-kickback oracle $O_{\mathrm{env}}$ and the diffusion $D = I - 2|\psi\rangle\langle\psi|$, where $|\psi\rangle = \sum_{\vec{a}} \sqrt{\hat{\pi}(\vec{a})}\,|\vec{a}\rangle_A|\vec{s}_{\mathrm{init}}\rangle_S|-\rangle_R$ is a weighted superposition of action sequences; iterating $G$ amplifies rewarded sequences, giving a success probability $p_{\mathrm{AA}}(Q,k)=\sin^2((2k+1)\arcsin\sqrt{Q})$. On the classical side, Projective Simulation updates h-values $h(s,a)$ with a dissipation term $-\gamma(h(s,a)-1)$, and the paper proves the batched equivalent $h^{(t+N)}-1 = (h^{(t)}-1)(1-\gamma)^N + r^{(t+N-1)}$, so the hybrid agent, which updates only every $2k+1$ episodes, forgets at the same rate as an agent that updates every episode. A purging step removes previously rewarded sequences from $\mathcal{R}_{\mathrm{found}}$ when they are sampled without reward, keeping the probability estimate $Q_{\mathrm{est}}$ close to the true success probability after a reward-path switch.

What would settle it

Replace exact preparation of $|\psi\rangle$ with a preparation that has a known fidelity error $\delta$ (for example, a truncated or variational circuit) and measure the hybrid agent's post-switch success probability as $\delta$ grows; if the advantage over the classical baseline disappears for small $\delta$, the result is specific to exact enumerative preparation rather than to the hybrid learning mechanism.

Watch

Extended reading notes

Core claim

The paper's central claim is that the hybrid agent, extended with a dissipation term and an adaptive probability estimate, can outperform a purely classical Projective Simulation agent with the same hyperparameters in a deterministic, strictly episodic environment with a time-dependent reward. In the fixed-reward-path layout, the hybrid agent reaches the first reward in 17.9 episodes on average versus 58.4 for the classical agent, and reaches a 20% success probability in 33.8 versus 84.6 episodes. After the reward path is exchanged for a second, disjoint path, the hybrid agent eventually overtakes the classical agent in all tested configurations; at dissipation $\gamma=0.05$ its average post-switch success probability is 73.2% versus 61.6%, and under a 4-out-of-5 competence criterion its lowest total training time is 112.7 episodes versus the classical agent's best 187.2 episodes.

Load-bearing premise

The central claim depends on being able to prepare the amplitude-amplification state $|\psi\rangle$, a weighted superposition of every possible action sequence, exactly; on the paper's toy grids this is classically feasible, but on any problem large enough for the quadratic speedup to matter, the paper only points to an unproven idea for doing it efficiently.

Editorial extensions

If this is right

  • In the fixed-reward-path regime, the hybrid agent's early-stage advantage persists: it reaches a 20% success probability in about 33.8 episodes versus 84.6 for the classical agent, consistent with the known quadratic speedup in sample complexity.
  • In the changing-reward-path regime, the hybrid agent completes the 4-out-of-5 competence criterion with fewer total episodes than the classical agent for every tested dissipation value, and its lowest total is 112.7 episodes at $\gamma=0.05$ versus the classical agent's best 187.2 episodes at $\gamma=0.02$.
  • Under a fixed training budget ($N_1=100$, $N_2=300$), the hybrid agent with $\gamma=0.05$ achieves the highest average post-switch success probability, 73.2%, compared with 61.6% for the classical agent at the same dissipation.
  • The adaptive estimate $Q_{\mathrm{est}}$ tracks the true success probability after the reward switch, overshooting only for the first few dozen episodes and then acting as a lower bound, which keeps amplitude amplification efficient.
  • Because the two reward paths are disjoint, the tested switch is a worst case; if reward sets partially overlapped, the hybrid agent's better initial learning would contribute positively rather than negatively to the second phase.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not test approximate preparation of $|\psi\rangle$; exact preparation requires enumerating all $|A|^T$ action sequences, so the practical range of the claimed advantage depends on an efficient preparation subroutine, which the paper only gestures at in a footnote.
  • The paper notes qualitatively that overlapping reward paths would further favor the hybrid agent; a quantitative test varying the overlap fraction from 0 to 1 would turn that remark into a concrete prediction.
  • The batched dissipation update is derived for Projective Simulation's h-values, and the same equivalence argument should apply to any additive policy update; porting the correction to Q-learning or policy-gradient updates and rerunning the changing-path scenario would check whether the forgetting mechanism is algorithm-independent.
  • The purging rule removes an obsolete sequence only if it is sampled after the switch; in larger environments where old rewarding sequences are sampled rarely, $Q_{\mathrm{est}}$ could stay inflated, so a test with sparse sampling of the old path would show whether the agent needs an explicit change detector.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper studies a hybrid quantum-classical reinforcement learning agent, based on amplitude amplification, in non-stationary environments with time-dependent rewards. The authors add a dissipation mechanism to the policy update of the hybrid agent, adapt the lower-bound estimate Qmin to changing reward paths by purging no-longer-rewarded action sequences, and compare the resulting agent against a classical Projective Simulation agent with identical hyperparameters on two Gridworld layouts. In the fixed reward path scenario, the hybrid agent learns faster, consistent with earlier results. In the changing reward path scenario, for dissipation values gamma >= 0.02, the hybrid agent achieves higher average success probabilities after the switch and lower total learning times, while for gamma = 0.01 the data show the hybrid agent is worse than the classical baseline. The paper includes an analytic equivalence proof for the adapted dissipation update and several appendices describing the Projective Simulation background and the estimation/purging procedures.

Significance. If the central empirical claim is accepted after proper qualification, the paper provides a useful first step toward extending the hybrid quantum-accessible RL agent to continual or non-stationary settings. The dissipation-equivalence proof in Appendix B.1 is a correct and compact analytic contribution, and the empirical comparison is a clean, reproducible-looking benchmark with the same hyperparameters for both agents. The paper also honestly discloses the exponential classical cost of preparing the amplitude-amplification state and of the fallback Qest procedure. However, the headline claim as stated in the abstract and Section 6 is contradicted by the paper's own Table 4 at gamma=0.01, so the contribution is currently overclaimed; the support is solid for gamma >= 0.02, but the 'eventually overtakes' statement needs quantitative backing in terms of crossing times or endpoint averages.

major comments (3)
  1. [Abstract; Section 6; Table 4 (Appendix A)] The abstract's claim that the modified hybrid agent 'lead[s] to a higher average success probability compared to its classical counterpart' is contradicted by the paper's own Table 4 for gamma=0.01 in scenario B.2: the hybrid average after the switch is 5.5% (SE 0.1%) versus 13.5% (SE 0.2%) for classical, and the total average is 21.6% (SE 0.2%) versus 22.7% (SE 0.2%). Section 6's statement that the hybrid agent 'eventually overtakes its classical counterpart ... in all tested configurations' is therefore false as written. The claim should be restricted to gamma >= 0.02, and the authors should report crossing-time statistics or final-window averages with confidence intervals to substantiate 'eventually overtakes' for the configurations where the hybrid does improve.
  2. [Section 2.2, footnote 2; Section 7; Appendix C.1] The practical relevance of the claimed speedup is limited by the cost of preparing the state |psi> in Eq. (9), which requires explicit computation of pi-hat(a) for all |A|^T action sequences. The authors acknowledge this in footnote 2 but only cite an unproven possibility, 'quantum policies based on variational quantum circuits', and Appendix C.1's fallback for Qest explicitly requires computing all action-sequence probabilities. In this paper the computations are feasible because |A|^T is small (5^7), but Section 7's conjecture that the results 'generalize well to a wider range of problem cases' and the framing of the speedup as persisting in dynamic environments go beyond what is demonstrated. Please state the exponential-preparation limitation in the abstract or conclusion and restrict the generalization claim to the demonstrated regime.
  3. [Section 5, Scenario B.2, Figure 5] For the headline configuration gamma=0.05, the claim that the hybrid agent surpasses the classical agent 'after roughly 60 episodes' on the new reward path is not supported by any statistical summary in the text: no crossing-time distribution, no endpoint average, and no test that the separation exceeds the reported 95% error bands. Please add a quantitative comparison of the final phase (e.g., the final 50 episodes) with standard errors or a paired test, and report the fraction of runs in which the hybrid crosses and stays above the classical curve.
minor comments (5)
  1. [Section 4.2; Table 1] There is an inconsistency in the stated episode length for layout B: Section 4.2 says the initial reward path 'goes down and then right, with an episode length of nine', while Table 1 reports an episode length of 7 for layout B. Please clarify which value is correct.
  2. [Appendix C, Algorithm 3] Algorithm 3 has no explicit loop header such as 'while not rewarded do' that encloses lines 5-28, even though lines 30 and 33 say 'repeat from step 4' and 'repeat from step 5'. Adding the missing loop structure would make the pseudocode unambiguous.
  3. [Section 6] The phrase 'in all tested configurations' should be removed or explicitly qualified, since it is contradicted by the gamma=0.01 row of Table 4; this point is already raised as a major comment but should also be fixed in the prose.
  4. [Figure 6 caption] The caption 'Success Probability Estimation True Value' is unclear; it should be reworded to indicate that the figure compares the estimated success probability Qest with the true success probability Q.
  5. [Abstract] The abstract should qualify the 'higher average success probability' claim by stating the dissipation range or by adding a phrase such as 'for sufficiently large dissipation values', to avoid contradicting the paper's own data.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the hybrid-vs-classical advantage is established by self-contained simulation against a classical baseline with identical hyperparameters; the self-cited speedup theorem is supported by an independent experiment.

full rationale

The paper's derivation chain is self-contained and its central claims are empirical. The only analytic result in the paper, the adjusted dissipation update h(t+N) = (h(t)-1)(1-gamma)^N + 1 + r (Appendix B.1), is proven by induction in the text and equates the hybrid's batched forgetting with the classical agent's per-episode forgetting; it is a design equivalence, not an assumed prediction. The probability-bound estimator Q_est (Eq. C9, Appendix C) is constructed by summing policy probabilities of found reward paths, which is a lower bound on the true success probability by subset inclusion, and after the reward-path switch the paper explicitly concedes that Q_est is no longer a lower bound (Appendix C.1), so no fitted quantity is relabeled as a prediction. The performance comparison in Section 5 and Table 4 pits the modified hybrid against a classical PS agent with identical hyperparameters (beta = 1, eta = 0.05, same gamma) in the same Gridworld layouts, and the reported success probabilities are computed by full enumeration of the 5^7 action sequences, so the measured quantities are not derived from the agent's own estimates. The cited speedup theory (Hamann and Woelk 2022, co-authored by Woelk) is a parameter-free proven bound with an independent experimental demonstration (Saggio et al. 2021), so those self-citations are real evidence rather than load-bearing circular premises. The one substantive flaw is an overgeneralization, not a circularity: the abstract's 'higher average success probability' and Section 6's 'eventually overtakes ... in all tested configurations' are contradicted by the paper's own Table 4 at gamma = 0.01, where the hybrid's post-switch average (5.5%, SE 0.1%) lies below the classical agent's (13.5%, SE 0.2%) and the total average is also lower (21.6% vs 22.7%). That inconsistency is a correctness or reporting issue and does not constitute a circular reduction of output to input.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the DSE environment model, the oracle construction from prior work, and the exact enumeration of action sequences in the toy problem. The dissipation compensation is proven, not postulated. No new physical entities are introduced; the Qest purging is an algorithmic heuristic, not an entity.

assumptions (4)
  • domain assumption DSE environment assumption: deterministic, strictly episodic, binary first-visit reward
    The quantum speedup and oracle construction (Eqs. 7-8) apply only to DSE environments; Section 2.2 and Section 3.1 restrict the Gridworld to a fixed episode length and deterministic moves.
  • standard math Existence of a phase-kickback oracle O_env using one or two queries of U_env
    Taken from Dunjko et al. (2016), cited in Section 2.2; the paper relies on this to define the Grover operator G.
  • ad hoc to paper Exact classical computation of all action-sequence probabilities to prepare |psi> and to evaluate success probabilities
    Footnote 2 and Section 4.2: the simulations enumerate 5^7 sequences. The paper acknowledges that efficient preparation is an open issue for larger problems.
  • domain assumption Reward paths in layout B have fully disjoint rewarded action sequences
    Section 4.2 states the sets are fully disjoint, which ensures the worst-case scenario for relearning and simplifies the purging heuristic.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Quantum reinforcement learning in dynamic environments." pith.science (2026). https://pith.science/paper/ZWHU5ZM6

@misc{pith2026250701691,
  author       = {Pith},
  title        = {Pith review of: Quantum reinforcement learning in dynamic environments},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZWHU5ZM6}},
  note         = {Machine review of arXiv:2507.01691}
}
read the original abstract

Combining quantum computing techniques in the form of amplitude amplification with classical reinforcement learning has led to the so-called "hybrid agent for quantum-accessible reinforcement learning", which achieves a quadratic speedup in sample complexity for certain learning problems. So far, this hybrid agent has only been applied to stationary learning problems, that is, learning problems without any time dependency within components of the Markov decision process. In this work, we investigate the applicability of the hybrid agent to dynamic RL environments. To this end, we enhance the hybrid agent by introducing a dissipation mechanism and, with the resulting learning agent, perform an empirical comparison with a classical RL agent in an RL environment with a time-dependent reward function. Our findings suggest that the modified hybrid agent can adapt its behavior to changes in the environment quickly, leading to a higher average success probability compared to its classical counterpart.

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Q-SpiRL: Quantum Spiking Reinforcement Learning for Adaptive Robot Navigation

    cs.RO 2026-05 unverdicted novelty 5.0 of 10

    QSNN agent in Q-SpiRL framework achieves up to 99% success rate with efficient paths in 20x20 to 40x40 grid worlds with static and dynamic obstacles, outperforming tabular Q-learning, MLP, SNN, and QMLP baselines unde...

Reference graph

Works this paper leans on

61 extracted references · 25 canonical work pages · cited by 1 Pith paper

  1. [1]

    , Barreto , A

    bchapter Abel , D. , Barreto , A. , Van Roy , B. , Precup , D. , Hasselt , H.P. , Singh , S. : A definition of continual reinforcement learning . In: Advances in Neural Information Processing Systems , vol. 36 ( 2023 ) bchapter

    work page 2023

  2. [2]

    , Brassard , G

    barticle Boyer , M. , Brassard , G. , H yer , P. , Tapp , A. : Tight bounds on quantum searching . Fortschr. Phys. 46 ( 4-5 ), 493 -- 505 ( 1998 ) 10.1002/(SICI)1521-3978(199806)46:4/5<493::AID-PROP493>3.0.CO;2-P barticle

    doi:10.1002/(sici)1521-3978(199806)46:4/5 1998

  3. [3]

    , H yer , P

    barticle Brassard , G. , H yer , P. , Mosca , M. , Tapp , A. : Quantum amplitude amplification and estimation . Contemp. Math. 305 , 53 -- 74 ( 2002 ) 10.1090/conm/305/05215 barticle

    doi:10.1090/conm/305/05215 2002

  4. [4]

    , Cuevas , G

    barticle Briegel , H.J. , Cuevas , G. : Projective simulation for artificial intelligence . Sci. Rep. 2 ( 1 ), 400 ( 2012 ) 10.1038/srep00400 barticle

    work page doi:10.1038/srep00400 2012

  5. [5]

    , Arrasmith , A

    barticle Cerezo , M. , Arrasmith , A. , Babbush , R. , Benjamin , S.C. , Endo , S. , Fujii , K. , McClean , J.R. , Mitarai , K. , Yuan , X. , Cincio , L. , Coles , P.J. : Variational quantum algorithms . Nat. Rev. Phys. 3 ( 9 ), 625 -- 644 ( 2021 ) 10.1038/s42254-021-00348-9 barticle

    doi:10.1038/s42254-021-00348-9 2021

  6. [6]

    , Larocca , M

    botherref Cerezo , M. , Larocca , M. , García-Martín , D. , Diaz , N.L. , Braccia , P. , Fontana , E. , Rudolph , M.S. , Bermejo , P. , Ijaz , A. , Thanasilp , S. , Anschuetz , E.R. , Holmes , Z. : Does provable absence of barren plateaus imply classical simulability? Or, why we need to rethink variational quantum computing (2024). https://arxiv.org/abs/2...

    arXiv 2024

  7. [7]

    , Yang , C.-H.H

    barticle Chen , S.Y.-C. , Yang , C.-H.H. , Qi , J. , Chen , P.-Y. , Ma , X. , Goan , H.-S. : Variational quantum circuits for deep reinforcement learning . IEEE Access 8 , 141007 -- 141024 ( 2020 ) 10.1109/ACCESS.2020.3010470 barticle

    arXiv 2020

  8. [8]

    , Chang , Y.-J

    barticle Chen , H.-Y. , Chang , Y.-J. , Liao , S.-W. , Chang , C.-R. : Deep q-learning with hybrid quantum neural network on solving maze problems . Quantum Mach. Intell. 6 ( 1 ), 2 ( 2024 ) 10.1007/s42484-023-00137-w barticle

    doi:10.1007/s42484-023-00137-w 2024

Show all 61 references
  1. [9]

    , Kerenidis , I

    barticle Cherrat , E.A. , Kerenidis , I. , Prakash , A. : Quantum reinforcement learning via policy iteration . Quantum Mach. Intell. 5 ( 2 ), 30 ( 2023 ) 10.1007/s42484-023-00116-1 barticle

    2023 doi

  2. [10]

    , Yeung , D.-Y

    bchapter Choi , S.P.M. , Yeung , D.-Y. , Zhang , N.L. : Hidden-mode markov decision processes . In: Proceedings of the 16th International Joint Conference on Artificial Intelligence (IJCAI-99), Workshop on Neural Symbolic, and Reinforcement Methods for Sequence Learning, Stock...

    1999

  3. [11]

    , Rocchetto , A

    barticle Ciliberto , C. , Rocchetto , A. , Rudi , A. , Wossnig , L. : Statistical limits of supervised quantum learning . Phys. Rev. A 102 , 042414 ( 2020 ) 10.1103/PhysRevA.102.042414 barticle

    2020 doi

  4. [12]

    , Meier , U

    bchapter Cire s an , D.C. , Meier , U. , Masci , J. , Gambardella , L.M. , Schmidhuber , J. : Flexible, high performance convolutional neural networks for image classification . In: Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence ( 20...

    2011 doi

  5. [13]

    , Buffoni , L

    barticle Dalla Pozza , N. , Buffoni , L. , Martina , S. , Caruso , F. : Quantum reinforcement learning: the maze problem . Quantum Mach. Intell. 4 ( 1 ), 11 ( 2022 ) 10.1007/s42484-022-00068-y barticle

    2022 doi

  6. [14]

    , Chen , C

    barticle Dong , D. , Chen , C. , Li , H. , Tarn , T.-J. : Quantum reinforcement learning . IEEE Trans. Syst. Man. Cybern. B Cybern. 38 ( 5 ), 1207 -- 1220 ( 2008 ) 10.1109/TSMCB.2008.925743 barticle

    2008

  7. [15]

    , Taylor , J.M

    barticle Dunjko , V. , Taylor , J.M. , Briegel , H.J. : Quantum-enhanced machine learning . Phys. Rev. Lett. 117 , 130501 ( 2016 ) 10.1103/PhysRevLett.117.130501 barticle

    2016 doi

  8. [16]

    , Liu , Y.-K

    botherref Dunjko , V. , Liu , Y.-K. , Wu , X. , Taylor , J.M. : Exponential improvements for quantum-accessible reinforcement learning (2018). https://arxiv.org/abs/1710.11160 botherref

    2018 arXiv

  9. [17]

    , Abbeel , P

    bchapter Finn , C. , Abbeel , P. , Levine , S. : Model-agnostic meta-learning for fast adaptation of deep networks . In: Proceedings of the 34th International Conference on Machine Learning ( 2017 ) bchapter

    2017

  10. [18]

    , Wu , Y

    botherref Ganguly , B. , Wu , Y. , Wang , D. , Aggarwal , V. , Ganguly , B. , Wu , Y. , Wang , D. , Aggarwal , V. : Quantum computing provides exponential regret improvement in episodic reinforcement learning (2023). https://arxiv.org/abs/2302.08617 botherref

    2023 arXiv

  11. [19]

    , Gyurik , C

    botherref Gil-Fuster , E. , Gyurik , C. , Pérez-Salinas , A. , Dunjko , V. : On the relation between trainability and dequantization of variational quantum learning models (2025). https://arxiv.org/abs/2406.07072 botherref

    2025 arXiv

  12. [20]

    : Quantum mechanics helps in searching for a needle in a haystack

    barticle Grover , L.K. : Quantum mechanics helps in searching for a needle in a haystack . Phys. Rev. Lett. 79 , 325 -- 328 ( 1997 ) 10.1103/PhysRevLett.79.325 barticle

    1997 doi

  13. [21]

    , Dunjko , V

    barticle Hamann , A. , Dunjko , V. , W \"o lk , S. : Quantum-accessible reinforcement learning beyond strictly epochal environments . Quantum Mach. Intell. 3 ( 2 ), 22 ( 2021 ) 10.1007/s42484-021-00049-7 barticle

    2021 doi

  14. [22]

    , W \"o lk , S

    barticle Hamann , A. , W \"o lk , S. : Performance analysis of a hybrid agent for quantum-accessible reinforcement learning . New J. Phys. 24 ( 3 ), 033044 ( 2022 ) 10.1088/1367-2630/ac5b56 barticle

    2022 doi

  15. [23]

    , Gyurik , C

    bchapter Jerbi , S. , Gyurik , C. , Marshall , S.C. , Briegel , H.J. , Dunjko , V. : Parametrized quantum policies for reinforcement learning . In: Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 20...

    2021

  16. [24]

    , Trenkwalder , L.M

    barticle Jerbi , S. , Trenkwalder , L.M. , Poulsen Nautrup , H. , Briegel , H.J. , Dunjko , V. : Quantum enhancements for deep reinforcement learning in large spaces . PRX Quantum 2 , 010328 ( 2021 ) 10.1103/PRXQuantum.2.010328 barticle

    2021 doi

  17. [25]

    , Riemer , M

    barticle Khetarpal , K. , Riemer , M. , Rish , I. , Precup , D. : Towards continual reinforcement learning: A review and perspectives . J. Artif. Intell. Res. 75 , 1401 -- 1476 ( 2022 ) 10.1613/jair.1.13673 barticle

    2022 doi

  18. [26]

    , Sutskever , I

    barticle Krizhevsky , A. , Sutskever , I. , Hinton , G.E. : Imagenet classification with deep convolutional neural networks . Commun. ACM 60 ( 6 ), 84 -- 90 ( 2017 ) 10.1145/3065386 barticle

    2017 doi

  19. [27]

    , Towers , M

    botherref Kwiatkowski , A. , Towers , M. , Terry , J. , Balis , J.U. , Cola , G.D. , Deleu , T. , Goulão , M. , Kallinteris , A. , Krimmel , M. , KG , A. , Perez-Vicente , R. , Pierré , A. , Schulhoff , S. , Tai , J.J. , Tan , H. , Younis , O.G. : Gymnasium: a standard interfa...

    2024 arXiv

  20. [28]

    , Hunt , J.J

    botherref Lillicrap , T.P. , Hunt , J.J. , Pritzel , A. , Heess , N. , Erez , T. , Tassa , Y. , Silver , D. , Wierstra , D. : Continuous control with deep reinforcement learning (2019). https://arxiv.org/abs/1509.02971 botherref

    2019 arXiv

  21. [29]

    , Arunachalam , S

    barticle Liu , Y. , Arunachalam , S. , Temme , K. : A rigorous and robust quantum speed-up in supervised machine learning . Nat. Phys. 17 ( 9 ), 1013 -- 1017 ( 2021 ) 10.1038/s41567-021-01287-z barticle

    2021 doi

  22. [30]

    , Si , M

    bchapter Lockwood , O. , Si , M. : Playing atari with hybrid quantum-classical reinforcement learning . In: NeurIPS 2020 Workshop on Pre-registration in Machine Learning, 11 December 2020, Virtual Event ( 2020 ) bchapter

    2020

  23. [31]

    , Si , M

    barticle Lockwood , O. , Si , M. : Reinforcement learning with quantum variational circuit . Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment 16 ( 1 ), 245 -- 251 ( 2020 ) 10.1609/aiide.v16i1.7437 barticle

    2020 doi

  24. [32]

    , Makmal , A

    barticle Mautner , J. , Makmal , A. , Manzano , D. , Tiersch , M. , Briegel , H.J. : Projective simulation for classical learning agents: a comprehensive investigation . New Gener. Comput. 33 , 69 -- 114 ( 2015 ) 10.1007/s00354-015-0102-0 barticle

    2015 doi

  25. [33]

    , Cohen , N.J

    botherref McCloskey , M. , Cohen , N.J. : Catastrophic interference in connectionist networks: the sequential learning problem. Psychology of Learning and Motivation, vol. 24, pp. 109--165. Academic Press (1989). 10.1016/S0079-7421(08)60536-8 botherref

    1989 doi

  26. [34]

    , Makmal , A

    barticle Melnikov , A.A. , Makmal , A. , Briegel , H.J. : Benchmarking projective simulation in navigation problems . IEEE Access 6 , 64639 -- 64648 ( 2018 ) 10.1109/ACCESS.2018.2876494 barticle

    2018

  27. [35]

    , Scherer , D.D

    bchapter Meyer , N. , Scherer , D.D. , Plinge , A. , Mutschler , C. , Hartmann , M.J. : Quantum natural policy gradients: towards sample-efficient reinforcement learning . In: 2023 IEEE International Conference on Quantum Computing and Engineering (QCE) ( 2023 ). 10.1109/QCE57...

    2023

  28. [36]

    , Kavukcuoglu , K

    barticle Mnih , V. , Kavukcuoglu , K. , Silver , D. , Rusu , A.A. , Veness , J. , Bellemare , M.G. , Graves , A. , Riedmiller , M. , Fidjeland , A.K. , Ostrovski , G. , Petersen , S. , Beattie , C. , Sadik , A. , Antonoglou , I. , King , H. , Kumaran , D. , Wierstra , D. , Leg...

    2015 doi

  29. [37]

    , Gyurik , C

    botherref Molteni , R. , Gyurik , C. , Dunjko , V. : Exponential quantum advantages in learning quantum observables from classical data (2024). https://arxiv.org/abs/2405.02027 botherref

    2024 arXiv

  30. [38]

    : A survey of reinforcement learning algorithms for dynamically varying environments

    barticle Padakandla , S. : A survey of reinforcement learning algorithms for dynamically varying environments . ACM Comput. Surv. 54 ( 6 ), 1 -- 25 ( 2021 ) 10.1145/3459991 barticle

    2021 doi

  31. [39]

    , Dunjko , V

    barticle Paparo , G.D. , Dunjko , V. , Makmal , A. , Martin-Delgado , M.A. , Briegel , H.J. : Quantum speedup for active learning agents . Phys. Rev. X 4 , 031002 ( 2014 ) 10.1103/PhysRevX.4.031002 barticle

    2014 doi

  32. [40]

    , Wiering , M.A

    bchapter Pieters , M. , Wiering , M.A. : Q-learning with experience replay in a dynamic environment . In: 2016 IEEE Symposium Series on Computational Intelligence (SSCI) ( 2016 ). 10.1109/SSCI.2016.7849368 bchapter

    2016

  33. [41]

    : Quantum computing in the NISQ era and beyond

    barticle Preskill , J. : Quantum computing in the NISQ era and beyond . Quantum 2 , 79 ( 2018 ) 10.22331/q-2018-08-06-79 barticle

    2018 doi

  34. [42]

    : Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1st edn

    bbook Puterman , M.L. : Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1st edn. John Wiley & Sons, Inc. , USA ( 1994 ) bbook

    1994

  35. [43]

    : Continual learning in reinforcement environments

    botherref Ring , M.B. : Continual learning in reinforcement environments. PhD thesis, The University of Texas at Austin (1994) botherref

    1994

  36. [44]

    o mberg , T. , Schiansky , P. , Dunjko , V. , Friis , N. , Harris , N.C. , Hochberg , M. , Englund , D. , W \

    barticle Saggio , V. , Asenbeck , B.E. , Hamann , A. , Str \"o mberg , T. , Schiansky , P. , Dunjko , V. , Friis , N. , Harris , N.C. , Hochberg , M. , Englund , D. , W \"o lk , S. , Briegel , H.J. , Walther , P. : Experimental quantum speed-up in reinforcement learning agents...

    2021 doi

  37. [45]

    , Antonoglou , I

    barticle Schrittwieser , J. , Antonoglou , I. , Hubert , T. , Simonyan , K. , Sifre , L. , Schmitt , S. , Guez , A. , Lockhart , E. , Hassabis , D. , Graepel , T. , Lillicrap , T. , Silver , D. : Mastering atari, go, chess and shogi by planning with a learned model . Nature 58...

    2020 doi

  38. [46]

    , Killoran , N

    barticle Schuld , M. , Killoran , N. : Is quantum advantage the right goal for quantum machine learning? PRX Quantum 3 , 030101 ( 2022 ) 10.1103/PRXQuantum.3.030101 barticle

    2022 doi

  39. [47]

    , W \"o lk , S

    barticle Sefrin , O. , W \"o lk , S. : A hybrid learning agent for episodic learning tasks with unknown target distance . Quantum Mach. Intell. 7 ( 1 ), 52 ( 2025 ) 10.1007/s42484-025-00269-1 barticle

    2025 doi

  40. [48]

    : Algorithms for quantum computation: discrete logarithms and factoring

    bchapter Shor , P.W. : Algorithms for quantum computation: discrete logarithms and factoring . In: Proceedings 35th Annual Symposium on Foundations of Computer Science ( 1994 ). 10.1109/SFCS.1994.365700 bchapter

    1994

  41. [49]

    , Basso , E.W

    bchapter da Silva , B.C. , Basso , E.W. , Bazzan , A.L.C. , Engel , P.M. : Dealing with non-stationary environments using context detection . In: Proceedings of the 23rd International Conference on Machine Learning ( 2006 ). 10.1145/1143844.1143872 bchapter

    2006

  42. [50]

    , Jerbi , S

    barticle Skolik , A. , Jerbi , S. , Dunjko , V. : Quantum agents in the G ym: a variational quantum algorithm for deep Q -learning . Quantum 6 , 720 ( 2022 ) 10.22331/q-2022-05-24-720 barticle

    2022 doi

  43. [51]

    , Weiss , E

    bchapter Sohl-Dickstein , J. , Weiss , E. , Maheswaranathan , N. , Ganguli , S. : Deep unsupervised learning using nonequilibrium thermodynamics . In: Proceedings of the 32nd International Conference on Machine Learning ( 2015 ) bchapter

    2015

  44. [52]

    , W \"o lk , S

    barticle Sriarunothai , T. , W \"o lk , S. , Giri , G.S. , Friis , N. , Dunjko , V. , Briegel , H.J. , Wunderlich , C. : Speeding-up the decision making of a learning agent using an ion trap quantum processor . Quantum Sci. Technol. 4 ( 1 ), 015014 ( 2018 ) 10.1088/2058-9565/a...

    2018 doi

  45. [53]

    , Barto , A.G

    bbook Sutton , R.S. , Barto , A.G. : Reinforcement Learning: an Introduction. 2nd Edition . A Bradford Book , Cambridge ( 2018 ). http://incompleteideas.net/book/the-book-2nd.html bbook

    2018

  46. [54]

    , Shazeer , N

    bchapter Vaswani , A. , Shazeer , N. , Parmar , N. , Uszkoreit , J. , Jones , L. , Gomez , A.N. , Kaiser , . , Polosukhin , I. : Attention is all you need . In: Advances in Neural Information Processing Systems ( 2017 ) bchapter

    2017

  47. [55]

    , Dayan , P

    barticle Watkins , C.J.C.H. , Dayan , P. : Q-learning . Mach. Learn. 8 ( 3 ), 279 -- 292 ( 1992 ) 10.1007/BF00992698 barticle

    1992 doi

  48. [56]

    , Hein , D

    botherref Wiedemann , S. , Hein , D. , Udluft , S. , Mendl , C. : Quantum policy iteration via amplitude estimation and grover search -- towards quantum advantage for reinforcement learning (2023). https://arxiv.org/abs/2206.04741 botherref

    2023 arXiv

  49. [57]

    : Simple statistical gradient-following algorithms for connectionist reinforcement learning

    barticle Williams , R.J. : Simple statistical gradient-following algorithms for connectionist reinforcement learning . Mach. Learn. 8 , 229 -- 256 ( 1992 ) 10.1007/bf00992696 barticle

    1992 doi

  50. [58]

    , Jin , S

    barticle Wu , S. , Jin , S. , Wen , D. , Han , D. , Wang , X. : Quantum reinforcement learning in continuous action space . Quantum 9 , 1660 ( 2025 ) 10.22331/q-2025-03-12-1660 barticle

    2025 doi

  51. [59]

    , Hu , J

    botherref Zhong , H. , Hu , J. , Xue , Y. , Li , T. , Wang , L. : Provably efficient exploration in quantum reinforcement learning with logarithmic worst-case regret (2024). https://arxiv.org/abs/2302.10796 botherref

    2024 arXiv

  52. [60]

    , Stiennon , N

    botherref Ziegler , D.M. , Stiennon , N. , Wu , J. , Brown , T.B. , Radford , A. , Amodei , D. , Christiano , P. , Irving , G. : Fine-tuning language models from human preferences (2020). https://arxiv.org/abs/1909.08593 botherref

    2020 arXiv

  53. [61]

    write newline

    " write newline "" before.all 'output.state := FUNCTION string.to.integer 't := t text.length 'k := #1 'char.num := t char.num #1 substring 's := s is.num s "." = or char.num k = not and char.num #1 + 'char.num := while char.num #1 - 'char.num := t #1 char.num substring FUNCTI...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.