Pith. sign in

REVIEW 3 major objections 4 minor 46 references

Observation-Grounded Self-Predictive Reinforcement Learning for Visual Continuous Control

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A model-free visual RL agent learns faster when its representations are trained by both latent self-prediction and next-observation prediction, routed through two adapters so neither objective over-constrains the shared encoder.

desk verdict A genuine adapter-based combination of latent self-prediction and observation prediction with consistent aggregate gains, but the dog-domain headline is partly a tuning artifact and the anonymous baseline needs disclosure. read the letter →

arxiv 2608.05989 v1 pith:7LNPCA2X submitted 2026-08-06 cs.LG cs.RO

Xinwei Liu , Junyuan Liang , Jianting Zhang , Wuhui Chen This is my paper · ORCID
classification cs.LGcs.RO
keywords visualreinforcementlearningcontinuouscontrolsampleefficiencyself-predictiverepresentationobservationpredictionmodel-freeDMControlbenchmarkauxiliarylosses
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a model-free reinforcement learning agent for pixel-based continuous control becomes more sample-efficient when its representation learner combines two complementary dynamics-prediction objectives: multi-step latent self-prediction and next-observation prediction. The authors argue that latent self-prediction alone leaves latent transitions unaligned with observation-level dynamics, while observation prediction alone does not enforce long-horizon latent predictability. The proposed method, OG-SPR, routes the self-prediction objective through two lightweight adapters so it does not directly constrain the shared encoder, which the paper's ablation study identifies as the key to making the combination work. On 28 tasks of the DMControl benchmark at 500k environment steps, OG-SPR reports higher aggregate raw and DrQ-v2-normalized scores than the strongest self-predictive and observation-predictive baselines, with the largest relative gains on dog and humanoid tasks.

What carries the argument

The load-bearing mechanism is the adapter-mediated self-prediction branch: two lightweight linear adapters, Adapter 1 ($u_1$) after the observation encoder and Adapter 2 ($u_2$) after the state-action encoder, define the latent space in which multi-step self-prediction targets are computed, while the shared state-action representation is trained only by value learning, next-observation reconstruction, and short-term value prediction. This decoupling prevents the self-prediction objective from over-constraining the shared representation; the paper's ablations show that removing or sharing the adapters degrades performance, so the decoupling itself is a necessary part of the method.

What would settle it

Re-run the 28-task comparison with $\lambda_{\text{Self-predictive}}$ tuned per task (or on a held-out validation set) and recompute the aggregate DrQ-v2-normalized mean; if the advantage over the two predictive baselines disappears or reverses, the claim that combining the two objectives drives the gain fails. A second check is to run the released implementation of the anonymous observation-predictive baseline and confirm its reported task scores.

Watch

Extended reading notes

Core claim

The paper's central claim is that latent self-prediction and observation prediction are complementary representation-learning signals for model-free visual control, and that they can be combined profitably once self-prediction is decoupled from the shared representation by an adapter-mediated branch. OG-SPR minimizes, alongside a TD3-style value loss, a $K=5$-step latent self-prediction MSE loss computed in the adapter-defined space, a next-observation reconstruction loss in a normalized observation space, and a short-term value-prediction loss. The reported aggregate result on 28 DMControl tasks at 500k environment steps is a mean raw score of 626 (versus 607 for the observation-predictive baseline and 599 for the self-predictive baseline) and a mean DrQ-v2-normalized score of 5.05, about 1.8 times the self-predictive baseline's 2.84 and 1.9 times the observation-predictive baseline's 2.65. Ablations removing either predictive objective, sharing the adapters, or removing them entirely all lower aggregate performance, which the authors read as evidence that both objectives and the adapter decoupling carry the reported gain.

Load-bearing premise

The aggregate win rests on the self-prediction weight $\lambda_{\text{Self-predictive}} = 5.0$, tuned on only quadruped-run and dog-run, transferring to all 28 tasks, and on the anonymous observation-predictive baseline being a strong, correctly re-implemented method.

Editorial extensions

If this is right

  • If the central claim is right, OG-SPR sets a new aggregate benchmark for model-free pixel-based continuous control at 500k environment steps, ahead of both the self-predictive and the observation-predictive baselines it is compared against.
  • The largest gains appear on the hardest tasks: improvements of 20.9-63.8% on dog tasks and 61.7-333.7% on humanoid tasks over the stronger baseline, with the humanoid gap persisting at a relaxed 2M-step budget.
  • Both auxiliary objectives are necessary: ablations that drop either self-prediction or observation prediction lose aggregate raw and normalized score, and the loss is largest when self-prediction is removed.
  • The adapter design matters: sharing the two adapters or removing them (so self-prediction is imposed directly on the shared representation) also lowers aggregate performance, supporting the over-constraining hypothesis.
  • The method remains competitive on discrete-action Atari100k without task-specific engineering, ranking first or second on three of the four aggregate metrics reported.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the two-adapter design suggests a general recipe for combining predictive auxiliary losses in RL: route each predictive objective through its own projection head that reads from, but does not write to, the shared representation, so conflicting geometric constraints do not cancel.
  • Editorial inference: the complementarity hypothesis implies that other observation-level grounding signals (optical flow, depth, object segmentation) could replace or augment image reconstruction as the grounding objective and may yield further data-efficiency gains on tasks where pixel reconstruction is cheap but not informative enough.
  • Editorial inference: because the paper fixes $\lambda_{\text{Self-predictive}} = 5.0$ across all 28 tasks after tuning on two dog tasks, a natural testable extension is per-task or adaptive weighting of the three auxiliary losses; the authors themselves list fixed weights as a limitation.
  • Editorial inference: the DrQ-v2-normalized aggregate amplifies low-scoring tasks (small denominator), so the large normalized gap (2.84/2.65 to 5.05) is driven substantially by dog and humanoid tasks; future comparisons should report per-task raw scores alongside normalized aggregates.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. This paper proposes OG-SPR, a model-free visual continuous-control RL algorithm that trains an actor-critic agent with three auxiliary objectives: multi-step latent self-prediction, next-observation prediction, and short-term value prediction. To prevent the latent self-prediction objective from over-constraining the shared representation, the authors introduce two lightweight adapters that form an adapter-mediated self-prediction branch. The paper evaluates OG-SPR on 28 DeepMind Control Suite tasks at 500k environment steps (plus 2M-step humanoid runs), reporting improvements over DrQ-v2, TD-MPC2, DreamerV3, MR.Q, and an anonymous observation-predictive baseline called AnonMethod. The authors also report ablations removing each auxiliary objective and each adapter component, a linear-probing representation analysis, and an Atari100k adaptation. The central claim is that OG-SPR achieves a new state of the art for aggregate performance on DMControl, with particularly large gains on dog and humanoid tasks.

Significance. If the empirical claims hold, OG-SPR would be a meaningful contribution: it demonstrates a design for combining latent self-prediction and observation prediction in model-free visual RL, and its aggregate improvements are supported by five-seed runs with bootstrap confidence intervals, controlled comparisons against strong baselines, and consistent ablations. The paper also ships implementation details in the supplementary material, reports wall-clock time, and honestly states limitations. However, the headline result is contingent on two load-bearing issues: the self-prediction weight is tuned on two of the very tasks used to advertise the dog-domain gains, and the observation-predictive baseline is an anonymous, suppressed-citation method whose correctness cannot be checked from the main text. The DrQv2-normalized mean is also sensitive to tasks with very small raw scores.

major comments (3)
  1. [Implementation Details] The choice of lambda_Self-predictive = 5.0 is made by a grid search over integer values on quadruped-run and dog-run only, and this same value is then used for all 28 tasks. Since dog-run is one of the two tasks on which the headline 'particularly pronounced gains in the dog domain' is based (dog-run 90 vs 55 for AnonMethod and 51 for MR.Q in Table 1), the dog-domain improvement is not independent evidence for the method. The manuscript provides no sensitivity analysis varying this weight, and the ablation in Table 2 only removes the self-prediction objective entirely rather than varying lambda. I request a lambda-sensitivity analysis (e.g., lambda in {2,3,4,5,6} on the tuning tasks plus several held-out tasks such as humanoid-walk, cheetah-run, quadruped-walk, and acrobot-swingup) and, if possible, a re-run of the full 28-task evaluation with the SPR default lambda=2.0 so the reader can see how much of the aggregate DrQv2-normalized mean of 5.05 is attributable to the selected value. This is load-bearing because the paper's own conclusion acknowledges that fixed auxiliary loss weights 'may not be optimal for all tasks.'
  2. [Table 1 and Evaluation Protocol] The headline aggregate metric 'DrQv2-normalized mean = 5.05' is driven in large part by tasks where raw scores are extremely small. For example, humanoid-run raw score is 3 for OG-SPR versus 1 for MR.Q and AnonMethod (Table 1), so the normalized score is roughly 3 on a baseline denominator of approximately 1; the dog and 2M humanoid tasks behave similarly. The paper reports the IQM (1.61 for OG-SPR vs 1.29 for MR.Q) as a robustness check, but the main text continues to emphasize the 1.8x mean improvement without reporting the per-task normalized scores or a median. Please report the full per-task DrQv2-normalized table (or at least the distribution and the contribution of dog/humanoid tasks to the mean), and discuss how much of the 5.05 mean is due to the small denominators in these tasks. If the aggregate claim is 'improves aggregate performance,' it should be stated at the level of both mean and IQM with the small-score caveat made explicit.
  3. [References and Supplementary Material] The comparison against observation-predictive methods rests entirely on 'AnonMethod (Anonymous 2026)', whose citation is suppressed and whose implementation is said to be provided only in the supplementary material. As written, a reader cannot verify that this is a strong, correctly re-implemented baseline: the baseline's exact architecture, hyperparameter values, and training protocol are not stated in the main text, and the suppressed reference prevents attribution or comparison with any published work. For a journal submission, this is a verifiability gap in a load-bearing comparison. The revision should either cite the actual published source (after de-anonymization) or, if the method is unpublished, provide a complete standalone description and a persistent repository for both AnonMethod and OG-SPR so that the baselines can be reproduced without relying on an anonymous supplement.
minor comments (4)
  1. [Figure 1] The caption should state precisely that the baseline score is the stronger result between MR.Q and AnonMethod per task, and the axes should be labeled with task names and percentage improvement.
  2. [Equation (9)] The two-hot encoding and the symexp return support are introduced without a formal definition; please specify the bin placement, the number of bins, and how symexp is inverted.
  3. [Table 3] On Atari100k, OG-SPR is not consistently better than AnonMethod on the human-normalized mean, yet the text says it 'remains competitive' and ranks first or second on three of four metrics; the narrative should more carefully acknowledge that the gains do not transfer to discrete-action domains.
  4. [Conclusion] The limitations paragraph mentions fixed auxiliary loss weights but not the sensitivity of the self-prediction horizon K or the choice of n-step returns; a sentence noting these additional fixed hyperparameters would be appropriate.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's claims are empirical, the auxiliary objectives are independently defined, and the disclosed hyperparameter choice on two tasks is a selection-bias caveat rather than a by-construction reduction.

full rationale

OG-SPR's claimed contribution is empirical: a model-free visual RL algorithm combining three auxiliary objectives. The losses in Eqs. (3)-(10) are independently defined (n-step TD target, decoder reconstruction, two-hot return prediction, latent MSE), and no equation defines a performance claim in terms of the method's own output. The only evaluation-dependence risk is that lambda_Self-predictive=5.0 was selected on quadruped-run and dog-run and then dog-run/quadruped-run appear among the improved results; this is disclosed in Implementation Details and is a standard selection-bias caveat, not a by-construction reduction, and the humanoid gains (no humanoid task was used for tuning) plus 26 untouched tasks provide independent content. The anonymous AnonMethod baseline makes external verification harder, but the paper re-runs baselines and does not derive its main result from that citation. The conclusion's limitation that fixed auxiliary weights may not be optimal across tasks confirms the empirical rather than derivational status. Therefore the derivation chain is not circular.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim does not rest on a formal derivation. It relies on standard RL assumptions, common architectural choices borrowed from prior methods, and one hyperparameter tuned on a small subset. The adapters are architectural components, not new physical entities.

free parameters (5)
  • lambda_Self-predictive = 5.0 (DMControl), 2.0 (Atari100k)
    Tuned on a two-task subset (quadruped-run and dog-run) for DMControl and on Alien/Amidar for Atari100k; fixed across tasks.
  • lambda_Rec = 0.1 (DMControl), 1.0 (Atari100k)
    Adopted from AnonMethod defaults, not tuned by this paper, but affects the balance between reconstruction and value loss.
  • lambda_n-step = 1.0
    Adopted from AnonMethod defaults; weight for the short-term value prediction auxiliary loss.
  • self-prediction horizon K = 5
    Follows SPR and MR.Q; number of future latent states predicted recursively.
  • n-step returns = 3
    Borrowed from DrQ-v2/MR.Q; horizon for value targets and short-term return prediction.
assumptions (4)
  • domain assumption Stacking three consecutive frames approximately satisfies the Markov property in the visual control MDP.
    Stated in Preliminaries and used to define state s_t as three stacked observations.
  • domain assumption The target networks provide stable, non-differentiable targets for latent self-prediction without causing representation collapse.
    Used in the self-prediction loss in Equation 7; relies on empirical practice from SPR, not proven.
  • domain assumption Random shift augmentation preserves the dynamics information needed for both reconstruction and self-prediction targets.
    Implemented in the observation encoder and applied with different sampling for the two auxiliary losses; no invariance proof is given.
  • ad hoc to paper Auxiliary prediction losses in latent and observation space improve policy-relevant representation quality without distorting the RL objective.
    The motivating assumption of the method; supported only by the reported ablations, not by a theoretical guarantee.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Observation-Grounded Self-Predictive Reinforcement Learning for Visual Continuous Control." pith.science (2026). https://pith.science/paper/7LNPCA2X

@misc{pith2026260805989,
  author       = {Pith},
  title        = {Pith review of: Observation-Grounded Self-Predictive Reinforcement Learning for Visual Continuous Control},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7LNPCA2X}},
  note         = {Machine review of arXiv:2608.05989}
}
read the original abstract

Sample-efficient policy learning from pixels is a long-standing challenge in reinforcement learning (RL). Recent dynamics-based representation learning methods have significantly improved the sample efficiency of model-free visual RL by learning dynamics-aware representations through auxiliary prediction performed either in latent space (self-prediction) or observation space (observation prediction). However, state-of-the-art methods from both categories still struggle on challenging visual control tasks when training data is limited. We posit that relying on either predictive objective alone may be insufficient. In contrast, observation prediction grounds learned representations in observation-level dynamics, but does not directly regularize the temporal predictability of latent representations over extended horizons. In this paper, we propose Observation-Grounded Self-Predictive Representations (OG-SPR), a model-free visual RL algorithm for continuous control that learns representations that are both temporally predictive in latent space and grounded in observation-level dynamics. OG-SPR incorporates two core auxiliary objectives: multi-step latent self-prediction and next-observation prediction. We empirically show that directly imposing latent self-prediction on the shared representation may over-constrain it and does not necessarily improve performance. To address this issue, OG-SPR introduces two lightweight adapters for latent self-prediction, allowing the shared representation to benefit from temporally predictive signals without being forced to directly satisfy the self-prediction objective. Experiments on 28 visual control tasks from the DeepMind Control Suite show that OG-SPR improves aggregate performance over state-of-the-art self-predictive and observation-predictive RL methods, with particularly pronounced gains in challenging domains such as dog and humanoid.

Figures

Figures reproduced from arXiv: 2608.05989 by the authors.

Figure 1
Figure 1. Performance improvements of OG-SPR over state [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. An illustration of the full OG-SPR method. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Aggregate learning curves on DMControl, com [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (3 more)
Figure 1
Figure 1. Figure 1: Per-task learning curves on the DMControl benchmark for OG-SPR and baselines (Part 1). Solid lines indicate average [PITH_FULL_IMAGE:figures/full_fig_p016_1.png]
Figure 2
Figure 2. Figure 2: Per-task learning curves on the DMControl benchmark for OG-SPR and baselines (Part 2). Solid lines indicate average [PITH_FULL_IMAGE:figures/full_fig_p017_2.png]
Figure 3
Figure 3. Figure 3: Per-task learning curves on the humanoid tasks from the DMControl benchmark for OG-SPR and baselines under the 2M-environment-step training budget. Solid lines indicate mean performance over 5 seeds, and shaded areas indicate 95% bootstrap confidence intervals [PITH_F…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 5 canonical work pages

  1. [1]

    The Tenth International Conference on Learning Representations,

    Denis Yarats and Rob Fergus and Alessandro Lazaric and Lerrel Pinto , title =. The Tenth International Conference on Learning Representations,. 2022 , url =

    2022

  2. [2]

    9th International Conference on Learning Representations,

    Denis Yarats and Ilya Kostrikov and Rob Fergus , title =. 9th International Conference on Learning Representations,. 2021 , url =

    2021

  3. [3]

    The Thirteenth International Conference on Learning Representations,

    Scott Fujimoto and Pierluca D'Oro and Amy Zhang and Yuandong Tian and Michael Rabbat , title =. The Thirteenth International Conference on Learning Representations,. 2025 , url =

    2025

  4. [4]

    2026 , eprint=

    NASDAQ: Normalized Observation Space Dynamics-Augmented Q-Learning , author=. 2026 , eprint=

    work page 2026

  5. [5]

    Devon Hjelm and Aaron C

    Max Schwarzer and Ankesh Anand and Rishab Goel and R. Devon Hjelm and Aaron C. Courville and Philip Bachman , title =. 9th International Conference on Learning Representations,. 2021 , url =

    2021

  6. [6]

    Max Schwarzer and Johan S. Obando. Bigger, Better, Faster: Human-level Atari with human-level efficiency , booktitle =. 2023 , url =

    2023

  7. [7]

    CoRR , volume =

    Kyungsoo Kim and Jeongsoo Ha and Yusung Kim , title =. CoRR , volume =. 2025 , url =. doi:10.48550/ARXIV.2506.05418 , eprinttype =. 2506.05418 , timestamp =

    arXiv doi:10.48550/arxiv.2506.05418 2025

  8. [8]

    Bootstrap Your Own Latent -

    Jean. Bootstrap Your Own Latent -. Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual , year =

    2020

Show all 46 references
  1. [9]

    Bellemare , editor =

    Carles Gelada and Saurabh Kumar and Jacob Buckman and Ofir Nachum and Marc G. Bellemare , editor =. DeepMDP: Learning Continuous Latent Space Models for Representation Learning , booktitle =. 2019 , url =

    2019

  2. [10]

    Nature , pages=

    Mastering diverse control tasks through world models , author=. Nature , pages=. 2025 , publisher=

    2025

  3. [11]

    The Twelfth International Conference on Learning Representations,

    Nicklas Hansen and Hao Su and Xiaolong Wang , title =. The Twelfth International Conference on Learning Representations,. 2024 , url =

    2024

  4. [12]

    1998 , publisher=

    Reinforcement learning: An introduction , author=. 1998 , publisher=

    1998

  5. [13]

    Lillicrap and Martin A

    Yuval Tassa and Yotam Doron and Alistair Muldal and Tom Erez and Yazhe Li and Diego de Las Casas and David Budden and Abbas Abdolmaleki and Josh Merel and Andrew Lefrancq and Timothy P. Lillicrap and Martin A. Riedmiller , title =. CoRR , volume =. 2018 , url =. 1801.00690 , t...

    2018 arXiv

  6. [14]

    Learning Representations via a Robust Behavioral Metric for Deep Reinforcement Learning , booktitle =

    Jianda Chen and Sinno Jialin Pan , editor =. Learning Representations via a Robust Behavioral Metric for Deep Reinforcement Learning , booktitle =. 2022 , url =

    2022

  7. [15]

    MICo: Improved representations via sampling-based state similarity for Markov decision processes , booktitle =

    Pablo Samuel Castro and Tyler Kastner and Prakash Panangaden and Mark Rowland , editor =. MICo: Improved representations via sampling-based state similarity for Markov decision processes , booktitle =. 2021 , url =

    2021

  8. [16]

    The Twelfth International Conference on Learning Representations,

    Guozheng Ma and Lu Li and Sen Zhang and Zixuan Liu and Zhen Wang and Yixin Chen and Li Shen and Xueqian Wang and Dacheng Tao , title =. The Twelfth International Conference on Learning Representations,. 2024 , url =

    2024

  9. [17]

    Why do We Need Large Batchsizes in Contrastive Learning?

    Changyou Chen and Jianyi Zhang and Yi Xu and Liqun Chen and Jiali Duan and Yiran Chen and Son Tran and Belinda Zeng and Trishul Chilimbi , editor =. Why do We Need Large Batchsizes in Contrastive Learning?. Advances in Neural Information Processing Systems 35: Annual Conferenc...

    2022

  10. [18]

    Contrastive Learning as Goal-Conditioned Reinforcement Learning , booktitle =

    Benjamin Eysenbach and Tianjun Zhang and Sergey Levine and Ruslan Salakhutdinov , editor =. Contrastive Learning as Goal-Conditioned Reinforcement Learning , booktitle =. 2022 , url =

    2022

  11. [19]

    The Thirteenth International Conference on Learning Representations,

    Grace Liu and Michael Tang and Benjamin Eysenbach , title =. The Thirteenth International Conference on Learning Representations,. 2025 , url =

    2025

  12. [20]

    Policy-Independent Behavioral Metric-Based Representation for Deep Reinforcement Learning , booktitle =

    Weijian Liao and Zongzhang Zhang and Yang Yu , editor =. Policy-Independent Behavioral Metric-Based Representation for Deep Reinforcement Learning , booktitle =. 2023 , url =. doi:10.1609/AAAI.V37I7.26052 , timestamp =

    2023 doi

  13. [21]

    Revisiting Bisimulation Metric for Robust Representations in Reinforcement Learning , journal =

    Leiji Zhang and Zeyu Wang and Xin Li and Yao. Revisiting Bisimulation Metric for Robust Representations in Reinforcement Learning , journal =. 2025 , url =. doi:10.48550/ARXIV.2507.18519 , eprinttype =. 2507.18519 , timestamp =

    2025 doi

  14. [22]

    Scott Fujimoto and Wei. For. Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023 , year =

    2023

  15. [23]

    Jha and Toshisada Mariyama and Daniel Nikovski , title =

    Kei Ota and Tomoaki Oiki and Devesh K. Jha and Toshisada Mariyama and Daniel Nikovski , title =. Proceedings of the 37th International Conference on Machine Learning,. 2020 , url =

    2020

  16. [24]

    Lillicrap and Jimmy Ba and Mohammad Norouzi , title =

    Danijar Hafner and Timothy P. Lillicrap and Jimmy Ba and Mohammad Norouzi , title =. 8th International Conference on Learning Representations,. 2020 , url =

    2020

  17. [25]

    Lillicrap and Mohammad Norouzi and Jimmy Ba , title =

    Danijar Hafner and Timothy P. Lillicrap and Mohammad Norouzi and Jimmy Ba , title =. 9th International Conference on Learning Representations,. 2021 , url =

    2021

  18. [26]

    Addressing Function Approximation Error in Actor-Critic Methods , booktitle =

    Scott Fujimoto and Herke van Hoof and David Meger , editor =. Addressing Function Approximation Error in Actor-Critic Methods , booktitle =. 2018 , url =

    2018

  19. [27]

    Emanuel Todorov and Tom Erez and Yuval Tassa , title =. 2012. 2012 , url =. doi:10.1109/IROS.2012.6386109 , timestamp =

    2012

  20. [28]

    PyTorch: An Imperative Style, High-Performance Deep Learning Library , booktitle =

    Adam Paszke and Sam Gross and Francisco Massa and Adam Lerer and James Bradbury and Gregory Chanan and Trevor Killeen and Zeming Lin and Natalia Gimelshein and Luca Antiga and Alban Desmaison and Andreas K. PyTorch: An Imperative Style, High-Performance Deep Learning Library ,...

    2019

  21. [29]

    Lukasz Kaiser and Mohammad Babaeizadeh and Piotr Milos and Blazej Osinski and Roy H. Campbell and Konrad Czechowski and Dumitru Erhan and Chelsea Finn and Piotr Kozakowski and Sergey Levine and Afroz Mohiuddin and Ryan Sepassi and George Tucker and Henryk Michalewski , title =...

    2020

  22. [30]

    Bellemare and Yavar Naddaf and Joel Veness and Michael Bowling , title =

    Marc G. Bellemare and Yavar Naddaf and Joel Veness and Michael Bowling , title =. J. Artif. Intell. Res. , volume =. 2013 , url =. doi:10.1613/JAIR.3912 , timestamp =

    2013 doi

  23. [31]

    iQRL - Implicitly Quantized Representations for Sample-efficient Reinforcement Learning , journal =

    Aidan Scannell and Kalle Kujanp. iQRL - Implicitly Quantized Representations for Sample-efficient Reinforcement Learning , journal =. 2024 , url =. doi:10.48550/ARXIV.2406.02696 , eprinttype =. 2406.02696 , timestamp =

    2024 doi arXiv

  24. [32]

    Ruijie Zheng and Xiyao Wang and Yanchao Sun and Shuang Ma and Jieyu Zhao and Huazhe Xu and Hal Daum. Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 202...

    2023

  25. [33]

    Riedmiller , title =

    David Silver and Guy Lever and Nicolas Heess and Thomas Degris and Daan Wierstra and Martin A. Riedmiller , title =. Proceedings of the 31th International Conference on Machine Learning,. 2014 , url =

    2014

  26. [34]

    Lillicrap and David Silver , title =

    Julian Schrittwieser and Ioannis Antonoglou and Thomas Hubert and Karen Simonyan and Laurent Sifre and Simon Schmitt and Arthur Guez and Edward Lockhart and Demis Hassabis and Thore Graepel and Timothy P. Lillicrap and David Silver , title =. Nat. , volume =. 2020 , url =. doi...

    2020 doi

  27. [35]

    Dueling Network Architectures for Deep Reinforcement Learning , booktitle =

    Ziyu Wang and Tom Schaul and Matteo Hessel and Hado van Hasselt and Marc Lanctot and Nando de Freitas , editor =. Dueling Network Architectures for Deep Reinforcement Learning , booktitle =. 2016 , url =

    2016

  28. [36]

    Temporal Difference Learning for Model Predictive Control , booktitle =

    Nicklas Hansen and Hao Su and Xiaolong Wang , editor =. Temporal Difference Learning for Model Predictive Control , booktitle =. 2022 , url =

    2022

  29. [37]

    Riedmiller , title =

    Volodymyr Mnih and Koray Kavukcuoglu and David Silver and Alex Graves and Ioannis Antonoglou and Daan Wierstra and Martin A. Riedmiller , title =. CoRR , volume =. 2013 , url =. 1312.5602 , timestamp =

    2013 arXiv

  30. [38]

    Ayoub Echchahed and Pablo Samuel Castro , title =. Trans. Mach. Learn. Res. , volume =. 2025 , url =

    2025

  31. [39]

    An Equivalence between Loss Functions and Non-Uniform Sampling in Experience Replay , booktitle =

    Scott Fujimoto and David Meger and Doina Precup , editor =. An Equivalence between Loss Functions and Non-Uniform Sampling in Experience Replay , booktitle =. 2020 , url =

    2020

  32. [40]

    7th International Conference on Learning Representations,

    Ilya Loshchilov and Frank Hutter , title =. 7th International Conference on Learning Representations,. 2019 , url =

    2019

  33. [41]

    Rusu and Joel Veness and Marc G

    Volodymyr Mnih and Koray Kavukcuoglu and David Silver and Andrei A. Rusu and Joel Veness and Marc G. Bellemare and Alex Graves and Martin A. Riedmiller and Andreas Fidjeland and Georg Ostrovski and Stig Petersen and Charles Beattie and Amir Sadik and Ioannis Antonoglou and Hel...

    2015 doi

  34. [42]

    Rainbow: Combining Improvements in Deep Reinforcement Learning , booktitle =

    Matteo Hessel and Joseph Modayil and Hado van Hasselt and Tom Schaul and Georg Ostrovski and Will Dabney and Dan Horgan and Bilal Piot and Mohammad Gheshlaghi Azar and David Silver , editor =. Rainbow: Combining Improvements in Deep Reinforcement Learning , booktitle =. 2018 ,...

    2018 doi

  35. [43]

    Bridging State and History Representations: Understanding Self-Predictive

    Tianwei Ni and Benjamin Eysenbach and Erfan Seyedsalehi and Michel Ma and Clement Gehring and Aditya Mahajan and Pierre. Bridging State and History Representations: Understanding Self-Predictive. The Twelfth International Conference on Learning Representations,. 2024 , url =

    2024

  36. [44]

    When does Self-Prediction help? Understanding Auxiliary Tasks in Reinforcement Learning , journal =

    Claas Voelcker and Tyler Kastner and Igor Gilitschenski and Amir. When does Self-Prediction help? Understanding Auxiliary Tasks in Reinforcement Learning , journal =. 2024 , url =

    2024

  37. [45]

    James Bradbury and Roy Frostig and Peter Hawkins and Matthew James Johnson and Yash Katariya and Chris Leary and Dougal Maclaurin and George Necula and Adam Paszke and Jake VanderPlas and Skye Wanderman-Milne and Qiao Zhang , title =

  38. [46]

    2026 , note =

    Anonymous , title =. 2026 , note =

    2026

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.