Pith. sign in

REVIEW 5 major objections 5 minor 63 references

Efficient Reinforcement Learning Through Adaptively Pretrained Visual Encoder

T0 review · 5 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that adaptively pretraining a ResNet18 encoder on natural images with a closed-loop augmentation schedule makes DreamerV3 and DrQ-v2 learn visual control tasks faster and reach higher scores, approaching state-based SAC.

desk verdict A useful empirical recipe for contrastively pretrained visual RL, but the central claim about adaptive augmentation is untested because the paper never compares against a fixed-augmentation MoCo v2 encoder on the same data. read the letter →

arxiv 2502.05555 v1 pith:MDTJICDG submitted 2025-02-08 cs.CV

Yuhan Zhang , Guoqing Ma , Guangfu Hao , Liangxuan Guo , Yang Chen , Shan Yu This is my paper · ORCID
classification cs.CV
keywords adaptivepretrainingcontrastivelearningvisualreinforcementsampleefficiencyDreamerV3DrQ-v2dataaugmentationImageNet-100
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the pretraining recipe of a visual encoder, not its depth, is what determines how well it transfers to reinforcement learning. APE pretrains a ResNet18 on ImageNet-100 with a contrastive objective (MoCo v2) while adaptively re-weighting five data augmentations: after each epoch, augmentations with lower pretext accuracy get sampled more often. When the first three layers of the resulting encoder are frozen and only the last layer is finetuned inside DreamerV3 or DrQ-v2, the agents learn faster and reach higher scores than with randomly initialized, supervised, or fixed-augmentation encoders. If the claim holds, adaptive pretraining augmentation becomes a simple and reusable ingredient for sample-efficient visual RL that needs no auxiliary tasks or extra sensors.

What carries the argument

The load-bearing mechanism is the adaptive augmentation update of Eq. 5, $p^{t+1}_i = \operatorname{softmax}(\alpha(1 - \mathrm{Acc}_t))$, which raises the sampling probability of augmentation compositions whose pretext accuracy is low. All compositions share the same query and key encoders, so the InfoNCE loss keeps the representations consistent across sub-batches. During policy learning, the first three ResNet18 layers are frozen and only the last layer is finetuned, preserving the pretrained generalization while allowing task-specific adaptation. Linear-probe accuracy on ImageNet-100 is used as the evaluation proxy for encoder quality.

What would settle it

Pretrain an identical ResNet18 with a fixed augmentation schedule that always uses the best single composition (for example, gaussian blur applied with probability 0.5) and compare downstream DMC scores against APE. If the fixed schedule matches or exceeds APE's scores, the adaptive update in Eq. 5 is not the cause of the gains. A second test: construct a pretext task whose linear-probe accuracy orders encoders differently from their downstream DMC performance, which would break the claimed proxy relation.

Watch

Extended reading notes

Core claim

APE's central claim is that a closed-loop augmentation schedule during contrastive pretraining produces visual features that are more generalizable to downstream control tasks than any fixed augmentation recipe or deeper architecture. On DMC, APE raises DreamerV3's mean 100K-step score from 458.6 to 836.7 (SAC:state scores 816 on the same three tasks), and its 500K-step score from 749.1 to 882.8; DrQ-v2's mean score rises from 284.6 to 560.2 at 100K steps. The same encoder also lifts DreamerV3 on Atari 100k and Memory Maze with fixed hyperparameters. The paper also reports that a deeper ResNet50 pretrained the same way performs worse than ResNet18, and that linear-probe accuracy on ImageNet-100 ranks pretrained encoders in the same order as their downstream RL performance.

Load-bearing premise

The claim holds only if pretext-task accuracy on ImageNet-100 is a reliable proxy for how useful the learned features will be for downstream RL control; if that correlation is weak, the adaptive schedule has no principled reason to outperform a fixed augmentation recipe.

Editorial extensions

If this is right

  • APE raises DreamerV3's mean DMC score from 458.6 to 836.7 at 100K steps and from 749.1 to 882.8 at 500K steps, bringing pixel-based control near the state-based SAC baseline on those tasks.
  • The benefit transfers across RL paradigms: the same pretrained encoder improves both model-based DreamerV3 and model-free DrQ-v2, so the gain lives in the representation rather than in a specific agent.
  • Because linear-probe accuracy on ImageNet-100 predicts downstream RL performance, cheap static-image evaluation can screen candidate pretrained encoders before any RL training is run.
  • Encoder depth is not the driver: ResNet50 pretrained with the same recipe underperforms ResNet18 on fine-grained control, so the pretraining strategy matters more than network size.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the linear-probe proxy holds generally, pretraining research for visual RL can be conducted offline on static image datasets, which is far cheaper than environment interaction and would accelerate iteration.
  • The adaptive schedule is only tested during pretraining; an untested extension is to keep adapting augmentation probabilities while the agent finetunes, which could further narrow the domain gap on tasks like Atari where ImageNet features hurt.
  • The paper's Atari Boxing analysis suggests that single-object ImageNet features can blind the agent to multi-item scenes; a testable fix is to include multi-object or scene-level datasets in the pretraining distribution.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes APE, a visual encoder pretrained on ImageNet-100 with MoCo v2 and an adaptive augmentation schedule (Eq. 5) that updates augmentation sampling probabilities based on pretext-task accuracy. The pretrained ResNet18 encoder, with its first three layers frozen and the last layer trainable, is then integrated into DreamerV3 and DrQ-v2. Experiments on DMC, five Atari 100k games, and Memory Maze report improved sample efficiency and final performance over non-pretrained baselines and over RRL and PIE-G, approaching state-based SAC on some DMC tasks. The paper also claims that linear-probe accuracy can serve as a useful metric for assessing pretrained encoder quality without running RL environments.

Significance. If the central claim were fully supported, APE would provide a simple, transferable ingredient for visual RL: an adaptively pretrained encoder that improves sample efficiency across model-based and model-free algorithms. The paper's strengths include evaluating two RL backbones (DreamerV3 and DrQ-v2), reporting results on multiple domains, and including visualizations (LayerCAM, PCA, reconstructions) that illustrate representational differences. However, the claimed contribution of adaptivity is not isolated: no downstream RL experiment compares against a non-adaptive MoCo v2 encoder pretrained on the same ImageNet-100 data. The selective Atari evaluation, mixed per-task results at longer horizons, and absence of error bars further weaken the empirical case. The paper builds directly on AdDA (Zhang et al. 2023), and the incremental contribution would need the missing control experiment to be credible.

major comments (5)
  1. [Methodology – Adaptive Pretraining; Experiments – Comparison with Other Pretrained Algorithms] The paper's central claim is that the adaptive augmentation schedule in Eq. (5) improves downstream RL performance. Yet no experiment compares APE against a non-adaptive MoCo v2 encoder pretrained on the same ImageNet-100 data with the same architecture and then evaluated in the same RL policies. Table 1 reports only linear-probe accuracy, where APE (91.7) exceeds MoCo v2 (90.84) by less than one point and without variance; Fig. 8 and Table 1 of Appendix B compare APE only against RRL, PIE-G, random initialization, and the base algorithm. As a result, the observed DMC, Atari, and Memory Maze gains could be attributed to contrastive pretraining on natural images generally rather than to the adaptive mechanism, and the contribution named in the title is not isolated. Please add an RL experiment with the fixed-augmentation MoCo v2 baseline under identical conditions.
  2. [Experiments – Results on Other Benchmarks] The Atari 100k evaluation uses only 5 of the 26 games, selected after acknowledging a large domain gap (Appendix B). The standard Atari 100k protocol reports results on all 26 games, and the paper itself notes a performance decrease on Atari Boxing with ImageNet-pretrained encoders. Claiming 'state-of-the-art performance' (Abstract) based on this selective subset is not supported. Please either expand to the full 26-game benchmark or qualify the claim accordingly.
  3. [Experiments – DMC Results; Appendix B, Tables 1–2] No variability measures are reported despite the statement that 'Results reported are averaged over at least 3 runs' (Experiments). Figures 3, 6, 7, 9, 10, 11 and Tables in Appendix B show only point estimates. Several reported differences are small or task-dependent (e.g., at 500K DMC steps, APE(DreamerV3) is lower than DreamerV3 on Walker Walk, 943.8 vs 950.4; at 1M steps APE is lower on Cheetah Run, 688.6 vs 728.7). Without error bars, confidence intervals, or significance tests, the headline improvements cannot be assessed. Please report standard deviations or comparable statistics for all main results.
  4. [Appendix C – Implementation Details, Algorithm 1; Methodology, Eq. (5)] Algorithm 1 is internally inconsistent with the formal update rule in Eq. (5). Line 2 computes sub-batch sizes as softmax(α p_i) × num_X, while Eq. (5) updates the sampling probability with softmax(α(1 − Acc_t)); the relationship between p_i used in the loss (Eq. (4)) and the sub-batch sizes is never defined. It is also unclear how the 'pretext task accuracy' Acc_t is computed (per sub-batch, using a linear probe, at which epoch). This ambiguity prevents exact reproduction of the central adaptive mechanism.
  5. [Experiments – Ablation Studies; Table 1; Figure 10] The paper claims that linear probes 'can serve as a useful metric to assess the quality of pretrained encoders quite effectively' (Introduction) and uses this to justify the adaptive update. However, the evidence is only the comparison of Table 1 with Figure 10 on DMC tasks for two augmentation choices (blur vs jitter); no quantitative correlation, no other domains, and no statistical test are provided. Given the paper's own finding that ImageNet-pretrained encoders can hurt on multi-object Atari observations (Appendix B), the claim that linear-probe accuracy predicts downstream RL performance is under-supported and should be either substantiated or softened.
minor comments (5)
  1. [Experiments – Results on Other Benchmarks] The sentence 'Following the common setup of Atari 100k, we set the environment steps to 40k' is inconsistent with the definition of Atari 100k (100k environment steps, 400k frames) stated in Appendix A; please correct the value.
  2. [Introduction] Several typos appear, e.g., 'high-demensional', 'Intersetingly', 'enviorments'; please copy-edit the manuscript.
  3. [Experiments – Comparison with Other Pretrained Algorithms, Fig. 8] In the lower panel of Fig. 8, the labels 'Leber(DrQ-v2)' and 'Leber(DreamerV3)' appear where 'APE(DrQ-v2)' and 'APE(DreamerV3)' are expected; please fix the figure.
  4. [Experiments – Results on Other Benchmarks] Memory Maze results are shown only as training curves in Fig. 7; providing a table of final scores (with variability) would allow quantitative comparison with DreamerV3.
  5. [Appendix C – Implementation Details] No information about code release is provided; sharing the implementation would substantially improve reproducibility of the adaptive schedule and the RL integrations.

Circularity Check

1 steps flagged · score 4.0 of 10

Core adaptive mechanism rests on same-author AdDA citation; RL gains are not isolated from fixed pretraining, but downstream benchmarks provide independent content.

  1. self citation load bearing [Related Works, Contrastive Learning (and Method, Adaptive Pretraining)]
    "AdDA (Zhang, Zhu, and Yu 2023) focuses on exploring the effect of dynamic adjustment on augmentation compositions, which enables the network to acquire more generalizable features. We adopt the feedback structure (Zhang, Zhu, and Yu 2023) in the pretraining period and implement it on a different network architecture, which proves to be more suitable for RL tasks (Yuan et al. 2022)."

    The headline contribution is the adaptive augmentation schedule of Eq. 5. That schedule is imported from the authors' own prior AdDA paper, with overlapping authorship (Y.-H. Zhang and S. Yu), and the only cited justification that dynamic adjustment of augmentation probabilities yields 'more generalizable features' is that same self-citation. The paper's own evidence does not isolate the adaptive mechanism: Table 1 compares adaptive APE variants (with blur or jitter as fmain) against standard MoCo v2 with a different augmentation recipe, and the RL ablation in Fig. 10 varies fmain only within adaptive APE. No fixed-augmentation MoCo v2 encoder pretrained on the same ImageNet-100 subset is evaluated in any downstream RL task.

full rationale

The downstream RL results on DMC, Atari 100k, and Memory Maze are genuine external benchmarks and are not equivalent to any fitted quantity or linear-probe score in the paper, so the overall APE-versus-baseline claim has independent content. The circularity concern is localized to the adaptive mechanism: Eq. 5 updates augmentation probabilities from pretext-task accuracy, and the paper's support for this closed-loop design comes from the same authors' AdDA work. No experiment in the present paper compares APE with a non-adaptive MoCo v2 encoder under the same augmentation recipe, so the observed gains could in principle stem from contrastive pretraining on natural images alone rather than from adaptivity. This makes the self-citation load-bearing for the key novel component, but it does not make the entire derivation circular, because the RL evaluation is externally grounded. Score 4 reflects a central self-citation with partial independent validation.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on several hand-chosen pretraining hyperparameters and on the transfer premise that ImageNet-100 features are useful for RL control. No new ontology is introduced. The most fragile components are alpha and the pretext-accuracy proxy, both of which are APE-specific and not theoretically grounded.

free parameters (5)
  • alpha in Eq. 5 = 0.8 for 7 compositions, 1.0 for 3 compositions
    Controls how aggressively augmentation sampling probabilities are reweighted; values are chosen by hand and stated without sensitivity analysis.
  • main augmentation strategy fmain = random gaussian blur (default probability 0.5)
    The default augmentation recipe is selected based on prior AdDA results and validated on linear probes; other choices underperform.
  • augmentation probabilities fJitter, fBlur, fFlip, fGray = varies, defaults listed in Table 5
    Applied frequencies of color jitter, blur, flip, and grayscale are manually set hyperparameters, not learned from data.
  • number of frozen encoder layers = first 3 of ResNet18 frozen, last layer trainable
    Design choice assumed to balance generalization and adaptation; not derived from theory.
  • sub-batch count N and update schedule = 7 compositions for the main setting, 3 for the comparison setting
    Number of augmentation compositions is a hand-chosen design choice in the adaptive pretraining loop.
assumptions (5)
  • standard math InfoNCE contrastive loss (Eq. 3) and KL-based observation loss (Eq. 6) are valid training objectives.
    Standard objectives from MoCo v2 and DreamerV3; the paper relies on their convergence properties.
  • domain assumption ImageNet-100 natural images provide a useful prior for RL visual observations across DMC, Atari, and Memory Maze.
    This is the central transfer premise; the paper evaluates it empirically but does not prove it.
  • domain assumption Freezing early layers preserves general features while finetuning the last layer adapts to the task distribution.
    Stated in the Policy Learning section; no ablation isolates this design choice.
  • ad hoc to paper The update rule p_{t+1} = Softmax(alpha(1 - Acc_t)) is a reasonable measure of augmentation difficulty.
    The functional form and alpha values are introduced for APE; no derivation or comparison to alternative update rules is given.
  • ad hoc to paper Pretext task top-5 accuracy is a sufficient signal for comparing augmentation strategies.
    Used both to drive Eq. 5 and as the evaluation metric in Table 1; this is the paper's screening assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Efficient Reinforcement Learning Through Adaptively Pretrained Visual Encoder." pith.science (2026). https://pith.science/paper/MDTJICDG

@misc{pith2026250205555,
  author       = {Pith},
  title        = {Pith review of: Efficient Reinforcement Learning Through Adaptively Pretrained Visual Encoder},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MDTJICDG}},
  note         = {Machine review of arXiv:2502.05555}
}
read the original abstract

While Reinforcement Learning (RL) agents can successfully learn to handle complex tasks, effectively generalizing acquired skills to unfamiliar settings remains a challenge. One of the reasons behind this is the visual encoders used are task-dependent, preventing effective feature extraction in different settings. To address this issue, recent studies have tried to pretrain encoders with diverse visual inputs in order to improve their performance. However, they rely on existing pretrained encoders without further exploring the impact of pretraining period. In this work, we propose APE: efficient reinforcement learning through Adaptively Pretrained visual Encoder -- a framework that utilizes adaptive augmentation strategy during the pretraining phase and extracts generalizable features with only a few interactions within the task environments in the policy learning period. Experiments are conducted across various domains, including DeepMind Control Suite, Atari Games and Memory Maze benchmarks, to verify the effectiveness of our method. Results show that mainstream RL methods, such as DreamerV3 and DrQ-v2, achieve state-of-the-art performance when equipped with APE. In addition, APE significantly improves the sampling efficiency using only visual inputs during learning, approaching the efficiency of state-based method in several control tasks. These findings demonstrate the potential of adaptive pretraining of encoder in enhancing the generalization ability and efficiency of visual RL algorithms.

Figures

Figures reproduced from arXiv: 2502.05555 by the authors.

Figure 1
Figure 1. Visualization of ResNet-18 model with different [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. APE pipeline for MBRL. The training phase is divided into two parts, namely the Adaptive Pretraining period (within [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Training curves for DMC vision benchmarks. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Loss comparison between DreamerV3, encoder with frozen random initialized parameters, encoder with trainable [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Exploration of states space in different phases dur [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Training curves for Atari 100k benchmarks. [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Training curves for Memory Maze benchmarks. [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 10
Figure 10. Figure 10: Different choices of augmentation strategy. APE [PITH_FULL_IMAGE:figures/full_fig_p007_10.png]
Figure 11
Figure 11. Figure 11: Different choices of network architectures. This [PITH_FULL_IMAGE:figures/full_fig_p007_11.png]
Figure 1
Figure 1. Figure 1: Memory Maze (Pašukonis, Lillicrap, and Hafner 2022) Agents in this benchmark is repeatedly tasked to navigate through randomized 3D mazes with various objects to reach. To succeed efficiently, agents must remember object locations, maze layouts, and their own positions…
Figure 2
Figure 2. Figure 2: Visualization of reconstructions in different phases during policy learning period of [PITH_FULL_IMAGE:figures/full_fig_p012_2.png]
Figure 3
Figure 3. Figure 3: Results on task with multi-item observations. [PITH_FULL_IMAGE:figures/full_fig_p012_3.png]
Figure 4
Figure 4. Figure 4: Visualization of different initialization of ResNet-18 model using LayerCAM (Jiang et al. 2021). [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

63 extracted references · 29 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    M.; Rupprecht, C.; and Vedaldi, A

    Asano, Y. M.; Rupprecht, C.; and Vedaldi, A. 2019. A critical analysis of self-supervision, or what we can learn from a single image. arXiv preprint arXiv:1904.13132

    arXiv 2019

  4. [4]

    Baevski, A.; Zhou, H.; rahman Mohamed, A.; and Auli, M. 2020. wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations. ArXiv, abs/2006.11477

    arXiv 2020

  5. [5]

    G.; Naddaf, Y.; Veness, J.; and Bowling, M

    Bellemare, M. G.; Naddaf, Y.; Veness, J.; and Bowling, M. 2012. The Arcade Learning Environment: An Evaluation Platform for General Agents. ArXiv, abs/1207.4708

    arXiv 2012

  6. [6]

    Bellman, R. 1957. A Markovian Decision Process. Indiana University Mathematics Journal, 6: 679--684

    work page 1957

  7. [7]

    Brown, N.; Bakhtin, A.; Lerer, A.; and Gong, Q. 2020. Combining Deep Reinforcement Learning and Search for Imperfect-Information Games. ArXiv, abs/2007.13544

    arXiv 2020

  8. [8]

    I.; Yu, T.; Finn, C.; and Hausman, K

    Burns, K.; Witzel, Z.; Hamid, J. I.; Yu, T.; Finn, C.; and Hausman, K. 2023. What Makes Pre-Trained Visual Representations Successful for Robust Manipulation? ArXiv, abs/2312.12444

    arXiv 2023

Show all 63 references
  1. [9]

    Chen, T.; Kornblith, S.; Norouzi, M.; and Hinton, G. E. 2020 a . A Simple Framework for Contrastive Learning of Visual Representations. ArXiv, abs/2002.05709

    2020 arXiv

  2. [10]

    Chen, X.; Fan, H.; Girshick, R.; and He, K. 2020 b . Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003.04297

    2020 arXiv

  3. [11]

    Delfosse, Q.; Stammer, W.; Rothenbacher, T.; Vittal, D.; and Kersting, K. 2022. Boosting Object Representation Learning via Motion and Object Continuity. ArXiv, abs/2211.09771

    2022 arXiv

  4. [12]

    Delfosse, Q.; Sztwiertnia, S.; Stammer, W.; Rothermel, M.; and Kersting, K. 2024. Interpretable Concept Bottlenecks to Align Reinforcement Learning Agents. ArXiv, abs/2401.05821

    2024 arXiv

  5. [13]

    Deng, J.; Dong, W.; Socher, R.; Li, L.-J.; Li, K.; and Fei-Fei, L. 2009 a . ImageNet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, 248--255

    2009

  6. [14]

    Deng, J.; Dong, W.; Socher, R.; Li, L.-J.; Li, K.; and Fei-Fei, L. 2009 b . ImageNet: A large-scale hierarchical image database. 2009 IEEE Conference on Computer Vision and Pattern Recognition, 248--255

    2009

  7. [15]

    Devlin, J.; Chang, M.-W.; Lee, K.; and Toutanova, K. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In North American Chapter of the Association for Computational Linguistics

    2019

  8. [16]

    Du, Y.; Gan, C.; and Isola, P. 2021. Curious Representation Learning for Embodied Intelligence. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 10388--10397

    2021

  9. [17]

    A.; and Brendel, W

    Geirhos, R.; Narayanappa, K.; Mitzkus, B.; Thieringer, T.; Bethge, M.; Wichmann, F. A.; and Brendel, W. 2021. Partial success in closing the gap between human and machine vision. ArXiv, abs/2106.07411

    2021 arXiv

  10. [18]

    Gidaris, S.; Singh, P.; and Komodakis, N. 2018. Unsupervised representation learning by predicting image rotations. arXiv preprint arXiv:1803.07728

    2018 arXiv

  11. [19]

    Ha , D.; and Schmidhuber , J. 2018. World Models . arXiv e-prints, arXiv:1803.10122

    2018 arXiv

  12. [20]

    Haarnoja, T.; Zhou, A.; Hartikainen, K.; Tucker, G.; Ha, S.; Tan, J.; Kumar, V.; Zhu, H.; Gupta, A.; Abbeel, P.; and Levine, S. 2018. Soft Actor-Critic Algorithms and Applications. ArXiv, abs/1812.05905

    2018 arXiv

  13. [21]

    P.; Ba, J.; and Norouzi, M

    Hafner, D.; Lillicrap, T. P.; Ba, J.; and Norouzi, M. 2019. Dream to Control: Learning Behaviors by Latent Imagination. ArXiv, abs/1912.01603

    2019 arXiv

  14. [22]

    P.; Norouzi, M.; and Ba, J

    Hafner, D.; Lillicrap, T. P.; Norouzi, M.; and Ba, J. 2020. Mastering Atari with Discrete World Models. ArXiv, abs/2010.02193

    2020 arXiv

  15. [23]

    Hafner, D.; Pa s ukonis, J.; Ba, J.; and Lillicrap, T. P. 2023. Mastering Diverse Domains through World Models. ArXiv, abs/2301.04104

    2023 arXiv

  16. [24]

    Hansen, N.; Su, H.; and Wang, X. 2021. Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentation. In Neural Information Processing Systems

    2021

  17. [25]

    He, K.; Chen, X.; Xie, S.; Li, Y.; Doll'ar, P.; and Girshick, R. B. 2021. Masked Autoencoders Are Scalable Vision Learners. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 15979--15988

    2021

  18. [26]

    He, K.; Fan, H.; Wu, Y.; Xie, S.; and Girshick, R. 2020. Momentum Contrast for Unsupervised Visual Representation Learning. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 9726--9735

    2020

  19. [27]

    He, K.; Zhang, X.; Ren, S.; and Sun, J. 2015. Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 770--778

    2015

  20. [28]

    Henaff, O. 2020. Data-efficient image recognition with contrastive predictive coding. In International conference on machine learning, 4182--4192. PMLR

    2020

  21. [29]

    Hu, Y.; Wang, R.; Li, L.; and Gao, Y. 2023. For Pre-Trained Vision Models in Motor Control, Not All Policy Learning Methods are Created Equal. In International Conference on Machine Learning

    2023

  22. [30]

    Jiang, P.-T.; Zhang, C.-B.; Hou, Q.; Cheng, M.-M.; and Wei, Y. 2021. LayerCAM: Exploring Hierarchical Class Activation Maps for Localization. IEEE Transactions on Image Processing, 30: 5875--5888

    2021

  23. [31]

    B.; Pion, N.; Weinzaepfel, P.; and Larlus, D

    Kalantidis, Y.; Sariyildiz, M. B.; Pion, N.; Weinzaepfel, P.; and Larlus, D. 2020. Hard negative mixing for contrastive learning. Advances in Neural Information Processing Systems, 33: 21798--21809

    2020

  24. [32]

    Kostrikov, I.; Yarats, D.; and Fergus, R. 2020. Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels. ArXiv, abs/2004.13649

    2020 arXiv

  25. [33]

    Laskin, M.; Lee, K.; Stooke, A.; Pinto, L.; Abbeel, P.; and Srinivas, A. 2020. Reinforcement Learning with Augmented Data. ArXiv, abs/2004.14990

    2020 arXiv

  26. [34]

    Lee, K.; Lee, K.; Shin, J.; and Lee, H. 2019. Network Randomization: A Simple Technique for Generalization in Deep Reinforcement Learning. In International Conference on Learning Representations

    2019

  27. [35]

    Lin, Y.-C.; Zeng, A.; Song, S.; Isola, P.; and Lin, T.-Y. 2020 a . Learning to See before Learning to Act: Visual Pre-training for Manipulation. 2020 IEEE International Conference on Robotics and Automation (ICRA), 7286--7293

    2020

  28. [36]

    Lin, Z.; Wu, Y.-F.; Peri, S.; Sun, W.; Singh, G.; Deng, F.; Jiang, J.; and Ahn, S. 2020 b . SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decomposition. ArXiv, abs/2001.02407

    2020 arXiv

  29. [37]

    Liu, P.; Wang, L.; Ranjan, R.; He, G.; and Zhao, L. 2022. A Survey on Active Deep Learning: From Model Driven to Data Driven. ACM Comput. Surv., 54(10s)

    2022

  30. [38]

    Liu, Q.; Zhou, Q.; Yang, R.; and Wang, J. 2023. Robust Representation Learning by Clustering with Bisimulation Metrics for Visual Reinforcement Learning with Distractions. In AAAI Conference on Artificial Intelligence

    2023

  31. [39]

    Ma, H.; Wu, J.; Feng, N.; Wang, J.; and Long, M. 2023. HarmonyDream: Task Harmonization Inside World Models

    2023

  32. [40]

    J.; Sodhani, S.; Jayaraman, D.; Bastani, O.; Kumar, V.; and Zhang, A

    Ma, Y. J.; Sodhani, S.; Jayaraman, D.; Bastani, O.; Kumar, V.; and Zhang, A. 2022. VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Training. ArXiv, abs/2210.00030

    2022 arXiv

  33. [41]

    P.; and Hafner, D

    Pa s ukonis, J.; Lillicrap, T. P.; and Hafner, D. 2022. Evaluating Long-Term Memory in 3D Mazes. ArXiv, abs/2210.13383

    2022 arXiv

  34. [42]

    Poudel, R. P. K.; Pandya, H.; Liwicki, S.; and Cipolla, R. 2023. ReCoRe: Regularized Contrastive Representation Learning of World Model. ArXiv, abs/2312.09056

    2023 arXiv

  35. [43]

    Raileanu, R.; Goldstein, M.; Yarats, D.; Kostrikov, I.; and Fergus, R. 2021. Automatic Data Augmentation for Generalization in Reinforcement Learning. In Neural Information Processing Systems

    2021

  36. [44]

    Schmid, M.; Moravc \'i k, M.; Burch, N.; Kadlec, R.; Davidson, J.; Waugh, K.; Bard, N.; Timbers, F.; Lanctot, M.; Holland, Z.; Davoodi, E.; Christianson, A.; and Bowling, M. H. 2021. Player of Games. ArXiv, abs/2112.03178

    2021 arXiv

  37. [45]

    P.; and Silver, D

    Schrittwieser, J.; Antonoglou, I.; Hubert, T.; Simonyan, K.; Sifre, L.; Schmitt, S.; Guez, A.; Lockhart, E.; Hassabis, D.; Graepel, T.; Lillicrap, T. P.; and Silver, D. 2019. Mastering Atari, Go, chess and shogi by planning with a learned model. Nature, 588: 604 -- 609

    2019

  38. [46]

    Schwarzer, M.; Rajkumar, N.; Noukhovitch, M.; Anand, A.; Charlin, L.; Hjelm, D.; Bachman, P.; and Courville, A. C. 2021. Pretraining Representations for Data-Efficient Reinforcement Learning. In Neural Information Processing Systems

    2021

  39. [47]

    Shah, R.; and Kumar, V. 2021. RRL: Resnet as representation for Reinforcement Learning. ArXiv, abs/2107.03380

    2021 arXiv

  40. [48]

    Song, X.; Jiang, Y.; Tu, S.; Du, Y.; and Neyshabur, B. 2019. Observational Overfitting in Reinforcement Learning. ArXiv, abs/1912.02975

    2019 arXiv

  41. [49]

    Srinivas, A.; Laskin, M.; and Abbeel, P. 2020. CURL: Contrastive Unsupervised Representations for Reinforcement Learning. ArXiv, abs/2004.04136

    2020 arXiv

  42. [50]

    Stooke, A.; Lee, K.; Abbeel, P.; and Laskin, M. 2020. Decoupling Representation Learning from Reinforcement Learning. ArXiv, abs/2009.08319

    2020 arXiv

  43. [51]

    P.; and Riedmiller, M

    Tassa, Y.; Doron, Y.; Muldal, A.; Erez, T.; Li, Y.; de Las Casas, D.; Budden, D.; Abdolmaleki, A.; Merel, J.; Lefrancq, A.; Lillicrap, T. P.; and Riedmiller, M. A. 2018. DeepMind Control Suite. ArXiv, abs/1801.00690

    2018 arXiv

  44. [52]

    van den Oord, A.; Li, Y.; and Vinyals, O. 2018. Representation Learning with Contrastive Predictive Coding. ArXiv, abs/1807.03748

    2018 arXiv

  45. [53]

    Wang, K.; Kang, B.; Shao, J.; and Feng, J. 2020. Improving Generalization in Reinforcement Learning with Mixture Regularization. ArXiv, abs/2010.10814

    2020 arXiv

  46. [54]

    Xiao, T.; Radosavovic, I.; Darrell, T.; and Malik, J. 2022. Masked Visual Pre-training for Motor Control. ArXiv, abs/2203.06173

    2022 arXiv

  47. [55]

    Yarats, D.; Fergus, R.; Lazaric, A.; and Pinto, L. 2021 a . Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learning. ArXiv, abs/2107.09645

    2021 arXiv

  48. [56]

    Yarats, D.; Fergus, R.; Lazaric, A.; and Pinto, L. 2021 b . Reinforcement Learning with Prototypical Representations. In International Conference on Machine Learning

    2021

  49. [57]

    Yarats, D.; Zhang, A.; Kostrikov, I.; Amos, B.; Pineau, J.; and Fergus, R. 2019. Improving Sample Efficiency in Model-Free Reinforcement Learning from Images. In AAAI Conference on Artificial Intelligence

    2019

  50. [58]

    Yu, T.; Zhang, Z.; Lan, C.; Chen, Z.; and Lu, Y. 2022. Mask-based Latent Reconstruction for Reinforcement Learning. ArXiv, abs/2201.12096

    2022 arXiv

  51. [59]

    Yuan, Z.; Xue, Z.; Yuan, B.; Wang, X.; Wu, Y.; Gao, Y.; and Xu, H. 2022. Pre-Trained Image Encoder for Generalizable Visual Reinforcement Learning. ArXiv, abs/2212.08860

    2022 arXiv

  52. [60]

    Zhan, A.; Zhao, P.; Pinto, L.; Abbeel, P.; and Laskin, M. 2020. Learning Visual Robotic Control Efficiently with Contrastive Pre-training and Data Augmentation. 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 4040--4047

    2020

  53. [61]

    Zhang, W.; Wang, G.; Sun, J.; Yuan, Y.; and Huang, G. 2023. STORM: Efficient Stochastic Transformer based World Models for Reinforcement Learning. ArXiv, abs/2310.09615

    2023 arXiv

  54. [62]

    Zhang, Y.-H.; Zhu, H.; and Yu, S. 2023. Adaptive Data Augmentation for Contrastive Learning. ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 1--5

    2023

  55. [63]

    H.; and Tang, J

    Zhao, Y.; Wu, K.; Yi, T.; Xu, Z.; Ju, X.; Che, Z.; Qiu, Q.; Liu, C. H.; and Tang, J. 2024. An Efficient Generalizable Framework for Visuomotor Policies via Control-aware Augmentation and Privilege-guided Distillation. ArXiv, abs/2401.09258

    2024 arXiv

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.