Pith. sign in

REVIEW 3 major objections 7 minor 4 cited by

Latent Action Learning Requires Supervision in the Presence of Distractors

T0 review · 3 major / 7 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read When distractors are present, unsupervised latent action learning fails, and spending 2.5% action labels during pretraining restores performance, improving downstream returns 4.2x.

desk verdict Solid empirical finding on LAPO failure and supervision benefit under DCS distractors, but the abstract's action-correlated framing exceeds what the benchmark actually tests. read the letter →

arxiv 2502.00379 v5 pith:B5EDEJAR submitted 2025-02-01 cs.CV cs.AIcs.LG

Alexander Nikulin , Ilya Zisman , Denis Tarasov , Nikita Lyubaykin , Andrei Polubarov , Igor Kiselev , Vladislav Kurenkov This is my paper · ORCID
classification cs.CVcs.AIcs.LG
keywords latentactionlearningmodelsdistractorsaction-correlatednoiseinversedynamicsimitationDistractingControlSuitesupervision
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether latent action learning — inferring the actions behind a video from observations alone — still works when the video contains distractors such as moving backgrounds, camera shake, and changing agent appearance. It reports that the standard method, LAPO, produces latent actions that encode distractor dynamics instead of the agent's true actions, and that an improved unsupervised variant, LAOM, still falls short of simple baselines. The central result is that spending a tiny fraction of the dataset's ground-truth action labels during latent action pretraining — as few as 2.5% of trajectories — improves downstream performance 4.2x on average, recovering nearly half of what full behavioral cloning achieves. This challenges the conventional pipeline of first learning latent actions on unlabeled video and only later decoding them to real actions. The question matters because unlabeled web video is the hoped-for fuel for embodied AI, and the paper argues that a small label budget can make that fuel usable.

What carries the argument

The load-bearing object is the latent action model (LAM), a pair of networks: an inverse dynamics model (IDM) that predicts a latent action from observations, and a forward dynamics model (FDM) that predicts the next observation from the current one plus the latent action, jointly trained so that the latent action must compress the change between frames. LAPO implements this with VQ-VAE quantization of latent actions and reconstruction in pixel space. LAOM carries the argument with four changes: a multi-step IDM that predicts the latent action from observations k steps apart with k sampled up to 10; removal of quantization, with the latent action dimension raised to 8192; prediction in a compact latent space via a temporal consistency loss instead of image reconstruction; and training-time augmentations. The supervision mechanism is a linear head that predicts ground-truth actions from latent actions during LAOM training, fed from a small labeled batch of 128 trajectories mixed with the unlabeled batches. Its role is to anchor latent actions to control-relevant features: without it, the IDM's simplicity bias favors encoding distractor noise, because noise explains video dynamics as easily as true actions do.

What would settle it

Train LAOM and LAOM+supervision on a distractor benchmark whose background dynamics react to the agent's actions, rather than the independent background videos used in this paper, and compare downstream normalized return at the 2.5% label budget: if unsupervised LAOM matches LAOM+supervision there, the claim that action-correlated distractors make supervision necessary would be disproven.

Watch

Extended reading notes

Core claim

On distractor-free data, latent action learning works because the agent's true actions are the simplest explanation of frame-to-frame changes, so the model's information bottleneck pushes it to rediscover something close to the action space. The paper shows that once Distracting Control Suite distractors are added, LAPO's latent actions become dominated by noise: with the quantization bottleneck in place, the inverse dynamics model prefers encoding distractor dynamics, which predict the next frame as easily as true actions do. The proposed LAOM removes quantization in favor of high-dimensional continuous latent actions (8192 dimensions), extends the inverse dynamics model to predict from observations up to 10 steps apart, replaces pixel-space reconstruction with a latent temporal consistency loss, and adds training-time augmentations, improving latent action quality 8x by linear probing and roughly doubling downstream returns. The paper's central claim is that supervision is required on top of these changes: linearly predicting ground-truth actions from latent actions during LAOM training, reusing the same label budget (up to 128 labeled trajectories, 2.5% of the dataset) that the conventional pipeline spends only at the decoding stage, improves downstream normalized return 4.2x on average and outperforms every baseline with the same budget. The authors conclude that the conventional pipeline — pretrain the latent action model on unlabeled data, then decode — is suboptimal when distractors are present.

Load-bearing premise

The load-bearing premise is that the Distracting Control Suite distractors — independent background videos, camera shake, and agent color changes — are a fair stand-in for the action-correlated noise of real web video; the paper's own limitations section concedes that real-world noise patterns may differ, and its background videos are not tied to the agent's actions.

Editorial extensions

If this is right

  • When distractors are present, the conventional three-stage pipeline — unsupervised latent action pretraining, behavior cloning on latent actions, then decoding to true actions — is suboptimal: spending the same small label budget during pretraining yields much stronger policies than spending it only at the decoding stage.
  • At 128 labeled trajectories (2.5% of the dataset), LAOM+supervision reaches an average normalized return of 0.44, recovering almost half of behavioral cloning trained on the full labeled dataset.
  • Supervision makes latent actions robust to dimensionality: cutting latent action dimension from 8192 to 64 loses only 16% of performance with supervision, versus 63% without.
  • Latent action learning with supervision generalizes better than IDM relabeling to never-seen distractor background videos at every label budget tested.
  • In cross-embodied pretraining (leave-one-out over the four environments), supervision during pretraining improves performance, though final returns do not beat behavioral cloning trained from scratch on the same labels.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The label-reallocation lesson probably extends beyond latent action models: whenever a pretraining objective has an information bottleneck that noise can saturate, ground-truth labels are worth more spent anchoring the latent space during pretraining than reserved for a final decoding head.
  • Because the paper shows that LAM representations still encode distractors even with supervision, latent action learning is likely unsuited as a stand-alone visual representation pretraining method for control with distractors; supervision grounds the latent actions without cleaning the representations.
  • A testable extension the paper itself gestures at: any signal that ties latent actions to control-relevant features, such as hand tracking in egocentric video, might replace ground-truth actions and deliver a similar improvement.
  • The cross-embodied result — supervision helps but does not beat scratch BC — suggests the benefit of supervision may shrink as pretraining data become more heterogeneous, leaving open whether the 4.2x gain transfers to web-scale video.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. arXiv:2502.00379 studies whether latent action models trained on observations alone remain useful when visual distractors are present. Using the Distracting Control Suite (DCS) on four continuous-control tasks, the authors show that LAPO's latent actions degrade sharply (probe MSE 22.09 with distractors versus 2.55 without). They introduce LAOM, which removes quantization, uses a multi-step inverse dynamics model, a larger latent action dimension, a latent temporal-consistency loss, and augmentations, improving probe MSE by roughly 8x. They then add a linear supervision head from latent actions to ground-truth actions using only 2.5% of labeled transitions during LAM training, reporting a 4.2x average improvement in normalized return over LAOM and outperforming IDM and BC baselines on the DCS environments. The paper concludes that integrating supervision during LAM pretraining is critical when distractors are present.

Significance. The central empirical observation—that supervision during LAM pretraining substantially improves downstream performance under DCS distractors—is useful, falsifiable, and supported by several independent measurements (probe MSE, downstream normalized return, and generalization to novel distractors). The component-wise ablation in Figure 6 is informative, and the release of datasets and code strengthens reproducibility. The broader significance is limited by two issues: DCS distractors are independent of the agent's actions, whereas the paper's motivation concerns action-correlated noise, and key hyperparameters for latent action learning were tuned on the linear-probe metric used to quantify the 8x improvement. In addition, the cross-embodied experiment, which is the closest proxy to the web-scale motivation, shows no gain over BC-from-scratch, a result the authors acknowledge but do not fully carry into their conclusions.

major comments (3)
  1. [Section 3, Section 8] The manuscript repeatedly describes the investigation as concerning 'action-correlated distractors' (Abstract, Section 8), but the DCS configuration used here (60 background videos, camera shake scale 0.1, agent color changes) generates distractors that are statistically independent of the agent's actions. The difficulty was selected in Section 3 as the hardest setting where fully supervised BC still recovers expert performance, and Section 7 concedes that the noise patterns may differ from real-world web video. As a result, the headline claim that supervision is critical for action-correlated distractors is not tested by the reported experiments. Please either implement an action-correlated distractor condition (for example, background or camera motion that depends on the agent's actions) and report results under it, or reframe the abstract, Section 8, and the paper's title-level claims to be about independent/exogenous distractors, explicitly stating that the action-correlated regime remains open.
  2. [Section 3, Figure 6, Table 7] The paper states in Section 3 that hyperparameters for latent action learning were tuned on the final linear probe MSE to real actions. This is the same metric used to report the 8x improvement in Figure 6, and Table 7 shows environment-specific tuning of the labeled loss coefficient (0.01, with 0.001 for cheetah-run). This creates a selection-on-the-evaluation-metric concern for the quantitative claims. Please report sensitivity of probe MSE and downstream normalized return to these hyperparameters, or re-evaluate with a validation split not used for method selection, and state how much of the 8x and 4.2x numbers survive when hyperparameters are fixed across methods.
  3. [Section 5, Figure 10] The cross-embodied pre-training experiment, which is the closest proxy to the web-data motivation, shows that LAOM+supervision does not outperform BC trained from scratch on the same labels (Figure 10). Section 7 already acknowledges this, but the abstract and conclusion still present supervision as critical for robust latent action learning in real-world scenarios. Please temper the broad practical claim and give the cross-embodied negative result a more prominent place in the main text's conclusions, or provide additional evidence that the within-environment benefit transfers to the cross-embodied setting.
minor comments (7)
  1. [Section 1] Typo: 'precense' should be 'presence'. Also, the abstract reports a 4.2x improvement while Section 1 reports 4.3x; make these numbers consistent.
  2. [Figure 4] The label 'FMD' appears in the LAOM panel; this should be 'FDM' to match the text and the rest of the figure.
  3. [Throughout] The phrase 'We access' appears in several places where 'We assess' is meant (for example, Sections 3 and 4). Please correct these occurrences.
  4. [Section 4] The sentence 'We apply then only during latent actions training' should read 'We apply them only during latent action training'.
  5. [Figure 14 caption] The caption says 'For details see Figure 14', which is self-referential; it should refer to Figure 10 or to the relevant section.
  6. [Appendix E] The IDM description says the model is trained on labels from 2 trajectories only, yet Figure 8 shows an IDM curve across all label budgets. Please clarify how the x-axis label budget applies to the IDM baseline.
  7. [Section 5] The phrase 'funetuning' should be 'fine-tuning'.

Circularity Check

1 steps flagged · score 4.0 of 10

Partial circularity: hyperparameters for latent action learning are tuned on the same linear-probe MSE that is then reported as the 8x quality improvement; the core downstream supervision claim is independently measured.

  1. fitted input called prediction [Section 3 (On hyperparameters tuning) and Abstract]
    "We tune the hyperparameters based on online performance for BC, on MSE to real actions on the full dataset for IDM, and on final linear probe MSE to real actions for latent action learning. In more practical tasks, we usually do not have this luxury, but since we are interested in estimating the upper bound performance of each method in a controlled setting, we believe that it is appropriate. ... We propose LAOM, a simple LAPO modification that improves the quality of latent actions by 8x, as measured by linear probing."

    The 8x quality figure is reported as linear-probe MSE to real actions, and the same metric is the stated selection criterion for latent-action-learning hyperparameters. Thus the reported gain is the value of the tuning objective after selection, not an independent measurement; the improvement is in part manufactured by choosing the configuration that minimizes the reported metric. This does not make the downstream normalized-return results circular, because those are environment evaluations not used for tuning, but it does make the headline '8x' claim a fitted input rather than an independent prediction.

full rationale

This is an empirical study, not a derivation chain, so there are no equation-level reductions and no load-bearing self-citation chain. The only circular step I can exhibit is the hyperparameter-selection contamination of the 8x latent-action-quality claim: Section 3 states that latent-action-learning methods are tuned on final linear probe MSE to real actions, and the Abstract reports that same metric as the evidence for the 8x improvement. That fits the fitted-input-called-prediction pattern, though it is disclosed and confined to the quality metric. The central claim that supervision is critical during latent action learning is supported by downstream environment returns under equal label budgets (Section 5, Figures 1 and 8), which are not the tuning metric, so the main conclusion retains independent empirical content. The mismatch between the motivating action-correlated noise and the action-independent DCS distractors is an external-validity limitation, conceded in Section 7, not a circularity, and therefore does not affect this score.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on empirical comparisons, so the main ledger items are domain assumptions about the benchmark proxy and the evaluation metric, plus hyperparameters tuned on the evaluation metric. The benchmark difficulty and architecture choices are the most consequential free parameters.

free parameters (5)
  • DCS distractor difficulty = 60 videos, scale 0.1
    Chosen as 'the hardest setting when BC can still recover expert performance' (Section 3). This selection directly creates the failure regime the paper reports.
  • Latent action dimension = 8192
    Increased from LAPO's 128 to 8192; rationale in Section 4: allows better next-observation prediction; value tuned on probe MSE.
  • Multi-step IDM horizon K = 10
    Section 4: 'we sampled k uniformly for each sample and found that K := 10 worked best' (tuned).
  • Supervision loss coefficient = 0.01 (0.001 for cheetah-run)
    Table 7: 'labeled loss coef 0.01 (0.001, cheetah-run)' is a per-environment tuned hyperparameter.
  • Evaluation episode normalization returns = BC returns per dataset (e.g., 823 cheetah-run with distractors)
    Table 3 reports the normalizing returns; these are empirical baselines, but the choice of normalization affects all reported ratios.
assumptions (5)
  • domain assumption DCS distractors (dynamic background videos, camera shake, color change) are representative of action-correlated distractors in real-world web video
    The paper motivates the problem with real-world action-correlated noise (Misra et al., 2024), but DCS backgrounds are independent of the agent's actions. Section 7 concedes the distribution may differ. This assumption is load-bearing for the practical conclusion.
  • domain assumption Expert trajectories rendered from proprioceptive policies with distractors are valid expert demonstrations
    Section C: experts are trained with proprioceptive states and no distractors, then rendered to 64px images with or without distractors. This assumes the rendering process preserves expert-level behavior.
  • domain assumption Linear probing MSE to ground-truth actions is a valid measure of latent action quality
    Section 3 uses linear probing as the primary evaluation of latent action quality, following Zhang et al. (2022b). The paper also uses this metric to tune hyperparameters.
  • domain assumption The official LAPO implementation is faithfully reproduced and adapted
    Section 4: 'we use similar architecture ... borrowed from the open-source official LAPO implementation'. The reproduction is not independently verified in the paper.
  • domain assumption Ground-truth actions are the correct supervision target for latent actions in control tasks
    The paper evaluates latent actions by their ability to predict ground-truth expert actions. This assumes the goal of LAM pretraining is imitation of the expert action distribution.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Latent Action Learning Requires Supervision in the Presence of Distractors." pith.science (2026). https://pith.science/paper/B5EDEJAR

@misc{pith2026250200379,
  author       = {Pith},
  title        = {Pith review of: Latent Action Learning Requires Supervision in the Presence of Distractors},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/B5EDEJAR}},
  note         = {Machine review of arXiv:2502.00379}
}
read the original abstract

Recently, latent action learning, pioneered by Latent Action Policies (LAPO), have shown remarkable pre-training efficiency on observation-only data, offering potential for leveraging vast amounts of video available on the web for embodied AI. However, prior work has focused on distractor-free data, where changes between observations are primarily explained by ground-truth actions. Unfortunately, real-world videos contain action-correlated distractors that may hinder latent action learning. Using Distracting Control Suite (DCS) we empirically investigate the effect of distractors on latent action learning and demonstrate that LAPO struggle in such scenario. We propose LAOM, a simple LAPO modification that improves the quality of latent actions by 8x, as measured by linear probing. Importantly, we show that providing supervision with ground-truth actions, as few as 2.5% of the full dataset, during latent action learning improves downstream performance by 4.2x on average. Our findings suggest that integrating supervision during Latent Action Models (LAM) training is critical in the presence of distractors, challenging the conventional pipeline of first learning LAM and only then decoding from latent to ground-truth actions.

Figures

Figures reproduced from arXiv: 2502.00379 by the authors.

Figure 1
Figure 1. We show that in the presence of distractors, LAPO strug￾gles to learn latent actions useful for pre-training and that simple BC or IDM are more effective. We propose LAOM, a simple modification that doubles the performance but still underperforms. Thus, we propose to reuse available ground-truth action labels to supervise latent action learning, which significantly improves the performance, achieving normalized scor… view at source ↗
Figure 2
Figure 2. Visualization of the environments from the Distracting Control Suite (DCS) used in our work. Top row: without any distractors, identical to the original DeepMind Control Suite. Bot￾tom row: with distractors, which consists of dynamic background videos, agent color change and camera shaking. See Section 3 for additional details. all changes between observations are mainly and most ef￾ficiently explained by ground tru… view at source ↗
Figure 4
Figure 4. Simplified architecture visualization of LAPO, and LAOM - our proposed modification. LAPO consists of IDM and FMD, both with separate encoders, uses latent action quantization and predict next observation in image space via the decoder in FDM. LAOM incorporates multi-step IDM, removes quantization and does not reconstruct images, relying on latent temporal consistency loss. Images are encoded by shared encoder, whil… view at source ↗
Figures from the paper (10 more)
Figure 5
Figure 5. Figure 5: Quality of latent actions learned by LAPO. We show that quantization of latent actions significantly reduces the quality of actions, even on data without distractors, where LAPO should work without problems. Removing the quantization recovers the latent action quality,…
Figure 7
Figure 7. Figure 7: Performance evaluation of the LAPO and the proposed LAOM with and without distractors. As can be seen, large gap in performance remains in the presence of distractors. Results are averaged across all four environments, each with three random seeds. demonstrating the im…
Figure 8
Figure 8. Figure 8: Performance evaluation of latent action learning approaches and baselines across different budgets of ground-truth action labels. As can be seen, LAPO struggles in the presence of distractors, being outperformed by simpler baselines. LAOM, our modification of LAPO, per…
Figure 9
Figure 9. Figure 9: (a) We show that latent action learning with supervision generalizes better than IDM to novel distractors for all considered budgets of ground-truth action labels available for pre-training. (b)-(c) Supervision with a small number of ground-truth actions during latent …
Figure 10
Figure 10. Figure 10: Evaluation of latent action learning approaches in cross￾embodied pre-training in the presence of distractors, e.g. pre￾training LAM on datasets from three environments and fune-tuning on action labeled data from the remaining one. Supervision dur￾ing latent action pr…
Figure 11
Figure 11. Figure 11: In contrast to IDM, latent action learning encode a lot of control-unrelated information, such as background videos, into the observation representations. This finding suggest that using latent action learning exclusively as a way to pre-train visual repre￾sentations …
Figure 12
Figure 12. Figure 12: We provide additional ablations on the walker environment with three random seeds. (a)-(b) We took the observation embedding from the LAOM visual encoder and trained the linear probe to predict real actions, similar to probing from latent actions. (a) As can be seen, …
Figure 13
Figure 13. Figure 13: Main results without distractors, analogously to our main result in [PITH_FULL_IMAGE:figures/full_fig_p013_13.png]
Figure 14
Figure 14. Figure 14: Mixed-embodied pre-training experiment results for each environment. For details see [PITH_FULL_IMAGE:figures/full_fig_p013_14.png]
Figure 15
Figure 15. Figure 15: Figure summarizing the results from [PITH_FULL_IMAGE:figures/full_fig_p014_15.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. JoyAI-RA 0.5: Scaling Robot Manipulation Learning via Dual Action Alignment

    cs.RO 2026-08 conditional novelty 6.0 of 10

    A dual action alignment framework (latent action world model plus canonical action space) converts heterogeneous human, simulated, and robot data into transferable supervision, with task scores rising monotonically as...

  2. LAVIFT: Latent-Action-Guided Vision Fine-Tuning for Surgical Interaction Recognition

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Latent-action modeling (inverse dynamics plus forward world model) with a patch-level anti-collapse regularizer improves surgical action-triplet recognition and makes encoder change features land more on instrument-ti...

  3. Open-AoE: An Open Egocentric Manipulation Dataset and Toolchain for Embodied Learning

    cs.RO 2026-07 conditional novelty 6.0 of 10

    Open-AoE releases 2,000 hours of smartphone egocentric manipulation video with MANO hand poses, camera trajectories, atomic action labels, and training tools for VLA and world-model pipelines.

  4. Factored Latent Action World Models

    cs.LG 2026-02 conditional novelty 6.0 of 10

    FLAM splits a scene into separate factors, each with its own latent action, and reports better video prediction and downstream policy learning than monolithic latent-action models.

Reference graph

Works this paper leans on

71 extracted references · 30 canonical work pages · cited by 4 Pith papers

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Understanding intermediate layers using linear classifier probes

    Alain, G. Understanding intermediate layers using linear classifier probes. arXiv preprint arXiv:1610.01644, 2016

    arXiv 2016

  3. [3]

    Almuzairee, A., Hansen, N., and Christensen, H. I. A recipe for unbounded data augmentation in visual reinforcement learning. arXiv preprint arXiv:2405.17416, 2024

    work page Pith review arXiv 2024

  4. [4]

    Playing hard exploration games by watching youtube

    Aytar, Y., Pfaff, T., Budden, D., Paine, T., Wang, Z., and De Freitas, N. Playing hard exploration games by watching youtube. Advances in neural information processing systems, 31, 2018

    work page 2018

  5. [5]

    Video pretraining (vpt): Learning to act by watching unlabeled online videos

    Baker, B., Akkaya, I., Zhokov, P., Huizinga, J., Tang, J., Ecoffet, A., Houghton, B., Sampedro, R., and Clune, J. Video pretraining (vpt): Learning to act by watching unlabeled online videos. Advances in Neural Information Processing Systems, 35: 0 24639--24654, 2022

    work page 2022

  6. [6]

    Zero-Shot Generalization of Vision-Based RL Without Data Augmentation

    Batra, S. and Sukhatme, G. S. Zero-shot generalization of vision-based rl without data augmentation. arXiv preprint arXiv:2410.07441, 2024

    work page Pith review arXiv 2024

  7. [7]

    Look where you look! saliency-guided q-networks for generalization in visual reinforcement learning

    Bertoin, D., Zouitine, A., Zouitine, M., and Rachelson, E. Look where you look! saliency-guided q-networks for generalization in visual reinforcement learning. Advances in Neural Information Processing Systems, 35: 0 30693--30706, 2022

    work page 2022

  8. [8]

    Crossq: Batch normalization in deep reinforcement learning for greater sample efficiency and simplicity

    Bhatt, A., Palenicek, D., Belousov, B., Argus, M., Amiranashvili, A., Brox, T., and Peters, J. Crossq: Batch normalization in deep reinforcement learning for greater sample efficiency and simplicity. arXiv preprint arXiv:1902.05605, 2019

    arXiv 1902

Show all 71 references
  1. [9]

    D., Edwards, A., Parker-Holder, J., Shi, Y., Hughes, E., Lai, M., Mavalankar, A., Steigerwald, R., Apps, C., et al

    Bruce, J., Dennis, M. D., Edwards, A., Parker-Holder, J., Shi, Y., Hughes, E., Lai, M., Mavalankar, A., Steigerwald, R., Apps, C., et al. Genie: Generative interactive environments. In Forty-first International Conference on Machine Learning, 2024

    2024

  2. [10]

    and Timofte, R

    Burchi, M. and Timofte, R. Mudreamer: Learning predictive world models without reconstruction. arXiv preprint arXiv:2405.15083, 2024

    2024 arXiv

  3. [11]

    and He, K

    Chen, X. and He, K. Exploring simple siamese representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 15750--15758, 2021

    2021

  4. [12]

    C., Zhao, L., and Bian, J

    Chen, X., Guo, J., He, T., Zhang, C., Zhang, P., Yang, D. C., Zhao, L., and Bian, J. Igor: Image-goal representations are the atomic control units for foundation models in embodied ai. arXiv preprint arXiv:2411.00785, 2024 a

    2024 arXiv

  5. [13]

    Moto: Latent motion token as the bridging language for robot manipulation

    Chen, Y., Ge, Y., Li, Y., Ge, Y., Ding, M., Shan, Y., and Liu, X. Moto: Latent motion token as the bridging language for robot manipulation. arXiv preprint arXiv:2412.04445, 2024 b

    2024

  6. [14]

    Leveraging procedural generation to benchmark reinforcement learning

    Cobbe, K., Hesse, C., Hilton, J., and Schulman, J. Leveraging procedural generation to benchmark reinforcement learning. In International conference on machine learning, pp.\ 2048--2056. PMLR, 2020

    2020

  7. [15]

    J., Pan, H., Iyer, A., Haldar, S., and Pinto, L

    Cui, Z. J., Pan, H., Iyer, A., Haldar, S., and Pinto, L. Dynamo: In-domain dynamics pretraining for visuo-motor control. arXiv preprint arXiv:2409.12192, 2024

    2024 arXiv

  8. [16]

    Dreamerpro: Reconstruction-free model-based reinforcement learning with prototypical representations

    Deng, F., Jang, I., and Ahn, S. Dreamerpro: Reconstruction-free model-based reinforcement learning with prototypical representations. In International conference on machine learning, pp.\ 4956--4975. PMLR, 2022

    2022

  9. [17]

    Imitating latent policies from observation

    Edwards, A., Sahni, H., Schroecker, Y., and Isbell, C. Imitating latent policies from observation. In International conference on machine learning, pp.\ 1755--1763. PMLR, 2019

    2019

  10. [18]

    Learning task informed abstractions

    Fu, X., Yang, G., Agrawal, P., and Jaakkola, T. Learning task informed abstractions. In International Conference on Machine Learning, pp.\ 3480--3491. PMLR, 2021

    2021

  11. [19]

    A., and Levine, S

    Ghosh, D., Bhateja, C. A., and Levine, S. Reinforcement learning from passive data via latent intentions. In International Conference on Machine Learning, pp.\ 11321--11339. PMLR, 2023

    2023

  12. [20]

    Ego4d: Around the world in 3,000 hours of egocentric video

    Grauman, K., Westbury, A., Byrne, E., Chavis, Z., Furnari, A., Girdhar, R., Hamburger, J., Jiang, H., Liu, M., Liu, X., et al. Ego4d: Around the world in 3,000 hours of egocentric video. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\...

    2022

  13. [21]

    Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor

    Haarnoja, T., Zhou, A., Abbeel, P., and Levine, S. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International conference on machine learning, pp.\ 1861--1870. PMLR, 2018

    2018

  14. [22]

    a schel, T., K \

    Hambro, E., Raileanu, R., Rothermel, D., Mella, V., Rockt \"a schel, T., K \"u ttler, H., and Murray, N. Dungeons and data: A large-scale nethack dataset. Advances in Neural Information Processing Systems, 35: 0 24864--24878, 2022

    2022

  15. [23]

    and Wang, X

    Hansen, N. and Wang, X. Generalization in reinforcement learning by soft data augmentation. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pp.\ 13611--13617. IEEE, 2021

    2021

  16. [24]

    Stabilizing deep q-learning with convnets and vision transformers under data augmentation

    Hansen, N., Su, H., and Wang, X. Stabilizing deep q-learning with convnets and vision transformers under data augmentation. Advances in neural information processing systems, 34: 0 3680--3693, 2021

    2021

  17. [25]

    Temporal difference learning for model predictive control

    Hansen, N., Wang, X., and Su, H. Temporal difference learning for model predictive control. arXiv preprint arXiv:2203.04955, 2022

    2022 arXiv

  18. [26]

    Deep residual learning for image recognition

    He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 770--778, 2016

    2016

  19. [27]

    Huang, S., Dossa, R. F. J., Ye, C., Braga, J., Chakraborty, D., Mehta, K., and Ara \ A s jo, J. G. Cleanrl: High-quality single-file implementations of deep reinforcement learning algorithms. Journal of Machine Learning Research, 23 0 (274): 0 1--18, 2022 a

    2022

  20. [28]

    Spectrum random masking for generalization in image-based reinforcement learning

    Huang, Y., Peng, P., Zhao, Y., Chen, G., and Tian, Y. Spectrum random masking for generalization in image-based reinforcement learning. In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. (eds.), Advances in Neural Information Processing Systems, volume ...

    2022

  21. [29]

    Islam, R., Tomar, M., Lamb, A., Efroni, Y., Zang, H., Didolkar, A., Misra, D., Li, X., Van Seijen, H., Combes, R. T. d., et al. Agent-controller representations: Principled offline rl with rich exogenous information. arXiv preprint arXiv:2211.00164, 2022

    2022 arXiv

  22. [30]

    Minerl diamond 2021 competition: Overview, results, and lessons learned

    Kanervisto, A., Milani, S., Ramanauskas, K., Topin, N., Lin, Z., Li, J., Shi, J., Ye, D., Fu, Q., Yang, W., Hong, W., Huang, Z., Chen, H., Zeng, G., Lin, Y., Micheli, V., Alonso, E., Fleuret, F., Nikulin, A., Belousov, Y., Svidchenko, O., and Shpilman, A. Minerl diamond 2021 c...

    2021

  23. [31]

    K., Chen, L

    Khazatsky, A., Pertsch, K., Nair, S., Balakrishna, A., Dasari, S., Karamcheti, S., Nasiriany, S., Srirama, M. K., Chen, L. Y., Ellis, K., et al. Droid: A large-scale in-the-wild robot manipulation dataset. arXiv preprint arXiv:2403.12945, 2024

    2024 arXiv

  24. [32]

    Investigating pre-training objectives for generalization in vision-based reinforcement learning

    Kim, D., Lee, H., Lee, K., Hwang, D., and Choo, J. Investigating pre-training objectives for generalization in vision-based reinforcement learning. arXiv preprint arXiv:2406.06037, 2024 a

    2024 arXiv

  25. [33]

    J., Pertsch, K., Karamcheti, S., Xiao, T., Balakrishna, A., Nair, S., Rafailov, R., Foster, E., Lam, G., Sanketi, P., et al

    Kim, M. J., Pertsch, K., Karamcheti, S., Xiao, T., Balakrishna, A., Nair, S., Rafailov, R., Foster, E., Lam, G., Sanketi, P., et al. Openvla: An open-source vision-language-action model. arXiv preprint arXiv:2406.09246, 2024 b

    2024 arXiv

  26. [34]

    Guaranteed discovery of control-endogenous latent states with multi-step inverse models

    Lamb, A., Islam, R., Efroni, Y., Didolkar, A., Misra, D., Foster, D., Molu, L., Chari, R., Krishnamurthy, A., and Langford, J. Guaranteed discovery of control-endogenous latent states with multi-step inverse models. arXiv preprint arXiv:2207.08229, 2022

    2022 arXiv

  27. [35]

    Multistep inverse is not all you need

    Levine, A., Stone, P., and Zhang, A. Multistep inverse is not all you need. arXiv preprint arXiv:2403.11940, 2024

    2024 arXiv

  28. [36]

    Offline reinforcement learning: Tutorial, review, and perspectives on open problems

    Levine, S., Kumar, A., Tucker, G., and Fu, J. Offline reinforcement learning: Tutorial, review, and perspectives on open problems. arXiv preprint arXiv:2005.01643, 2020

    2005 arXiv

  29. [37]

    Robust representation learning by clustering with bisimulation metrics for visual reinforcement learning with distractions

    Liu, Q., Zhou, Q., Yang, R., and Wang, J. Robust representation learning by clustering with bisimulation metrics for visual reinforcement learning with distractions. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp.\ 8843--8851, 2023 a

    2023

  30. [38]

    Learning world models with identifiable factorization

    Liu, Y., Huang, B., Zhu, Z., Tian, H., Gong, M., Yu, Y., and Zhang, K. Learning world models with identifiable factorization. Advances in Neural Information Processing Systems, 36: 0 31831--31864, 2023 b

    2023

  31. [39]

    A comprehensive survey of data augmentation in visual reinforcement learning

    Ma, G., Wang, Z., Yuan, Z., Wang, X., Yuan, B., and Tao, D. A comprehensive survey of data augmentation in visual reinforcement learning. arXiv preprint arXiv:2210.04561, 2022

    2022 arXiv

  32. [40]

    C., Schmidt, D., Acero, F., Herr, N., Du, Y., Thuruthel, T

    McCarthy, R., Tan, D. C., Schmidt, D., Acero, F., Herr, N., Du, Y., Thuruthel, T. G., and Li, Z. Towards generalist robot learning from internet video: A survey. arXiv preprint arXiv:2404.19664, 2024

    2024 arXiv

  33. [41]

    Finite scalar quantization: Vq-vae made simple

    Mentzer, F., Minnen, D., Agustsson, E., and Tschannen, M. Finite scalar quantization: Vq-vae made simple. arXiv preprint arXiv:2309.15505, 2023

    2023 arXiv

  34. [42]

    Towards principled representation learning from videos for reinforcement learning

    Misra, D., Saran, A., Xie, T., Lamb, A., and Langford, J. Towards principled representation learning from videos for reinforcement learning. arXiv preprint arXiv:2403.13765, 2024

    2024 arXiv

  35. [43]

    Bridging state and history representations: Understanding self-predictive rl

    Ni, T., Eysenbach, B., Seyedsalehi, E., Ma, M., Gehring, C., Mahajan, A., and Bacon, P.-L. Bridging state and history representations: Understanding self-predictive rl. arXiv preprint arXiv:2401.08898, 2024

    2024 arXiv

  36. [44]

    and Taniguchi, T

    Okada, M. and Taniguchi, T. Dreaming: Model-based reinforcement learning by latent imagination without reconstruction. In 2021 ieee international conference on robotics and automation (icra), pp.\ 4209--4215. IEEE, 2021

    2021

  37. [45]

    Dmc-vb: A benchmark for representation learning for control with visual distractors

    Ortiz, J., Dedieu, A., Lehrach, W., Guntupalli, S., Wendelken, C., Humayun, A., Zhou, G., Swaminathan, S., L \'a zaro-Gredilla, M., and Murphy, K. Dmc-vb: A benchmark for representation learning for control with visual distractors. arXiv preprint arXiv:2409.18330, 2024

    2024 arXiv

  38. [46]

    Stable-baselines3: Reliable reinforcement learning implementations

    Raffin, A., Hill, A., Gleave, A., Kanervisto, A., Ernestus, M., and Dormann, N. Stable-baselines3: Reliable reinforcement learning implementations. Journal of Machine Learning Research, 22 0 (268): 0 1--8, 2021. URL http://jmlr.org/papers/v22/20-1364.html

    2021

  39. [47]

    iqrl--implicitly quantized representations for sample-efficient reinforcement learning

    Scannell, A., Kujanp \"a \"a , K., Zhao, Y., Nakhaei, M., Solin, A., and Pajarinen, J. iqrl--implicitly quantized representations for sample-efficient reinforcement learning. arXiv preprint arXiv:2406.02696, 2024

    2024 arXiv

  40. [48]

    Reinforcement learning with videos: Combining offline observations with interaction

    Schmeckpeper, K., Rybkin, O., Daniilidis, K., Levine, S., and Finn, C. Reinforcement learning with videos: Combining offline observations with interaction. arXiv preprint arXiv:2011.06507, 2020

    2011 arXiv

  41. [49]

    and Jiang, M

    Schmidt, D. and Jiang, M. Learning to act without actions. arXiv preprint arXiv:2312.10812, 2023

    2023 arXiv

  42. [50]

    Proximal policy optimization algorithms

    Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  43. [51]

    D., Courville, A., and Bachman, P

    Schwarzer, M., Anand, A., Goel, R., Hjelm, R. D., Courville, A., and Bachman, P. Data-efficient reinforcement learning with self-predictive representations. arXiv preprint arXiv:2007.05929, 2020

    2007 arXiv

  44. [52]

    The pitfalls of simplicity bias in neural networks

    Shah, H., Tamuly, K., Raghunathan, A., Jain, P., and Netrapalli, P. The pitfalls of simplicity bias in neural networks. Advances in Neural Information Processing Systems, 33: 0 9573--9585, 2020

    2020

  45. [53]

    The distracting control suite--a challenging benchmark for reinforcement learning from pixels

    Stone, A., Ramirez, O., Konolige, K., and Jonschkowski, R. The distracting control suite--a challenging benchmark for reinforcement learning from pixels. arXiv preprint arXiv:2101.02722, 2021

    2021 arXiv

  46. [54]

    and Kyrki, V

    Struckmeier, O. and Kyrki, V. Preventing mode collapse when imitating latent policies from observations, 2023. URL https://openreview.net/forum?id=Mf9fQ0OgMzo

    2023

  47. [55]

    A., Zhang, A., and Taylor, M

    Tomar, M., Mishra, U. A., Zhang, A., and Taylor, M. E. Learning representations for pixel-based control: What matters and why? arXiv preprint arXiv:2111.07775, 2021

    2021 arXiv

  48. [56]

    Behavioral cloning from observation

    Torabi, F., Warnell, G., and Stone, P. Behavioral cloning from observation. arXiv preprint arXiv:1805.01954, 2018

    2018 arXiv

  49. [57]

    Recent advances in imitation learning from observation

    Torabi, F., Warnell, G., and Stone, P. Recent advances in imitation learning from observation. arXiv preprint arXiv:1905.13566, 2019

    1905 arXiv

  50. [58]

    Neural discrete representation learning

    Van Den Oord, A., Vinyals, O., et al. Neural discrete representation learning. Advances in neural information processing systems, 30, 2017

    2017

  51. [59]

    Attention is all you need

    Vaswani, A. Attention is all you need. Advances in Neural Information Processing Systems, 2017

    2017

  52. [60]

    Semail: eliminating distractors in visual imitation via separated models

    Wan, S., Wang, Y., Shao, M., Chen, R., and Zhan, D.-C. Semail: eliminating distractors in visual imitation via separated models. In International Conference on Machine Learning, pp.\ 35426--35443. PMLR, 2023

    2023

  53. [61]

    S., Torralba, A., Isola, P., Zhang, A., and Tian, Y

    Wang, T., Du, S. S., Torralba, A., Isola, P., Zhang, A., and Tian, Y. Denoised mdps: Learning world models better than the world itself. arXiv preprint arXiv:2206.15477, 2022

    2022 arXiv

  54. [62]

    Ad3: Implicit action is the key for world models to distinguish the diverse visual distractors

    Wang, Y., Wan, S., Gan, L., Feng, S., and Zhan, D.-C. Ad3: Implicit action is the key for world models to distinguish the diverse visual distractors. arXiv preprint arXiv:2403.09976, 2024

    2024 arXiv

  55. [63]

    Yamada, J., Pertsch, K., Gunjal, A., and Lim, J. J. Task-induced representation learning. arXiv preprint arXiv:2204.11827, 2022

    2022 arXiv

  56. [64]

    Y., et al

    Ye, S., Jang, J., Jeon, B., Joo, S., Yang, J., Peng, B., Mandlekar, A., Tan, R., Chao, Y.-W., Lin, B. Y., et al. Latent action pretraining from videos. arXiv preprint arXiv:2410.11758, 2024

    2024 arXiv

  57. [65]

    Learning invariant representations for reinforcement learning without reconstruction

    Zhang, A., McAllister, R., Calandra, R., Gal, Y., and Levine, S. Learning invariant representations for reinforcement learning without reconstruction. arXiv preprint arXiv:2006.10742, 2020

    2006 arXiv

  58. [66]

    Learning to drive by watching youtube videos: Action-conditioned contrastive policy pretraining

    Zhang, Q., Peng, Z., and Zhou, B. Learning to drive by watching youtube videos: Action-conditioned contrastive policy pretraining. In European Conference on Computer Vision, pp.\ 111--128. Springer, 2022 a

    2022

  59. [67]

    Light-weight probing of unsupervised representations for reinforcement learning

    Zhang, W., GX-Chen, A., Sobal, V., LeCun, Y., and Carion, N. Light-weight probing of unsupervised representations for reinforcement learning. arXiv preprint arXiv:2208.12345, 2022 b

    2022 arXiv

  60. [68]

    Simplified temporal consistency reinforcement learning

    Zhao, Y., Zhao, W., Boney, R., Kannala, J., and Pajarinen, J. Simplified temporal consistency reinforcement learning. In International Conference on Machine Learning, pp.\ 42227--42246. PMLR, 2023

    2023

  61. [69]

    Semi-supervised offline reinforcement learning with action-free trajectories

    Zheng, Q., Henaff, M., Amos, B., and Grover, A. Semi-supervised offline reinforcement learning with action-free trajectories. In International conference on machine learning, pp.\ 42339--42362. PMLR, 2023

    2023

  62. [70]

    Learning robust representation for reinforcement learning with distractions by reward sequence prediction

    Zhou, Q., Wang, J., Liu, Q., Kuang, Y., Zhou, W., and Li, H. Learning robust representation for reinforcement learning with distractions by reward sequence prediction. In Evans, R. J. and Shpitser, I. (eds.), Proceedings of the Thirty-Ninth Conference on Uncertainty in Artific...

    2023

  63. [71]

    Repo: Resilient model-based reinforcement learning by regularizing posterior predictability

    Zhu, C., Simchowitz, M., Gadipudi, S., and Gupta, A. Repo: Resilient model-based reinforcement learning by regularizing posterior predictability. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing Syst...

    2023

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.