Self-Consistent Model-based Adaptation for Visual Reinforcement Learning

Chengyang Ying; Hang Su; Jun Zhu; Xinning Zhou; Yao Feng

REVIEW 3 major objections 6 minor 26 references

Self-Consistent Model-based Adaptation for Visual Reinforcement Learning

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A policy-agnostic denoising model can strip visual distractions from RL observations without fine-tuning the policy.

desk verdict A practically useful policy-agnostic denoising adaptation method with strong empirical results, but the main theorem as stated is false without a Dirac restriction the appendix quietly imposes. read the letter →

arxiv 2502.09923 v1 pith:ECY4AFVA submitted 2025-02-14 cs.CV cs.LG

Xinning Zhou , Chengyang Ying , Yao Feng , Hang Su , Jun Zhu This is my paper

classification cs.CVcs.LG

keywords visualreinforcementlearningdistractionspolicy-agnosticadaptationdenoisingmodelworldunsuperviseddistributionmatchingplug-and-playenhancement

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a single policy-agnostic denoising module can close most of the performance gap that visual reinforcement learning agents suffer when clean-looking observations are swapped for cluttered ones at test time. The module maps cluttered observations to clean ones and is trained with an unsupervised objective, so no paired clean/cluttered data and no modification of the policy's parameters are needed. The theoretical analysis shows that, when the clean-to-cluttered mapping is injective, the solution set of the objective always contains the true denoising map. Experiments on standard continuous-control distractions and on real robot grasping data report that the module restores most of the clean-environment score, works in front of policies trained by different algorithms, and needs far fewer downstream samples than policy fine-tuning. The practical payoff is a reusable adaptation step: one trained denoiser serves many policies and tasks.

What carries the argument

The load-bearing object is the unsupervised KL objective $L_{KL}$, which compares $p(o^n|a)q(o|o^n)$ with $p(o|a)q(o^n|o)$, together with its practical reformulation $L_{SCMA}$. The first term of $L_{SCMA}$ uses a pre-trained world model as a frozen probabilistic simulator of clean observations: denoised observations must be predictable under clean latent dynamics. The second term ties each denoised output back to the observed cluttered input through a noisy model $m_n$, preventing the denoiser from collapsing to any clean-looking but unrelated image. The third term, when rewards are available, forces denoised observations to carry reward-relevant information. The proof that the solution set of $L_{KL}$ is exactly the set of posterior denoising distributions of homogeneous noise functions is what turns this empirical recipe into an inversion result.

What would settle it

Build a synthetic MDP where two clean observations have equal marginal probability and the true noise function swaps them while leaving reward identical; train SCMA and test whether the denoiser outputs the true clean state or the swapped one. If it outputs the swapped state, distribution matching alone does not recover the true inversion, as the homogeneous-noise ambiguity predicts.

Watch

Extended reading notes

Core claim

The paper's central claim is that visual distraction can be handled by a deterministic denoising model $m_{de}$ that inverts the unknown noise function $f_n$, and that this inverse can be learned without paired data by matching distributions over action-conditioned trajectories. The formal setting is a Noisy Partially-Observed Markov Decision Process in which $f_n$ is injective; under that assumption the posterior $p(o_t|o^n_t)$ is a Dirac distribution, so a deterministic denoiser is the right object. SCMA minimizes $L_{SCMA} = L_{sc} + L_n + L_{rew}$, where $L_{sc}$ pushes denoised observations through a frozen pre-trained world model to match the clean dynamics, $L_n$ requires re-noising the denoised observation to reproduce the cluttered input, and $L_{rew}$ predicts rewards from denoised observations. The theory states that the unsupervised objective's solution set equals the posterior denoising distributions of all noise functions homogeneous to $f_n$, which in particular contains the true $p(o_t|o^n_t)$; rewards and architecture constraints shrink that set. The empirical claim is that this suffices: on video-background, moving-camera, color, and occlusion benchmarks, the denoiser narrows the clean-to-distraction gap, and on a real robot grasping pipeline it reduces inverse-dynamics action prediction error.

Load-bearing premise

The theory requires the clean-to-cluttered observation map to be injective, but the paper's occlusion experiment masks a quarter of each frame with a grey rectangle, destroying pixels so that distinct clean observations can merge into one cluttered observation and the theorem no longer applies.

Editorial extensions

If this is right

One trained denoiser can be inserted in front of any policy: the paper reports that adding SCMA to the SGQN policy under occlusion raises average score from 276.0 to 518.8, an 88% increase.
Adaptation needs fewer cluttered samples: SCMA reaches 90% of its final performance in about 10% of the allowed adaptation steps on most video-hard tasks.
Reward-free adaptation is still feasible: dropping the reward loss keeps SCMA above adaptation-based baselines in the video-hard experiments, though the fuller objective performs better.
The denoiser transfers between tasks: a model adapted on walker-walk works on walker-stand and vice versa in the video-hard environment, suggesting reuse across tasks on the same robot.
For real-world deployment, denoised observations reduce the mean-squared-error of an inverse dynamics model's action predictions under background and lighting changes on a real robot grasping dataset.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: because the theory only pins down the true inversion up to homogeneous noise functions, the denoiser may swap clean states that have identical marginal probability under clean dynamics; a synthetic task with symmetric clean states would show how often this ambiguity matters.
Editorial inference: the injectivity assumption is the real boundary of the method; attaching a stochastic or set-valued forward model would be a natural way to extend the guarantee to occlusions, which the paper's own grey-mask experiment does not cover.
Editorial inference: rewards and architecture are tie-breakers among homogeneous noise functions, so an adversarial distraction that keeps reward marginals unchanged would test whether visual fidelity alone is enough for task performance.
Editorial inference: since the frozen world model and denoiser do not depend on the policy, the same module could be trained once on diverse cluttered trajectories and shipped as a pre-processing layer for many robots and tasks.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick read on arXiv:2502.09923 (SCMA). The core idea is simple and useful: train a plug-and-play denoising network that maps cluttered observations to clean ones by matching the distribution of a pre-trained world model's latents, plus a cycle/noisy reconstruction term and reward prediction. That combination is genuinely new relative to PAD/MoVie/SVEA, and the empirical package is solid: DMControl, RL-ViGen, ablations, zero-shot across tasks, plug-and-play with SGQN, and a real-robot IDM demo. The gains over adaptation baselines are large in video-hard and occlusion settings and hold with only 3 seeds but with consistent margins. Credit where due: the authors also test the no-reward variant and report wall-clock time to 90% performance.

The soft spots, in order of importance. First, Theorem 1 as stated in the main text is false. The solution set of LKL over all conditional distributions includes every coupling with matching marginals, not just posteriors of injective homogeneous noise functions. The appendix proof only goes through after implicitly restricting q(o|on) and q(on|o) to Dirac distributions. That restriction is exactly what the practical denoiser uses, so the useful version of the theorem holds, but the advertised characterization is not correct as written and needs a caveat in the theorem statement. The reader's Pith Report caught this; the stress-test note is right. Second, the occlusion experiments randomly mask 1/4 of the frame, which breaks the injectivity assumption that underlies the theory. The method still works empirically, which is nice, but the theory does not cover that setting. Third, minor: the real-robot results in Table 6 report no error bars, and there is no public code repository or commit, just "provided in supplementary." For a method whose value is practical, reproducibility matters.

On citation pattern: the related work is fair and the self-citations are not load-bearing. One quibble: the claim that existing adaptation-based methods show narrow empirical increases is supported by the table, but some baseline scores are taken from other papers rather than re-run; the authors note this, but it still weakens the comparison.

Overall: the central empirical claim holds up; the theoretical argument needs a corrected statement and the paper needs code before it is fully established. It deserves a serious referee. I'd accept it for review and ask for a rewritten Theorem 1, error bars on the real-robot data, and a code release.

Referee Report

3 major / 6 minor

Summary. The paper proposes SCMA, a policy-agnostic denoising module for visual reinforcement learning under distraction. SCMA trains a denoising model to map cluttered observations to clean ones, using an unsupervised distribution-matching objective. The authors formalize the setting as a Noisy Partially-Observed Markov Decision Process, derive the objective LKL, state Theorem 1 about its solution set, and then replace intractable terms with a frozen pre-trained world model, obtaining the practical loss L_SCMA = L_sc + L_n + L_rew. Experiments on DMControlGB, DMControlView, RL-ViGen, and real Mobile ALOHA robot data show consistent gains over baselines, with ablations on loss components, cross-task transfer, and plug-and-play combination with an SGQN policy.

Significance. If the empirical results hold, SCMA is a practically useful module: it is policy-agnostic, requires no policy fine-tuning, and improves performance across several distraction types and on a real-robot inverse dynamics task. The paper ships a broad empirical study with ablations, cross-task transfer tests, and a real-world robot evaluation, which are genuine strengths. The sample-efficiency claims in Section 4.5 are supported by the learning curves. However, the advertised theoretical analysis has a load-bearing flaw: Theorem 1 is false as stated, and the practical objective is only an approximate surrogate of the analyzed LKL. These issues weaken the theoretical foundation but do not invalidate the empirical contribution.

major comments (3)

[Section 3.2, Theorem 1 and Appendix A.3] Theorem 1 is false as stated. The theorem claims that the solution set Q of LKL equals the set of posterior denoising distributions of homogeneous noise functions. The proof in Appendix A.3, however, explicitly restricts q(o|on) and q(on|o) to Dirac distributions ('q* also constrained to be Dirac'), while the theorem statement does not. Without this restriction, any coupling with the correct marginals achieves LKL=0. For example, with O=On={o1,o2} and uniform marginals, the uniform coupling q(o|on)=1/2 minimizes LKL, but it is not the posterior of any injective homogeneous noise function. This matters because the abstract and introduction motivate L_SCMA through Theorem 1; as written, the theoretical guarantee does not follow. Please restate Theorem 1 with the Dirac parameterization made explicit, and clarify that the equality holds only within the deterministic class, or provide a correct characterization of the full stochastic solution set.
[Section 3.3 and Appendix A.4, Eq. (4) and Eq. (11)] The practical objective L_SCMA is not the LKL analyzed in Theorem 1. In Eq. (11), -log p(o1:T|a1:T) is replaced by a world-model ELBO, and then the KL term DKL(qwm||pwm) is dropped. The derivation gives an upper bound before the drop; after dropping, the objective is an additional approximation. Thus Theorem 1 does not directly justify the actual optimized loss. The paper does explain the empirical rationale for dropping the KL term, but the theoretical connection between LKL and L_SCMA is not established. Please state explicitly that L_SCMA is an approximate surrogate of LKL and discuss what, if anything, the optimality analysis guarantees for the actual objective.
[Section 4.2, Table 1d and Appendix B.1] The occlusion experiment randomly covers 1/4 of each observation with a grey rectangle. This makes the noise function fn non-injective: different clean observations can map to the same cluttered observation because the masked region contains no information about the underlying clean pixels. This violates the injectivity assumption stated in Section 3.1 and used in Appendix A.1 and A.3 to guarantee that the posterior p(o|on) is Dirac and that Theorem 1 applies. Since the occlusion setting is one of the headline benchmark results, please either extend the analysis to non-injective noise or explicitly state that this experiment falls outside the theoretical guarantee. The empirical success is valuable, but it currently lacks a theoretical explanation.

minor comments (6)

[Section 3.2] The definition of LKL writes q(on1:T|o1:T) = ∏_t q(ot|on_t); this should be ∏_t q(on_t|ot).
[Figure 1 caption] The caption says 'provised'; it should be 'provided'.
[Definition 1] The phrase 'ad defined below' should be 'as defined below'.
[Figure 15 caption] The caption mentions 'visual loss Lt_visual', 'mask penalty loss Lt_reg', but the text in Section 4.4 refers to 'self-consistent reconstruction loss Lt_sc', 'reward prediction loss Lt_rew', and 'noisy reconstruction loss Lt_n'. Please align the caption with the text.
[Table 5] The table reports 'Time/Episode' but the associated text describes 'adaptation steps' and 'total adaptation time-steps'; please clarify the units and whether 'episode' means environment episodes or gradient update steps.
[Section 4.2] The paper describes the moving-view and color-hard results from Table 1 as 'competitive' with baselines, but for several tasks the differences are within one standard deviation; a statistical comparison or caveat would help interpret the results.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the denoiser is evaluated against external policies, benchmarks, and real robot data; the world-model-based 'clean' target is a stated approximation, not a construction that forces the claimed results.

full rationale

The paper's central claim is that optimizing L_SCMA trains a denoising model that transfers cluttered observations to clean ones, and this is supported empirically on DMControl, RL-ViGen, and Mobile ALOHA data. The evaluation is external: the denoising model is combined with pre-trained policies and its output is scored by task return or IDM action-prediction error, not by a quantity that was itself fitted into the objective. The unsupervised objective L_KL is derived from KL divergence between joint distributions, and Theorem 1 is presented as a mathematical characterization of its solution set; it is not a fitted parameter renamed as a prediction. The use of a pre-trained world model to estimate p(o1:T | a1:T ) is an explicit modeling approximation, and the authors openly acknowledge the underdetermination caused by homogeneous noise functions, which is an honest limitation rather than a circular step. The only self-citation (Ying et al., 2024) appears in a list of related challenges and is not load-bearing. One non-circular correctness concern should be noted: Appendix A.3 restricts the proof of Theorem 1 to Dirac q distributions ('q*(o|on), q*(on|o) are also constrained to be Dirac distributions') while the main-text theorem statement omits that restriction, so the theorem as stated is not fully supported. This is a proof gap, not a case of the derivation reducing to its own inputs by construction. Overall, the derivation chain does not equate a prediction with its input or rely on a self-citation chain, so the circularity score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No scalar quantities are hand-fitted to the benchmark scores; all hyperparameters are inherited from the Dreamer repository. The core assumptions are the injective noise function, the transferability of the clean-environment world model to score denoised images, and the empirically motivated drop of the KL term. No new physical entities or mediators are postulated.

assumptions (4)

domain assumption The noise function f_n is injective, so cluttered observations preserve all information about clean observations.
Assumed in Sec. 3.1 and Appendix A.1; used to prove the posterior denoising distribution is a Dirac delta and that supervised denoising is well-posed. It is violated by the occlusion experiment, where a random mask removes 1/4 of the pixels.
domain assumption A world model pre-trained on clean environments provides an accurate estimate of the action-conditioned clean observation distribution p(o1:T|a1:T), and its likelihood can evaluate denoised images.
The practical losses in Sec. 3.3 replace -log p(o1:T|a1:T) with the world model's ELBO; if the world model cannot judge out-of-distribution denoised images, Lsc may reward arbitrary clean-looking images.
ad hoc to paper Dropping the KL term DKL(qwm||pwm) from the objective does not change the optimum in the intended direction.
Justified only empirically in Appendix A.4 ('we empirically find it to have a negative impact'), so the practical objective is an upper-bound approximation, not exactly the theoretically analyzed LKL.
domain assumption The reward signal is available in distracting environments and the reward predictor transfers across clean and denoised observations.
Used to reduce homogeneous noise functions via Lrew (Sec. 3.4); required for the best variant, while the ablated no-reward variant is weaker but still competitive.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Self-Consistent Model-based Adaptation for Visual Reinforcement Learning." pith.science (2026). https://pith.science/paper/ECY4AFVA

@misc{pith2026250209923,
  author       = {Pith},
  title        = {Pith review of: Self-Consistent Model-based Adaptation for Visual Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ECY4AFVA}},
  note         = {Machine review of arXiv:2502.09923}
}

read the original abstract

Visual reinforcement learning agents typically face serious performance declines in real-world applications caused by visual distractions. Existing methods rely on fine-tuning the policy's representations with hand-crafted augmentations. In this work, we propose Self-Consistent Model-based Adaptation (SCMA), a novel method that fosters robust adaptation without modifying the policy. By transferring cluttered observations to clean ones with a denoising model, SCMA can mitigate distractions for various policies as a plug-and-play enhancement. To optimize the denoising model in an unsupervised manner, we derive an unsupervised distribution matching objective with a theoretical analysis of its optimality. We further present a practical algorithm to optimize the objective by estimating the distribution of clean observations with a pre-trained world model. Extensive experiments on multiple visual generalization benchmarks and real robot data demonstrate that SCMA effectively boosts performance across various distractions and exhibits better sample efficiency.

Figures

Figures reproduced from arXiv: 2502.09923 by the authors.

**Figure 1.** The graphical model of a NPOMDP, where ot and o n t denote the clean and cluttered observation respectively. 3 Methodology We first present our problem formulation and the supervised objective LO in Sec. 3.1. Then we introduce an unsupervised distribution matching surrogate LKL and analyze the connection between LKL and LO in Sec. 3.2. Finally, we transform LKL into several optimizable adaptation losses in Sec. 3.3… view at source ↗

**Figure 2.** An overview of Self-Consistent Model-based Adaption (SCMA). SCMA adapts the agent to distracting environments by transferring cluttered observations to clean ones with the denoising model mde. Leveraging a pre-trained world model, mde can be efficiently optimized with self-consistent reconstruction, noisy reconstruction, and reward prediction loss. We further show that p(ot|o n t ) is a Dirac distribution when fn i… view at source ↗

**Figure 3.** Visualization of the raw observations and the denoising model’s outputs in various distracting environments. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Figures from the paper (9 more)

**Figure 4.** Figure 4: Performance curves of different algorithms in the [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

**Figure 5.** Figure 5: Visualization of the raw observations and denoising [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

**Figure 7.** Figure 7: An overview of involved environments with Robosuite. [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]

**Figure 9.** Figure 9: Adaptation performance of SCMA in the video hard environment. 0.5 1.0 Step ×10 5 0 250 500 750 1000 Episode Return ball in cup-catch 0.5 1.0 Step ×10 5 cartpole-swingup 0.5 1.0 Step ×10 5 finger-spin 0.5 1.0 Step ×10 5 walker-stand 0.5 1.0 Step ×10 5 walker-walk [PIT…

**Figure 8.** Figure 8: Average performance of SCMA, SCMA (w/o r), and other adaptation-based baselines across [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]

**Figure 12.** Figure 12: Adaptation performance of SCMA in the occlusion environment. B.3 Adaptation without Rewards We report the detailed performance of SCMA, SCMA (w/o r), and other adaptation-based baselines in 4 different distracting environments, where SCMA (w/o r) means removing Lrew…

**Figure 14.** Figure 14: Adaptation performance of SCMA in the eval extreme environment in RL-ViGen [PITH_FULL_IMAGE:figures/full_fig_p014_14.png]

**Figure 15.** Figure 15: Ablation for different loss components’ effects on the adaptation results in [PITH_FULL_IMAGE:figures/full_fig_p015_15.png]

**Figure 16.** Figure 16: Visualization of SCMA in different distracting environ [PITH_FULL_IMAGE:figures/full_fig_p016_16.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

26 extracted references · 9 canonical work pages

[1]

Unsupervised neural machine translation

Mikel Artetxe, Gorka Labaka, Eneko Agirre, and Kyunghyun Cho. Unsupervised neural machine translation. arXiv preprint arXiv:1710.11041,

arXiv
[5]

Generalization in re- inforcement learning by soft data augmentation

Nicklas Hansen and Xiaolong Wang. Generalization in re- inforcement learning by soft data augmentation. In 2021 IEEE International Conference on Robotics and Automa- tion (ICRA), pages 13611–13617. IEEE,

work page 2021
[8]

Unsupervised translation of pro- gramming languages

Marie-Anne Lachaux, Baptiste Roziere, Lowik Chanussot, and Guillaume Lample. Unsupervised translation of pro- gramming languages. arXiv preprint arXiv:2006.03511 ,

arXiv 2006
[9]

1 2 3 4 Step ×105 0 250 500 750 1000Episode Return ball in cup-catch 1 2 3 4 Step ×105 cartpole-swingup 1 2 3 4 Step ×105 ﬁnger-spin 1 2 3 4 Step ×105 walker-stand 1 2 3 4 Step ×105 walker-walk Figure 9: Adaptation performance of SCMA in the video hard en- vironment. 0.5 1.0 Step ×105 0 250 500 750 1000Episode Return ball in cup-catch 0.5 1.0 Step ×105 ca...

work page 2024
[11]

R3m: A universal visual representation for robot manipulation

Suraj Nair, Aravind Rajeswaran, Vikash Kumar, Chelsea Finn, and Abhinav Gupta. R3m: A universal visual representation for robot manipulation. arXiv preprint arXiv:2203.12601,

arXiv
[12]

Isolating and leveraging controllable and noncon- trollable visual dynamics in world models

Minting Pan, Xiangming Zhu, Yunbo Wang, and Xiaokang Yang. Isolating and leveraging controllable and noncon- trollable visual dynamics in world models. arXiv preprint arXiv:2205.13817,

arXiv
[13]

Sumers, Kenneth Marino, Arun Ahuja, Rob Fergus, and Ishita Dasgupta

Theodore R. Sumers, Kenneth Marino, Arun Ahuja, Rob Fergus, and Ishita Dasgupta. Distilling internet-scale vision-language models into embodied agents. ArXiv, abs/2301.12507,

arXiv
[14]

Deepmind control suite

Yuval Tassa, Yotam Doron, Alistair Muldal, Tom Erez, Yazhe Li, Diego de Las Casas, David Budden, Abbas Abdol- maleki, Josh Merel, Andrew Lefrancq, et al. Deepmind control suite. arXiv preprint arXiv:1801.00690,

arXiv

Show all 26 references

[16]

A survey on unsu- pervised transfer clustering

Feng Wang, Lianmeng Jiao, and Quan Pan. A survey on unsu- pervised transfer clustering. In 2021 40th Chinese Control Conference (CCC), pages 7361–7365. IEEE,

2021
[17]

Drivedreamer: Towards real-world-driven world models for autonomous driving

Xiaofeng Wang, Zheng Zhu, Guan Huang, Xinze Chen, and Jiwen Lu. Drivedreamer: Towards real-world-driven world models for autonomous driving. arXiv preprint arXiv:2309.09777,

arXiv
[18]

Peac: Unsu- pervised pre-training for cross-embodiment reinforcement learning

Chengyang Ying, Zhongkai Hao, Xinning Zhou, Xuezhou Xu, Hang Su, Xingxing Zhang, and Jun Zhu. Peac: Unsu- pervised pre-training for cross-embodiment reinforcement learning. arXiv preprint arXiv:2405.14073,

arXiv
[19]

Egsde: Unpaired image-to-image translation via energy-guided stochastic differential equations

Min Zhao, Fan Bao, Chongxuan Li, and Jun Zhu. Egsde: Unpaired image-to-image translation via energy-guided stochastic differential equations. ArXiv, abs/2207.06635,

arXiv
[21]

robosuite: A modular simulation framework and benchmark for robot learning

Yuke Zhu, Josiah Wong, Ajay Mandlekar, Roberto Mart ´ın- Mart´ın, Abhishek Joshi, Soroush Nasiriany, and Yifeng Zhu. robosuite: A modular simulation framework and benchmark for robot learning. arXiv preprint arXiv:2009.12293,

2009 arXiv
[22]

A Theoretical Analyses In this section, we will provide proof of all our theoretical results in detail. A.1 Noisy Partially-Observed Markov Decision Process For NPOMDP Mn = ⟨S, O, A, T , R, γ, ρ0, fn⟩, the action- conditioned joint distribution is defined as following: p(o1:T,...

2021
[23]

A.4 Self-Consistent Model-based Adaptation Below we provide a detailed derivation of SCMA’s adaptation loss

As a result, it is clear that fn1 is homogeneous to fn with- out rewards, yet it is no longer homogeneous to fn with re- wards. A.4 Self-Consistent Model-based Adaptation Below we provide a detailed derivation of SCMA’s adaptation loss. From Eq. 8, LKL leads to the following o...

2017
[25]

Following Yuan et al

For SCMA, the agent is trained in clean environments for 0.5M timesteps and then adapts to visually distracting environments for an- other 0.5M timesteps. Following Yuan et al. [2024], we eval- uate each trained agent with10 trails on each scene (100 trails in total) and repor...

2024
[26]

(a) video hard (b) moving view (c) color hard (d) occlusionw Figure 16: Visualization of SCMA in different distracting environ- ments. The columns from left to right separately represent (1) clut- tered observations (2) outputs of the denoising model mde (3) out- puts of the n...

2020
[2015]

Learning to act from actionless videos through dense correspondences

Po-Chen Ko, Jiayuan Mao, Yilun Du, Shao-Hua Sun, and Josh Tenenbaum. Learning to act from actionless videos through dense correspondences. ArXiv, abs/2310.08576,

arXiv
[2017]

Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros. Unpaired image-to-image translation using cycle- consistent adversarial networks. 2017 IEEE International Conference on Computer Vision (ICCV) , pages 2242– 2251,

2017
[2018]

Learning representations for pixel- based control: What matters and why? arXiv preprint arXiv:2111.07775,

Manan Tomar, Utkarsh A Mishra, Amy Zhang, and Matthew E Taylor. Learning representations for pixel- based control: What matters and why? arXiv preprint arXiv:2111.07775,

arXiv
[2019]

Mastering diverse domains through world mod- els

Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy Lillicrap. Mastering diverse domains through world mod- els. arXiv preprint arXiv:2301.04104,

arXiv
[2020]

Crossloco: Human motion driven control of legged robots via guided unsu- pervised reinforcement learning

Tianyu Li, Hyunyoung Jung, Matthew Gombolay, Yong Kwon Cho, and Sehoon Ha. Crossloco: Human motion driven control of legged robots via guided unsu- pervised reinforcement learning. ArXiv, abs/2309.17046,

arXiv
[2021]

Abbeel, Alexei A

Nicklas Hansen, Yu Sun, P. Abbeel, Alexei A. Efros, Lerrel Pinto, and Xiaolong Wang. Self-supervised policy adapta- tion during deployment. ArXiv, abs/2007.04309,

2007 arXiv
[2022]

Tenenbaum, Dale Schuurmans, and P

Yilun Du, Mengjiao Yang, Bo Dai, Hanjun Dai, Ofir Nachum, Joshua B. Tenenbaum, Dale Schuurmans, and P. Abbeel. Learning universal policies via text-guided video genera- tion. ArXiv, abs/2302.00111,

arXiv
[2023]

Dream to control: Learning behaviors by la- tent imagination

Danijar Hafner, Timothy Lillicrap, Jimmy Ba, and Moham- mad Norouzi. Dream to control: Learning behaviors by la- tent imagination. arXiv preprint arXiv:1912.01603,

1912 arXiv
[2024]

Cross-domain random pre-training with prototypes for reinforcement learning

Xin Liu, Yaran Chen, Haoran Li, Boyu Li, and Dong- bin Zhao. Cross-domain random pre-training with prototypes for reinforcement learning. arXiv preprint arXiv:2302.05614,

arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Unsupervised neural machine translation

Mikel Artetxe, Gorka Labaka, Eneko Agirre, and Kyunghyun Cho. Unsupervised neural machine translation. arXiv preprint arXiv:1710.11041,

arXiv

[2] [5]

Generalization in re- inforcement learning by soft data augmentation

Nicklas Hansen and Xiaolong Wang. Generalization in re- inforcement learning by soft data augmentation. In 2021 IEEE International Conference on Robotics and Automa- tion (ICRA), pages 13611–13617. IEEE,

work page 2021

[3] [8]

Unsupervised translation of pro- gramming languages

Marie-Anne Lachaux, Baptiste Roziere, Lowik Chanussot, and Guillaume Lample. Unsupervised translation of pro- gramming languages. arXiv preprint arXiv:2006.03511 ,

arXiv 2006

[4] [9]

1 2 3 4 Step ×105 0 250 500 750 1000Episode Return ball in cup-catch 1 2 3 4 Step ×105 cartpole-swingup 1 2 3 4 Step ×105 ﬁnger-spin 1 2 3 4 Step ×105 walker-stand 1 2 3 4 Step ×105 walker-walk Figure 9: Adaptation performance of SCMA in the video hard en- vironment. 0.5 1.0 Step ×105 0 250 500 750 1000Episode Return ball in cup-catch 0.5 1.0 Step ×105 ca...

work page 2024

[5] [11]

R3m: A universal visual representation for robot manipulation

Suraj Nair, Aravind Rajeswaran, Vikash Kumar, Chelsea Finn, and Abhinav Gupta. R3m: A universal visual representation for robot manipulation. arXiv preprint arXiv:2203.12601,

arXiv

[6] [12]

Isolating and leveraging controllable and noncon- trollable visual dynamics in world models

Minting Pan, Xiangming Zhu, Yunbo Wang, and Xiaokang Yang. Isolating and leveraging controllable and noncon- trollable visual dynamics in world models. arXiv preprint arXiv:2205.13817,

arXiv

[7] [13]

Sumers, Kenneth Marino, Arun Ahuja, Rob Fergus, and Ishita Dasgupta

Theodore R. Sumers, Kenneth Marino, Arun Ahuja, Rob Fergus, and Ishita Dasgupta. Distilling internet-scale vision-language models into embodied agents. ArXiv, abs/2301.12507,

arXiv

[8] [14]

Deepmind control suite

Yuval Tassa, Yotam Doron, Alistair Muldal, Tom Erez, Yazhe Li, Diego de Las Casas, David Budden, Abbas Abdol- maleki, Josh Merel, Andrew Lefrancq, et al. Deepmind control suite. arXiv preprint arXiv:1801.00690,

arXiv