Pith. sign in

REVIEW 4 major objections 6 minor 2 cited by

CausalCOMRL: Context-Based Offline Meta-Reinforcement Learning with Causal Representation

T0 review · 4 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a task encoder built on a learned causal graph among state, action, and reward components generalizes better in offline meta-RL than encoders that just fit correlations.

desk verdict A reasonable empirical paper with an overstated causal claim: the gains are real but the causal mechanism is not demonstrated. read the letter →

arxiv 2502.00983 v1 pith:2GTSE74V submitted 2025-02-03 cs.LG stat.ML

Zhengzhe Zhang , Wenjia Meng , Haoliang Sun , Gang Pan This is my paper · ORCID
classification cs.LGstat.ML
keywords offlinemeta-reinforcementlearningcontext-basedcausalrepresentationtaskencoderout-of-distributiongeneralizationvariationalautoencodercontrastivemutualinformation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether a context encoder for offline meta-reinforcement learning can be made robust to spurious correlations by building its task representation from a causal model. It proposes CausalCOMRL, whose task encoder is a causal variational autoencoder: a context trajectory is mapped to a latent code through the linear structural causal model $z=(I-A^T)^{-1}\epsilon$, with a learned adjacency matrix $A$ constrained to be a directed acyclic graph. The paper claims this injects causal relationships among state, action, and reward components into the representation, so the meta-policy generalizes when the confounders in the test task differ from those in training. It reports that CausalCOMRL outperforms Offline-PEARL, FOCAL, and CORRO on most of four continuous-control benchmarks, with the largest out-of-distribution gains on Hopper-Rand-Params and Walker-Rand-Params.

What carries the argument

The carrying mechanism is the causal variational autoencoder task encoder, specifically the structural equation $z=(I-A^T)^{-1}\epsilon$. The adjacency matrix $A$ is the object that is supposed to be causal: each entry $A_{ji}$ gives the causal strength from latent component $z_j$ to $z_i$. Two constraints make it learnable as a graph: the task-information constraint $\mathbb{E}_{\mathcal{D}}\lVert u-\sigma(A^T u)\rVert_2^2 \le k_1$, which ties the graph to task information $u$, and the DAG constraint $\mathrm{tr}((I+\frac{c}{n}A\circ A)^n)-n=0$, which keeps the graph acyclic. The latent variable $z$ produced by this layer is the task representation that conditions the policy, so this layer is where causal structure enters the learning pipeline.

What would settle it

Train CausalCOMRL on a synthetic MDP whose state, action, and reward variables are generated from a known linear structural causal model, then compare the learned adjacency matrix to the true graph and measure behavior under an intervention that changes one edge; if the learned graph disagrees with the ground truth or the out-of-distribution gain disappears, the causal claim is falsified.

Watch

Extended reading notes

Core claim

The central claim is that an encoder which learns the causal graph among task components, rather than raw statistical correlations, produces task representations that transfer better under distribution shift. In the proposed architecture, the encoder outputs $\epsilon$, and a causal layer transforms it by the linear structural causal model $z=(I-A^T)^{-1}\epsilon$, where the adjacency matrix $A$ encodes causal strengths between latent task components. Training enforces two constraints on $A$: a task-information constraint tying the graph to task-specific statistics, and a DAG constraint guaranteeing acyclicity. The resulting representation, further separated across tasks by mutual-information and contrastive losses, conditions a Soft Actor-Critic meta-policy. On four continuous-control benchmarks, the paper reports higher average returns than the compared context encoders on most tasks, and attributes the improvement to the causal structure rather than to the auxiliary losses.

Load-bearing premise

The load-bearing premise is that the matrix of connections the encoder learns between task components is the true causal structure, not just a convenient pattern that fits the offline data.

Editorial extensions

If this is right

  • Context-based OMRL encoders that enforce a causal latent structure should generalize better under confounder shift than encoders trained only with reconstruction, mutual-information, or contrastive objectives.
  • The learned task representation separates tasks more cleanly, as the paper's embedding visualizations show, which should make policy conditioning more reliable on unseen tasks.
  • The causal encoder's largest gains appear on benchmarks whose transition dynamics vary, suggesting the benefit is strongest when the mechanism, not just the reward, changes.
  • The causal layer can be added to an existing context encoder while leaving the downstream policy learner unchanged, so the approach is a drop-in modification of the task-inference module.
  • Out-of-distribution returns improve on most benchmarks without sacrificing in-distribution performance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct test of the causal interpretation would compare the learned adjacency matrix to a known ground-truth graph in a synthetic MDP and then intervene on individual edges; the paper leaves such verification open.
  • If the learned graph is truly causal, perturbing one edge should change the task representation in a predictable way; this is a testable consequence that follows from the framework but is not stated in the paper.
  • The linear structural equation likely sets a ceiling on tasks with nonlinear mechanisms; extending the same causal-layer idea to a nonlinear structural causal model is the natural next step.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This paper proposes CausalCOMRL, a context-based offline meta-reinforcement learning method whose task encoder is a causal variational autoencoder. The encoder produces a latent task representation z from context trajectories through the linear structural equation z=(I−A^T)^−1ϵ, where A is a DAG-constrained adjacency matrix intended to encode causal relations among task components, subject to a task-information constraint. The representation is additionally trained with an InfoNCE-style mutual information loss and metric-based contrastive losses, and the frozen encoder is used with SAC to train a meta-policy. Experiments on four MuJoCo OMRL benchmarks compare the method against Offline-PEARL, FOCAL, and CORRO, and include t-SNE visualizations and ablations. The paper claims that the causal encoder improves out-of-distribution generalization on most benchmarks.

Significance. The contribution would be significant if the causal layer demonstrably recovered task-component causal structure and if that structure drove the out-of-distribution gains: it would be the first causal representation method for context-based OMRL and would offer a principled way to remove spurious correlations. The paper also presents a clean modular pipeline (encoder pre-training, frozen encoder, SAC meta-training) and compares against three reasonable baselines with five seeds. However, the empirical evidence for the central causal claim is weak: the learned graph is never validated, the key ablation is inconclusive, and two of the auxiliary losses are not implemented as described by the equations. As it stands, the manuscript does not establish that causality, rather than the additional regularization and contrastive machinery, is responsible for the reported gains.

major comments (4)
  1. [§4.2, Eqs. (5), (12), (13)] The claim that the encoder recovers causal relationships among task components is not supported. Under the linear-Gaussian model z=(I−A^T)^−1ϵ with ϵ∼N(0,I), the marginal distribution of z is Gaussian with covariance (I−A^T)^−1(I−A)^−1, and infinitely many DAGs give the same covariance; the reconstruction objective (11) with constraints (12)–(13) does not identify A. The paper does not compare the learned A to a ground-truth graph, does not perform intervention or counterfactual tests, and does not include a randomized-A control. Moreover, the coordinates of z are unlabeled dimensions of a 16-dimensional bottleneck (Table B.4), so even a valid DAG over z would not establish causal relations among the named task components (goal, state, action, reward). This gap is load-bearing because the OOD-robustness argument rests on the recovered causal structure.
  2. [§5.4, Table 2] The ablation does not demonstrate that the causal encoder improves over the plain encoder. In Hopper-Rand-Params with the combined loss the causal encoder is numerically worse (305.8±13.4 vs 310.7±15.3), and in Walker-Rand-Params without the combined loss it is worse (416.0±14.7 vs 421.6±30.1). All reported differences are within one standard deviation, and no significance tests are given. The 'Causal Encoder' also adds parameters and the DAG/task-information regularizers relative to 'Encoder', so the comparison conflates causal structure with extra model capacity and regularization. The central claim that causal representations drive OOD gains is therefore not established by the experiments.
  3. [§4.3.1, Eqs. (17)–(18), Algorithm 1] The InfoNCE bound in Eq. (17) is derived for negative samples τ* drawn from tasks M* different from M. In the implementation, Eq. (18) and Algorithm 1 (lines 5–8) define τ* by adding Gaussian noise to the same task context τ, so the negative samples come from the same task, not from other tasks. Consequently, the bound I(z;M)≥I_NCE does not apply to the loss actually minimized, and the claim that Linfo optimizes a lower bound on task mutual information is unjustified. The authors should either implement true cross-task negatives or present an alternative justification for the same-task corrupted negatives.
  4. [§4.3.2, Eq. (20)] Equation (20) uses max_{i≠k}||zi−zk||² in the denominator of L_hardest. For hard negative mining, the hardest negative for an anchor is the closest negative, which requires a minimum over negative samples (or a min over pairs), not a maximum over all pairs. Minimizing 1/max distance as written would increase the distance to the farthest pair, which is the opposite of pulling closest negatives apart. If the intended loss uses a minimum, the equation must be corrected; otherwise the contrastive component does not implement the described hard-sample mining.
minor comments (6)
  1. [§5.1] There is a duplicated article in 'we adopt SAC [34] to train a a single-task policy', and §5 uses 'presentative' instead of 'representative'.
  2. [Page 1] The header line 'Preprint submitted to Nuclear Physics B' is a template artifact from another journal and should be removed for an ML submission.
  3. [§5.3, Figure 4] The t-SNE visualizations are purely qualitative; the paper should report a quantitative separation metric (e.g., silhouette score or nearest-neighbor accuracy) to support the claim that causal embeddings are more distinct.
  4. [§4.2, Eq. (8)] The functions λ1 and λ2 in the prior pθ(z|u) are left unspecified; the paper should state their parameterization, since the prior is part of the ELBO in Eq. (11).
  5. [Eq. (3) and Algorithm 1] The objective J(πθ) in Eq. (3) uses πθ, while the policy network is denoted πψ in Algorithm 1 and elsewhere; the notation should be unified.
  6. [References] Reference [59] is a linear non-Gaussian causal discovery method; since Eq. (5) assumes Gaussian noise, citing it as the basis for the causal model is misleading and should be replaced or qualified.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the causal encoder is learned from data, and the reported policy returns are genuine downstream evaluations rather than re-statements of training targets.

full rationale

The derivation chain is not circular. The adjacency matrix A in Eq. (5) is learned from offline context data through the causal-VAE ELBO (Eq. 11) with the DAG constraint (Eq. 13) and the task-information constraint (Eq. 12); it is not constructed from the test-task returns or from the reported performance metric. After encoder training, z is held fixed while SAC optimizes the policy (Algorithm 1, part B), and Algorithm 2 evaluates on separately sampled OOD tasks. The reported returns are therefore genuine predictions, not re-statements of training targets or fitted values. The InfoNCE bound (Statement 1, Eq. 17) is an externally grounded result, and the replacement of f(tau,z) by cosine similarity is an explicitly stated approximation rather than a hidden reuse of the claimed outcome. The ablations in Table 2 compare encoder variants with and without the causal layer and combined losses, which is a standard decomposition, not a circular argument. The paper's main weakness is evidential: the learned A is never checked against a ground-truth graph or interventions, and under the linear-Gaussian model it is not identifiable, so the label 'causal' and the attribution of OOD gains to recovered causal structure are not established. That is a validity and identifiability concern, not a circularity, and it does not raise the circularity score.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The central claim rests on standard VAE and InfoNCE machinery plus a set of hyperparameters that are tuned on the benchmarks. No new physical or mathematical entity is introduced. The main gap is that the causal semantics of the learned adjacency matrix A are assumed rather than demonstrated.

free parameters (7)
  • ELBO loss weight alpha = 0.3
    Selected from range [1e-1, 1] as optimal in Appendix B.
  • Task-information constraint weight beta = 1
    Selected from range [1e-1, 3] in Appendix B.
  • Triplet loss weight delta = 2
    Selected from range [1e-1, 3] in Appendix B.
  • Hard-sample mining weight kappa = 2
    Selected from range [0, 3] in Appendix B.
  • InfoNCE loss weight nu = 1
    Selected from range [1e-1, 3] in Appendix B.
  • DAG constraint constant c = 4
    Fixed in Table B.4; no sensitivity analysis provided.
  • Latent dimension n = 16
    Fixed in Table B.4; central to the causal layer size.
assumptions (5)
  • domain assumption Posterior factorization q_phi(z,eps|tau,u)=q_phi(z|eps)q_phi(eps|tau,u) holds.
    Section 4.2 Eq. (6). Assumes eps captures all information from tau and u needed to generate z.
  • domain assumption The linear SCM z=(I-A^T)^{-1}eps with a global adjacency matrix A is an adequate model of causal structure among task components.
    Section 4.2 Eq. (5). No identifiability argument is given, and no per-task graph structure is learned.
  • domain assumption The InfoNCE bound is valid and cosine similarity is a sufficient approximation of the density ratio f(tau,z)=q_phi(z|tau)/q_phi(z).
    Section 4.3.1 and Appendix A.1. The practical loss uses cosine similarity, which is a heuristic substitute for the theoretical quantity.
  • standard math The DAG constraint tr((I+c/n A*A)^n)-n=0 is differentiable and sufficient to enforce acyclicity of A.
    Section 4.2 Eq. (13), adopted from DAG-GNN literature.
  • ad hoc to paper The task-information constraint L_u = E||u - sigma(A^T u)||^2 <= k1 makes different tasks correspond to different causal graphs.
    Section 4.2 Eq. (12). The paper introduces this regularizer without a derivation showing it enforces per-task graph separation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CausalCOMRL: Context-Based Offline Meta-Reinforcement Learning with Causal Representation." pith.science (2026). https://pith.science/paper/2GTSE74V

@misc{pith2026250200983,
  author       = {Pith},
  title        = {Pith review of: CausalCOMRL: Context-Based Offline Meta-Reinforcement Learning with Causal Representation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2GTSE74V}},
  note         = {Machine review of arXiv:2502.00983}
}
read the original abstract

Context-based offline meta-reinforcement learning (OMRL) methods have achieved appealing success by leveraging pre-collected offline datasets to develop task representations that guide policy learning. However, current context-based OMRL methods often introduce spurious correlations, where task components are incorrectly correlated due to confounders. These correlations can degrade policy performance when the confounders in the test task differ from those in the training task. To address this problem, we propose CausalCOMRL, a context-based OMRL method that integrates causal representation learning. This approach uncovers causal relationships among the task components and incorporates the causal relationships into task representations, enhancing the generalizability of RL agents. We further improve the distinction of task representations from different tasks by using mutual information optimization and contrastive learning. Utilizing these causal task representations, we employ SAC to optimize policies on meta-RL benchmarks. Experimental results show that CausalCOMRL achieves better performance than other methods on most benchmarks.

Figures

Figures reproduced from arXiv: 2502.00983 by the authors.

Figure 1
Figure 1. Causal graph and encoder performance comparison. (a) Causal Graph Example. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Framework of CausalCOMRL: (a) Causal task encoder training module. (b) [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Average test returns of CausalCOMRL against representative context-based [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: The t-SNE visualization of the task representation space in Half-Cheetah-Vel. [PITH_FULL_IMAGE:figures/full_fig_p017_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. LLaPipe: LLM-Guided Reinforcement Learning for Automated Data Preparation Pipeline Construction

    cs.DB 2025-07 conditional novelty 5.0 of 10

    A framework that uses LLM suggestions, retrieved past experience, and adaptive triggering to speed up RL search for data preprocessing pipelines, showing improved accuracy on 18 tabular datasets.

  2. SoftPipe: A Soft-Guided Reinforcement Learning Framework for Automated Data Preparation

    cs.DB 2025-07 reject novelty 5.0 of 10

    SoftPipe replaces hard constraints in data-preparation search with a tuned softmax policy over LLM, ranker, and Q-value signals, reporting the best average accuracy among 11 methods on 18 tabular datasets.

Reference graph

Works this paper leans on

64 extracted references · 55 canonical work pages · cited by 2 Pith papers

  1. [1]

    Reinforcement learning in robotics: A survey

    Jens Kober, J Andrew Bagnell, and Jan Peters. Reinforcement learning in robotics: A survey. The International Journal of Robotics Research , 32(11):1238–1274, 2013

    2013

  2. [2]

    Sim-to-real transfer of robotic control with dynamics randomization

    Xue Bin Peng, Marcin Andrychowicz, Wojciech Zaremba, and Pieter Abbeel. Sim-to-real transfer of robotic control with dynamics randomization. In IEEE international conference on robotics and automation , pages 3803–3810, 2018

    work page 2018

  3. [3]

    Modular deep reinforcement learning from reward and punishment for robot navigation

    Jiexin Wang, Stefan Elfwing, and Eiji Uchibe. Modular deep reinforcement learning from reward and punishment for robot navigation. Neural Networks, 135:115–126, 2021

    work page 2021

  4. [4]

    Human-level control through deep reinforcement learning

    Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Ve- ness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidje- land, Georg Ostrovski, et al. Human-level control through deep reinforcement learning. Nature, 518(7540):529–533, 2015

    work page 2015

  5. [5]

    A general reinforcement learning algorithm that masters chess, shogi, and go through self-play

    David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, et al. A general reinforcement learning algorithm that masters chess, shogi, and go through self-play. Science, 362(6419):1140–1144, 2018

    2018

  6. [6]

    Improved robustness of reinforcement learning policies upon conversion to spiking neuronal network platforms applied to atari breakout game

    Devdhar Patel, Hananel Hazan, Daniel J Saunders, Hava T Siegelmann, and Robert Kozma. Improved robustness of reinforcement learning policies upon conversion to spiking neuronal network platforms applied to atari breakout game. Neural Networks, 120:108–115, 2019

    work page 2019

  7. [7]

    Domain randomization for transferring deep neural networks from simulation to the real world

    Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojciech Zaremba, and Pieter Abbeel. Domain randomization for transferring deep neural networks from simulation to the real world. In International Conference on Intelligent Robots and Systems , pages 23–30, 2017

    work page 2017

  8. [8]

    Brain-inspired meta-reinforcement learning cognitive control in conflictual in- hibition decision-making task for artificial agents

    Federica Robertazzi, Matteo Vissani, Guido Schillaci, and Egidio Falotico. Brain-inspired meta-reinforcement learning cognitive control in conflictual in- hibition decision-making task for artificial agents. Neural Networks, 154:283– 302, 2022

    work page 2022

Show all 64 references
  1. [9]

    Explaining aha! moments in artificial agents through ike-xai: Implicit knowledge extraction for explainable ai

    Ikram Chraibi Kaadoud, Adrien Bennetot, Barbara Mawhin, Vicky Charisi, and Natalia D ´ ıaz-Rodr ´ ıguez. Explaining aha! moments in artificial agents through ike-xai: Implicit knowledge extraction for explainable ai. Neural Networks, 155:95–118, 2022. 22

    2022

  2. [10]

    Deep reinforcement learning in a handful of trials using probabilistic dynamics models

    Kurtland Chua, Roberto Calandra, Rowan McAllister, and Sergey Levine. Deep reinforcement learning in a handful of trials using probabilistic dynamics models. In Advances in Neural Information Processing Systems , pages 4759– 4770, 2018

    2018

  3. [11]

    Offline meta-reinforcement learning for industrial insertion

    Tony Z Zhao, Jianlan Luo, Oleg Sushkov, Rugile Pevceviciute, Nicolas Heess, Jon Scholz, Stefan Schaal, and Sergey Levine. Offline meta-reinforcement learning for industrial insertion. In International Conference on Robotics and Automation, pages 6386–6393, 2022

    2022

  4. [12]

    Offline reinforce- ment learning: Tutorial, review, and perspectives on open problems

    Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu. Offline reinforce- ment learning: Tutorial, review, and perspectives on open problems. arXiv preprint arXiv:2005.01643, 2020

    2005 arXiv

  5. [13]

    Meta-learning in reinforcement learn- ing

    Nicolas Schweighofer and Kenji Doya. Meta-learning in reinforcement learn- ing. Neural Networks, 16(1):5–9, 2003

    2003

  6. [14]

    Model-agnostic meta- learning for fast adaptation of deep networks

    Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta- learning for fast adaptation of deep networks. In International Conference on Machine Learning , pages 1126–1135, 2017

    2017

  7. [15]

    Meta-reinforcement learning of structured exploration strategies

    Abhishek Gupta, Russell Mendonca, YuXuan Liu, Pieter Abbeel, and Sergey Levine. Meta-reinforcement learning of structured exploration strategies. In Advances in Neural Information Processing Systems , pages 5307–5316, 2018

    2018

  8. [16]

    Enhanced meta reinforcement learning via demonstra- tions in sparse reward environments

    Desik Rengarajan, Sapana Chaudhary, Jaewon Kim, Dileep Kalathil, and Srinivas Shakkottai. Enhanced meta reinforcement learning via demonstra- tions in sparse reward environments. In Advances in Neural Information Processing Systems, pages 2737–2749, 2022

    2022

  9. [17]

    Efficient off-policy meta-reinforcement learning via probabilistic context vari- ables

    Kate Rakelly, Aurick Zhou, Chelsea Finn, Sergey Levine, and Deirdre Quillen. Efficient off-policy meta-reinforcement learning via probabilistic context vari- ables. In International Conference on Machine Learning , pages 5331–5340, 2019

    2019

  10. [18]

    Focal: Efficient fully-offline meta- reinforcement learning via distance metric learning and behavior regulariza- tion

    Lanqing Li, Rui Yang, and Dijun Luo. Focal: Efficient fully-offline meta- reinforcement learning via distance metric learning and behavior regulariza- tion. In International Conference on Learning Representations , 2020

    2020

  11. [19]

    Context meta-reinforcement learning via neuromodulation

    Eseoghene Ben-Iwhiwhu, Jeffery Dick, Nicholas A Ketz, Praveen K Pilly, and Andrea Soltoggio. Context meta-reinforcement learning via neuromodulation. Neural Networks, 152:70–79, 2022. 23

    2022

  12. [20]

    Representation learning for continuous action spaces is beneficial for efficient policy learning

    Tingting Zhao, Ying Wang, Wei Sun, Yarui Chen, Gang Niu, and Masashi Sugiyama. Representation learning for continuous action spaces is beneficial for efficient policy learning. Neural Networks, 159:137–152, 2023

    2023

  13. [21]

    A survey on negative transfer

    Wen Zhang, Lingfei Deng, Lei Zhang, and Dongrui Wu. A survey on negative transfer. IEEE/CAA Journal of Automatica Sinica , 10(2):305–329, 2022

    2022

  14. [22]

    A survey of meta-reinforcement learn- ing

    Jacob Beck, Risto Vuorio, Evan Zheran Liu, Zheng Xiong, Luisa Zintgraf, Chelsea Finn, and Shimon Whiteson. A survey of meta-reinforcement learn- ing. arXiv preprint arXiv:2301.08028 , 2023

    2023 arXiv

  15. [23]

    Episodic task agnostic contrastive training for multi-task learning

    Fan Zhou, Yuyi Chen, Jun Wen, Qiuhao Zeng, Changjian Shui, Charles X Ling, Shichun Yang, and Boyu Wang. Episodic task agnostic contrastive training for multi-task learning. Neural Networks, 162:34–45, 2023

    2023

  16. [24]

    Provably improved context-based offline meta-rl with attention and contrastive learning

    Lanqing Li, Yuanhao Huang, Mingzhe Chen, Siteng Luo, Dijun Luo, and Jun- zhou Huang. Provably improved context-based offline meta-rl with attention and contrastive learning. arXiv preprint arXiv:2102.10774 , 2021

    2021 arXiv

  17. [25]

    Robust task representations for offline meta- reinforcement learning via contrastive learning

    Haoqi Yuan and Zongqing Lu. Robust task representations for offline meta- reinforcement learning via contrastive learning. In International Conference on Machine Learning , pages 25747–25759, 2022

    2022

  18. [26]

    Meta-reinforcement learning based on self-supervised task representation learning

    Mingyang Wang, Zhenshan Bing, Xiangtong Yao, Shuai Wang, Huang Kai, Hang Su, Chenguang Yang, and Alois Knoll. Meta-reinforcement learning based on self-supervised task representation learning. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 37, pages ...

    2023

  19. [27]

    Context shift reduction for offline meta-reinforcement learning

    Yunkai Gao, Rui Zhang, Jiaming Guo, Fan Wu, Qi Yi, Shaohui Peng, Sim- ing Lan, Ruizhi Chen, Zidong Du, Xing Hu, et al. Context shift reduction for offline meta-reinforcement learning. In Advances in Neural Information Processing Systems, volume 36, 2024

    2024

  20. [28]

    Towards an information theoretic framework of context- based offline meta-reinforcement learning

    Lanqing Li, Hai Zhang, Xinyu Zhang, Shatong Zhu, Junqiao Zhao, and Pheng-Ann Heng. Towards an information theoretic framework of context- based offline meta-reinforcement learning. arXiv preprint arXiv:2402.02429 , 2024

    2024 arXiv

  21. [29]

    Seeing is not believing: Robust reinforcement learning against spurious correlation

    Wenhao Ding, Laixi Shi, Yuejie Chi, and Ding Zhao. Seeing is not believing: Robust reinforcement learning against spurious correlation. In Advances in Neural Information Processing Systems , volume 36, 2024. 24

    2024

  22. [30]

    Improving multi-task generalization via regularizing spurious correlation

    Ziniu Hu, Zhe Zhao, Xinyang Yi, Tiansheng Yao, Lichan Hong, Yizhou Sun, and Ed Chi. Improving multi-task generalization via regularizing spurious correlation. In Advances in Neural Information Processing Systems , pages 11450–11466, 2022

    2022

  23. [31]

    Mujoco: A physics engine for model-based control

    Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In IEEE/RSJ international conference on intelligent robots and systems , pages 5026–5033, 2012

    2012

  24. [32]

    Toward causal repre- sentation learning

    Bernhard Sch¨ olkopf, Francesco Locatello, Stefan Bauer, Nan Rosemary Ke, Nal Kalchbrenner, Anirudh Goyal, and Yoshua Bengio. Toward causal repre- sentation learning. Proceedings of the IEEE, 109(5):612–634, 2021

    2021

  25. [33]

    Provably efficient causal reinforcement learning with confounded observational data

    Lingxiao Wang, Zhuoran Yang, and Zhaoran Wang. Provably efficient causal reinforcement learning with confounded observational data. In Advances in Neural Information Processing Systems , pages 21164–21175, 2021

    2021

  26. [34]

    Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor

    Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International Conference on Machine Learning , pages 1861–1870. PMLR, 2018

    2018

  27. [35]

    Awac: Ac- celerating online reinforcement learning with offline datasets

    Ashvin Nair, Abhishek Gupta, Murtaza Dalal, and Sergey Levine. Awac: Ac- celerating online reinforcement learning with offline datasets. arXiv preprint arXiv:2006.09359, 2020

    2006 arXiv

  28. [36]

    A perspective of q-value estimation on offline-to-online reinforcement learning

    Yinmin Zhang, Jie Liu, Chuming Li, Yazhe Niu, Yaodong Yang, Yu Liu, and Wanli Ouyang. A perspective of q-value estimation on offline-to-online reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 16908–16916, 2024

    2024

  29. [37]

    Offline meta reinforcement learning with in-distribution online adaptation

    Jianhao Wang, Jin Zhang, Haozhe Jiang, Junyu Zhang, Liwei Wang, and Chongjie Zhang. Offline meta reinforcement learning with in-distribution online adaptation. In International Conference on Machine Learning , pages 36626–36669, 2023

    2023

  30. [38]

    Weakly supervised representation learning with sparse perturbations

    Kartik Ahuja, Jason S Hartford, and Yoshua Bengio. Weakly supervised representation learning with sparse perturbations. In Advances in Neural Information Processing Systems, pages 15516–15528, 2022

    2022

  31. [39]

    Weakly supervised causal representation learning

    Johann Brehmer, Pim De Haan, Phillip Lippe, and Taco S Cohen. Weakly supervised causal representation learning. In Advances in Neural Information Processing Systems, pages 38319–38331, 2022. 25

    2022

  32. [40]

    Independent mechanism analysis, a new concept? In Advances in Neural Information Processing Systems, volume 34, pages 28233– 28248, 2021

    Luigi Gresele, Julius Von K¨ ugelgen, Vincent Stimper, Bernhard Sch¨ olkopf, and Michel Besserve. Independent mechanism analysis, a new concept? In Advances in Neural Information Processing Systems, volume 34, pages 28233– 28248, 2021

    2021

  33. [41]

    Disentanglement via mechanism sparsity regularization: A new principle for nonlinear ica

    S´ ebastien Lachapelle, Pau Rodriguez, Yash Sharma, Katie E Everett, R´ emi Le Priol, Alexandre Lacoste, and Simon Lacoste-Julien. Disentanglement via mechanism sparsity regularization: A new principle for nonlinear ica. In Conference on Causal Learning and Reasoning , pages 4...

    2022

  34. [42]

    Causalgan: Learning causal implicit generative models with ad- versarial training

    Murat Kocaoglu, Christopher Snyder, Alexandros G Dimakis, and Sriram Vishwanath. Causalgan: Learning causal implicit generative models with ad- versarial training. In International Conference on Learning Representations , 2018

    2018

  35. [43]

    Causalvae: Disentangled representation learning via neural structural causal models

    Mengyue Yang, Furui Liu, Zhitang Chen, Xinwei Shen, Jianye Hao, and Jun Wang. Causalvae: Disentangled representation learning via neural structural causal models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 9593–9602, 2021

    2021

  36. [44]

    Weakly supervised disentangled generative causal representation learning

    Xinwei Shen, Furui Liu, Hanze Dong, Qing Lian, Zhitang Chen, and Tong Zhang. Weakly supervised disentangled generative causal representation learning. Journal of Machine Learning Research , 23(241):1–55, 2022

    2022

  37. [45]

    On causally disentangled representations

    Abbavaram Gowtham Reddy, Vineeth N Balasubramanian, et al. On causally disentangled representations. In Proceedings of the AAAI Conference on Ar- tificial Intelligence , pages 8089–8097, 2022

    2022

  38. [46]

    Causal influence detection for improving efficiency in reinforcement learning

    Maximilian Seitzer, Bernhard Sch¨ olkopf, and Georg Martius. Causal influence detection for improving efficiency in reinforcement learning. In Advances in Neural Information Processing Systems , pages 22905–22918, 2021

    2021

  39. [47]

    Generalizing goal- conditioned reinforcement learning with variational causal reasoning

    Wenhao Ding, Haohong Lin, Bo Li, and Ding Zhao. Generalizing goal- conditioned reinforcement learning with variational causal reasoning. In Ad- vances in Neural Information Processing Systems , pages 26532–26548, 2022

    2022

  40. [48]

    Passive learning of active causal strategies in agents and language models

    Andrew Lampinen, Stephanie Chan, Ishita Dasgupta, Andrew Nam, and Jane Wang. Passive learning of active causal strategies in agents and language models. In Advances in Neural Information Processing Systems , volume 36, 2024

    2024

  41. [49]

    Representation learning with contrastive predictive coding

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 , 2018. 26

    2018 arXiv

  42. [50]

    Domino: Decomposed mutual information optimization for generalized context in meta-reinforcement learning

    Yao Mu, Yuzheng Zhuang, Fei Ni, Bin Wang, Jianyu Chen, Jianye Hao, and Ping Luo. Domino: Decomposed mutual information optimization for generalized context in meta-reinforcement learning. In Advances in Neural Information Processing Systems, pages 27563–27575, 2022

    2022

  43. [51]

    Contrabar: Contrastive bayes-adaptive deep rl

    Era Choshen and Aviv Tamar. Contrabar: Contrastive bayes-adaptive deep rl. In International Conference on Machine Learning, pages 6005–6027, 2023

    2023

  44. [52]

    Improved baselines with momentum contrastive learning

    Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003.04297, 2020

    2003 arXiv

  45. [53]

    Emerging properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv´ e J´ egou, Julien Mairal, Pi- otr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF International Confer- ence on Computer Vision , pages 9650–9660, 2021

    2021

  46. [54]

    Consert: A contrastive framework for self-supervised sentence represen- tation transfer

    Yuanmeng Yan, Rumei Li, Sirui Wang, Fuzheng Zhang, Wei Wu, and Weiran Xu. Consert: A contrastive framework for self-supervised sentence represen- tation transfer. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th Internatio...

    2021

  47. [55]

    Simcse: Simple contrastive learning of sentence embeddings

    Tianyu Gao, Xingcheng Yao, and Danqi Chen. Simcse: Simple contrastive learning of sentence embeddings. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , pages 6894–6910, 2021

    2021

  48. [56]

    Contrastive identity-aware learning for multi-agent value decomposition

    Shunyu Liu, Yihe Zhou, Jie Song, Tongya Zheng, Kaixuan Chen, Tongtian Zhu, Zunlei Feng, and Mingli Song. Contrastive identity-aware learning for multi-agent value decomposition. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 11595–11603, 2023

    2023

  49. [57]

    Reinforcement learning: An intro- duction

    Richard S Sutton and Andrew G Barto. Reinforcement learning: An intro- duction. MIT press, 2018

    2018

  50. [58]

    Auto-encoding variational bayes

    Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013

    2013 arXiv

  51. [59]

    A linear non-gaussian acyclic model for causal discovery

    Shohei Shimizu, Patrik O Hoyer, Aapo Hyv¨ arinen, Antti Kerminen, and Michael Jordan. A linear non-gaussian acyclic model for causal discovery. Journal of Machine Learning Research , 7(10), 2006

    2006

  52. [60]

    D’ya like dags? a survey on structure learning and causal discovery

    Matthew J Vowels, Necati Cihan Camgoz, and Richard Bowden. D’ya like dags? a survey on structure learning and causal discovery. ACM Computing Surveys, 55(4):1–36, 2022. 27

    2022

  53. [61]

    Dag-gnn: Dag structure learning with graph neural networks

    Yue Yu, Jie Chen, Tian Gao, and Mo Yu. Dag-gnn: Dag structure learning with graph neural networks. In International conference on machine learning, pages 7154–7163, 2019

    2019

  54. [62]

    Multi-task batch reinforcement learning with metric learning

    Jiachen Li, Quan Vuong, Shuang Liu, Minghua Liu, Kamil Ciosek, Henrik Christensen, and Hao Su. Multi-task batch reinforcement learning with metric learning. In Advances in Neural Information Processing Systems , volume 33, pages 6197–6210, 2020

    2020

  55. [63]

    Training region- based object detectors with online hard example mining

    Abhinav Shrivastava, Abhinav Gupta, and Ross Girshick. Training region- based object detectors with online hard example mining. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 761–769, 2016

    2016

  56. [64]

    Visualizing data using t-sne

    Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of Machine Learning Research , 9(11), 2008. 28

    2008

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.