Pith. sign in

REVIEW 3 major objections 6 minor 59 references

Consistent Zero-Shot Imitation with Contrastive Goal Inference

T0 review · 3 major / 6 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Zero-shot imitation reduces to goal inference: a self-supervised agent infers the demonstrator's goal from one trajectory and commands a contrastive goal-conditioned policy, and this procedure is proven consistent.

desk verdict CIRL's consistency theorem doesn't apply to the GoalKDE-based algorithm that was actually run, though the FB counterexample is a genuine contribution. read the letter →

arxiv 2510.17059 v2 pith:364SNZSI submitted 2025-10-20 cs.LG

Kathryn Wantlin , Chongyi Zheng , Benjamin Eysenbach This is my paper · ORCID
classification cs.LG
keywords zero-shotimitationgoal-conditionedreinforcementlearninginversecontrastiveRLmaximumentropyvariationalinferencegoalself-supervisedpretraining
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces CIRL, a self-supervised pretraining method for zero-shot imitation. The central claim is that when tasks can be summarized by a goal state, the multi-task inverse reinforcement learning problem reduces to goal inference: instead of recovering a reward function from a demonstration, the agent infers which goal the demonstrator was pursuing, then commands a goal-conditioned policy trained entirely through autonomous exploration. The paper proves that this reduction is consistent (Lemma 1) — that is, it recovers the correct distribution over expert goals by accounting for the relative difficulty of reaching different states, something occupancy-based methods like Forward-Backward fail to do (Lemma 2). The practical payoff is that an agent can be pretrained with no rewards, no demonstrations, and no preferences, and then imitate a single expert trajectory at test time. Experiments on goal-conditioned and reward-maximizing benchmarks show CIRL outperforms prior zero-shot imitation baselines.

What carries the argument

The key machinery is Lemma 1, an identity that converts MaxEnt IRL into a variational goal-inference objective. The forward KL between expert and model trajectory distributions decomposes via the KL chain rule; because the same goal prior appears in both joints, the objective becomes a variational posterior-matching problem. The crucial trick is that trajectories are sampled from the MaxEnt policy (g∼p(g), τ∼p*(τ|g)), so the intractable partition function Z_g is accounted for implicitly in the sample distribution. This is coupled with two supporting components: a mean-field variational family (Corollary 1) that contains the true posterior for finite trajectories, and GoalKDE, an automatic go

What would settle it

On a locomotion task, train CIRL with GoalKDE exploration but evaluate on expert trajectories whose true goals lie in a region GoalKDE rarely visited during training (e.g., a far corner of the arena). Plot the MAP-inferred goal against the true goal across many demonstrations: if the consistency claim holds for the algorithm as run, the inferred goals should be unbiased; any systematic bias toward the visited low-density region — and a corresponding drop in imitation score relative to oracle goal sampling — would refute the claim as applied.

Watch

Extended reading notes

Core claim

The paper's central claim is an equivalence (Lemma 1): under MaxEnt IRL with goal-conditioned rewards, minimizing the forward KL between expert and model trajectory distributions (with a shared goal prior) is equivalent to maximizing a variational goal-inference objective — min_θ E_{p(g)} D_KL(p_E(τ|g)∥p*(τ|g)) ⟹ max_ξ E_{g∼p(g),τ∼p*(τ|g)} log q_ξ(g|τ). The partition function encoding the relative difficulty of reaching goals is never computed; it is implicitly captured by sampling trajectories from the MaxEnt policy. The paper further proves (Corollary 1) that a mean-field variational posterior is without loss of generality for finite trajectories, and gives a counterexample (Lemma 2) showi

Load-bearing premise

The consistency proof assumes the goal distribution used during pretraining is the same as the expert's goal prior and that the learned MaxEnt policy is optimal; in practice GoalKDE explores low-density states rather than sampling from the test-time goal prior, so the theorem does not apply to the algorithm as run — a gap the authors attribute for most of the performance loss.

Editorial extensions

If this is right

  • Agents can be pretrained for imitation entirely through self-supervised interaction — no rewards, no demonstrations, no human preference data — and then imitate a single expert trajectory at test time.
  • Because goal inference accounts for the relative difficulty of reaching states, an infrequently visited but intended state is correctly identified as the goal, something occupancy-based IRL methods provably fail to do.
  • By enlarging the goal space (e.g., adding velocity to position), the same framework can imitate reward-maximizing policies that are not explicitly goal-reaching, as demonstrated on standard benchmark tasks.
  • Mean-field goal inference, which predicts the goal independently from each state-action pair, is both computationally cheaper and, when the true posterior is mean-field, exactly as expressive as full-trajectory encoders.
  • The proven failure of Forward-Backward representations (Lemma 2) is a caution: using occupancy measures directly as rewards is fundamentally inadequate for zero-shot imitation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The consistency theorem requires the goal prior used in pretraining to equal the expert's goal prior. The deployed algorithm uses GoalKDE, which samples low-density states rather than drawing from p(g); the authors acknowledge in Appendix D that most of the measured performance gap stems from this distribution shift. The theoretical guarantee therefore does not directly cover the algorithm as run.
  • If GoalKDE were replaced by a method that samples goals from the true test-time prior, the imitation score should approach the oracle baseline whenever the variational goal-inference model is well calibrated; the gap observed between GoalKDE and oracle sampling is thus a measurable violation of the theorem's assumptions.
  • The variational sampling trick in Lemma 1 may extend beyond goal-reaching rewards to any reward family parameterized by a low-dimensional latent variable, though the mean-field corollary would not survive without the goal structure.
  • Lemma 2's counterexample generalizes to a broader lesson for IRL: any method that treats state visitation frequency as a proxy for reward will misread policies that spend time in easy-to-reach rather than rewarding states.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes CIRL, a self-supervised pretraining method for zero-shot imitation. CIRL trains a goal-conditioned maximum-entropy policy with contrastive RL, proposes training goals via a KDE-based low-density sampler (GoalKDE), and learns an amortized variational posterior q_xi(g|tau) over goals from trajectories. At test time, given a single demonstration, it infers a goal and commands the learned goal-conditioned policy. The paper claims a consistency guarantee (Lemma 1) that MaxEnt IRL over goal-conditioned rewards is equivalent to maximizing L_Info = E_{g~p(g), tau~p^*(tau|g)} log q_xi(g|tau), and reports experiments on JaxGCRL and URLB showing improved imitation scores over FB and 1-NN baselines.

Significance. The conceptual goal—reducing multi-task IRL to goal inference while accounting for the partition function—is appealing, and the paper ships reproducible code and conducts informative ablations. However, the central theoretical contribution is not established as stated. The proof of Lemma 1 effectively assumes the learned policy trajectory distribution equals the expert's, and the implemented GoalKDE goal sampler violates the common-prior assumption of the theorem. The paper therefore does not currently deliver a consistency guarantee for the algorithm that is actually evaluated. The empirical results may still be of interest, but the main novelty is the claimed consistency, and that claim is unsupported.

major comments (3)
  1. [Section 4.3 / Lemma 1 / Appendix A.1] The proof never bridges the IRL objective and the L_Info objective. After writing min_theta E_p(g) D_KL(p_E(tau|g) || p^*(tau|g)), the proof switches to FA VI over p^*(tau,g) and shows that max_xi E_{p^*(g,tau)} log q_xi(g|tau) makes q_xi match p^*(g|tau). But matching the posterior of the learned policy is not the same as matching the expert posterior p_E(g|tau). The two coincide only if p^*(tau|g) = p_E(tau|g), which is assumed nowhere in the lemma and is not guaranteed by optimality of the learned policy unless the expert is itself the MaxEnt-optimal policy for the same entropy coefficient. Under that assumption the result is almost tautological: q is trained on samples from p^*(g,tau) and hence recovers p^*(g|tau) by definition. This does not constitute a consistency proof for IRL.
  2. [Algorithm 1 line 5 / Section 4.4 / Appendix D] The theorem requires goals to be drawn from p(g), the same prior used at test time ('When the same prior p(g) is used'). GoalKDE instead selects g = argmin_g KDE(D), a deterministic, policy-dependent low-density proposal mu(g). Consequently the variational objective trains q on p_mu(g|tau) proportional to mu(g) p^*(tau|g), not the expert posterior under p(g). Recomputing the KL decomposition with mu leaves an uncancelled E[log(p(g)/mu(g))] term, so maximizing L_Info is not equivalent to MaxEnt IRL under the test-time prior. The authors' own Appendix D statement—'most of the performance gap is likely due to distribution shift between goals explored by GoalKDE and those commanded at test time'—concedes this distribution shift. Thus Lemma 1 does not apply to CIRL as implemented.
  3. [Section 4.2 / Corollary 1] The claim that the mean-field family 'without loss of generality' contains the true posterior is asserted without proof. The true posterior p^*(g|tau) is proportional to p(g) exp(sum_t r_g(s_t,a_t) - log Z_g); the goal-dependent factors p(g) and Z_g are global and do not factor over time. No argument shows they can be absorbed into per-timestep Gaussian conditionals q_xi(g|s_t,a_t), and the Gaussian MLP class is not closed under arbitrary such reparameterizations. This is not merely technical, because the corollary is used to justify the goal-inference architecture. The empirical superiority of mean field over full-trajectory inference in Figure 5 can stand on its own, but the corollary should be proved with appropriate capacity assumptions or removed.
minor comments (6)
  1. [Equation (4)] The symbol '=⇒' is ambiguous. The paper should state whether Lemma 1 claims an equivalence, an implication, or a reduction, and in which direction. There is also a missing closing parenthesis in the KL expression.
  2. [Section 4.2] The sentence 'By additionally noting that the g we infer should have high mutual information with tau, we simplify...' is misleading. The simplification from D_KL(p^*(g|tau) || q_xi(g|tau)) to E log q_xi(g|tau) is just the forward KL objective with respect to xi; the mutual-information remark is unnecessary.
  3. [Appendix A.1] In the chain-rule step, the notation 'E_{p(g)}[D_KL(p_E(tau,g) || p^*(tau,g))]' is not correct; the joint KL is D_KL(p_E(tau,g) || p^*(tau,g)) after defining the joints with the same prior. Also, the roles of theta and g are not consistently distinguished: the IRL objective is written as min_theta, but the goal-conditioned family is indexed by g.
  4. [Section 5 / Appendix C] The expert is trained with CRL under oracle goal sampling, while CIRL is trained with GoalKDE. The paper should clarify whether the expert satisfies the MaxEnt optimality assumption of Lemma 1; the reported entropy coefficient alpha = 1e-5 is very small, so the expert is close to deterministic and may not be the soft-Boltzmann policy used in the theory.
  5. [Appendix D / Figure 8] The admission that 'most of the performance gap is likely due to distribution shift between goals explored by GoalKDE and those commanded at test time' directly conflicts with the consistency claim in the main text. This is currently relegated to the appendix; it should be discussed in the main text where Lemma 1 is presented.
  6. [Appendix A.2 / Lemma 2] The FB inconsistency counterexample assumes the backward representation can be the one-hot occupancy vector. For learned FB representations this equality is not guaranteed, so the lemma is illustrative rather than a rigorous refutation of FB as a family. It is not central to CIRL, but the paper should temper the wording.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; consistency theorem is a conditional reduction and the GoalKDE mismatch is a validity limitation, not a circular step.

full rationale

The central derivation (Appendix A.1, Lemma 1) is a mathematical reduction: under the stated assumptions (expert is a MaxEnt goal-optimizer and the same goal prior p(g) is used in data collection and modeling), the MaxEnt IRL objective equals a joint KL that decomposes into a trajectory term and a posterior term, and the posterior term is optimized by maximizing E_{g∼p(g),τ∼p^*(τ|g)} log q_ξ(g|τ). This is a standard consistency statement conditional on correct model specification, not a prediction equivalent to its input by construction. The paper does not fit q to expert goals and then rename the fit as a prediction; q is trained on trajectories from the learned policy, and the empirical evaluations include experts trained with CRL as well as PPO on URLB, providing external checkpoints. The main caveat is that the implemented GoalKDE sampler (Algorithm 1, line 5: g = argmin_g KDE(D)) does not sample from p(g), so the theorem's prior-cancellation condition is not met by the algorithm as run; the authors acknowledge this in Appendix D ('most of the performance gap is likely due to distribution shift between goals explored by GoalKDE and those commanded at test time'). This is a gap between theorem and implementation, not a circular reduction. Self-citations to Eysenbach et al. (2022) and Myers et al. (2024) supply the underlying contrastive RL machinery and are externally published, code-reproduced methods; they are not invoked as a uniqueness theorem or to forbid alternatives.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the goal-conditioned reward hypothesis, on the assumption that the expert and the learned model share the same MaxEnt goal-conditioned trajectory distribution, on the assumed equality of the pretraining and expert goal priors, and on the mean-field family containing the true posterior. The first and last are asserted rather than demonstrated; the second and third are explicitly violated or unverified in the experiments (GoalKDE vs oracle goals, suboptimal learned policies).

free parameters (2)
  • MaxEnt entropy regularization weight α = 1e-5
    Chosen in Appendix C; balances reward vs entropy in the soft Q / actor objective and in p*(τ|g); not fitted to data but a hand-set hyperparameter the consistency guarantee depends on.
  • GoalKDE kernel bandwidth = unspecified
    The KDE bandwidth controls which states are considered low-density and therefore selected as goals; not reported in the paper, likely defaults from implementation.
assumptions (5)
  • domain assumption Many tasks can be summarized by a single goal state; reward depends only on reaching that state
    Section 1/3.1. If false, goal inference cannot represent the demonstrator's intent; authors acknowledge extension to larger goal spaces for URLB.
  • domain assumption The expert follows the same MaxEnt goal-conditioned policy as the learned model, and the learned policy is trained to optimality so p*(τ|g)=p_E(τ|g)
    Section 4.2 "When our policy is trained to optimality..."; Appendix A.1. Consistency proof cancels the difference between expert and model trajectory distributions; this is assumed, not proven.
  • domain assumption The goal prior p(g) used during pretraining matches the expert's goal prior
    Appendix A.1: "When the same prior p(g) is used". CIRL's GoalKDE selects low-density states rather than sampling p(g), so the axiom is violated in the actual algorithm; Appendix D attributes performance gap to distribution shift.
  • ad hoc to paper The mean-field variational family contains the true posterior (Corollary 1)
    Section 4.2. The proof replaces log Z_g by a per-timestep term and ignores p(g); whether an arbitrary positive posterior factorizes as ∏ q_t(g|s_t,a_t) is not established.
  • standard math Forward amortized variational inference (Ambrogioni et al. 2019) is valid and its sampling distribution k(τ) can be chosen as p*(τ)
    Appendix A.1 relies on FA VI objective; the method is cited from prior work.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Consistent Zero-Shot Imitation with Contrastive Goal Inference." pith.science (2026). https://pith.science/paper/364SNZSI

@misc{pith2026251017059,
  author       = {Pith},
  title        = {Pith review of: Consistent Zero-Shot Imitation with Contrastive Goal Inference},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/364SNZSI}},
  note         = {Machine review of arXiv:2510.17059}
}
read the original abstract

Zero-shot imitation learning requires an agent to reproduce expert behavior from a single demonstration without additional environment interaction or gradient updates at test time. We introduce Contrastive Inverse Reinforcement Learning (CIRL), a self-supervised framework for pre-training zero-shot imitation agents. Our methods rests on a key observation that many useful tasks can be summarized by a single goal state. We can thus convert the multi-task inverse RL problem into a more tractable goal-inference problem, and utilize state-of-the-art goal-conditioned RL methods to recover a policy that reaches the goal. During pre-training, CIRL jointly employs three components to learn without any rewards or demonstrations: (1) a variant of contrastive RL designed to learn maximum-entropy goal-conditioned policies, (2) an automatic goal proposal mechanism (GoalKDE) that drives exploration, and (3) a mean-field variational model that performs amortized goal inference from trajectories. We prove that this procedure consistently recovers the demonstrator's intent by accounting for the relative difficulty of reaching different states and show how structurally similar prior work may otherwise fail to infer the correct reward. Experiments on goal-conditioned and standard reward-maximizing control tasks show that CIRL outperforms prior zero-shot imitation methods, supporting the expressiveness of goals as a compact summary of behavior.

Figures

Figures reproduced from arXiv: 2510.17059 by the authors.

Figure 1
Figure 1. Zero shot imitation learning. Assum￾ing access to a multi-task environment, our gen￾eralist agent must imagine and practice its own tasks to effectively imitate unknown task demon￾strations at test time. Today’s AI agents, whether in language or robotics, are trained primarily by mimicking human demonstrations. But, in the same way that children conduct a large degree of learning in an unsupervised (adult-free) fash… view at source ↗
Figure 2
Figure 2. Zero-shot imitation learning with CIRL via goal inference. CIRL combines goal-conditioned contrastive RL pre-training, automatic goal sampling for exploration, and a mean field goal inference model to imitate expert demonstrations. Here we see how an Ant’s imitation policy and posterior distribution over goal states evolve across timesteps toward a final maximum a posteriori (MAP) estimate. 5 EXPERIMENTS Our method … view at source ↗
Figure 3
Figure 3. Value of self-supervised RL pre￾training CIRL consistently outperforms the al￾ternative FB representation zero-shot imitation method as well as the naive 1-NN policy baseline. We first compare CIRL against several base￾lines for imitation learning, including those with and without access to expert data during training. For each environment, we compared the reward earned by an expert policy (CRL) and the imitation le… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Summarizing behavior via goals yields better imitation than reward-based ex￾planations. When using the last expert demon￾stration state as the goal, CIRL achieves high im￾itation scores on goal-conditioned environments while FB struggles to infer goal-conditioned re￾wa…
Figure 5
Figure 5. Figure 5: Mean field goal inference models out￾perform alternative full τ input models For all environments, the Mean Field model should con￾tain the true posterior and is computationally eas￾ier to train, making it the superior choice for CIRL. Our theory suggests that inferrin…
Figure 6
Figure 6. Figure 6: GoalKDE exploration vs. oracle goal sampling during CRL pre-training Holding the goal inference method constant (mean field in￾ference), we find that GoalKDE sampling can achieve a significant fraction of imitation score compared to the oracle baseline, and can even ou…
Figure 7
Figure 7. Figure 7: CIRL inferred goals efficiently sum￾marize complex rewards. CIRL achieves lower regret than FB when imitating URLB policies with non-goal-reaching rewards. Future work could extend our framework to explore richer goal representations, such as language or multi-modal sp…
Figure 8
Figure 8. Figure 8: Improving CIRL imitation scores For the Pusher environment, we can achieve perfect imitation by providing the true goal to the CRL + GoalKDE policy, but for the Ant environment, most of the performance gap is likely due to distri￾bution shift between goals explored by …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

59 extracted references · 2 canonical work pages

  1. [1]

    u c l\" u , Julia Berezutskaya, Eva van den Borne, Ya g mur G\

    Luca Ambrogioni, Umut G\" u c l\" u , Julia Berezutskaya, Eva van den Borne, Ya g mur G\" u c l\" u t\" u rk, Max Hinne, Eric Maris, and Marcel van Gerven. Forward amortized inference for likelihood-free variational marginalization. In Kamalika Chaudhuri and Masashi Sugiyama (eds.), Proceedings of the Twenty-Second International Conference on Artificial I...

    2019

  2. [2]

    Concrete problems in ai safety

    Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Francis Christiano, John Schulman, and Dandelion Man \'e . Concrete problems in ai safety. ArXiv, abs/1606.06565, 2016. URL https://api.semanticscholar.org/CorpusID:10242377

    arXiv 2016

  3. [3]

    Hindsight experience replay

    Marcin Andrychowicz, Filip Wolski, Alex Ray, Jonas Schneider, Rachel Fong, Peter Welinder, Bob McGrew, Josh Tobin, OpenAI Pieter Abbeel, and Wojciech Zaremba. Hindsight experience replay. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 30. Curr...

    2017

  4. [4]

    End to end learning for self-driving cars

    Mariusz Bojarski, Davide Del Testa, Daniel Dworakowski, Bernhard Firner, Beat Flepp, Prasoon Goyal, Lawrence D Jackel, Mathew Monfort, Urs Muller, Jiakai Zhang, Xin Zhang, Jake Zhao, and Karol Zieba. End to end learning for self-driving cars. arXiv [cs.CV], April 2016

    2016

  5. [5]

    Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S

    Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, S. Buch, Dallas Card, Rodrigo Castellon, Niladri S. Chatterji, Annie S. Chen, Kathleen A. Creel, Jared Davis, Dora Demszky, Chris Donahue, Moussa Doumbouya, Esin Durmus, Stefano...

    2021

  6. [6]

    Goodman, Elizabeth Spelke, and Laura Schulz

    Elizabeth Bonawitz, Patrick Shafto, Hyowon Gweon, Noah D. Goodman, Elizabeth Spelke, and Laura Schulz. The double-edged sword of pedagogy: Instruction limits spontaneous exploration and discovery. Cognition, 120 0 (3): 0 322--330, 2011. ISSN 0010-0277. doi:https://doi.org/10.1016/j.cognition.2010.10.001. URL https://www.sciencedirect.com/science/article/p...

    doi:10.1016/j.cognition.2010.10.001 2011

  7. [7]

    Accelerating Goal-Conditioned RL Algorithms and Research

    Micha Bortkiewicz, W adek Pa ucki, Vivek Myers, Tadeusz Dziarmaga, Tomasz Arczewski, ukasz Kuci\' n ski, and Benjamin Eysenbach. Accelerating Goal-Conditioned RL Algorithms and Research . In International Conference on Learning Representations , 2025. URL https://arxiv.org/pdf/2408.11052

    arXiv 2025

  8. [8]

    JAX : composable transformations of P ython+ N um P y programs, 2018

    James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake Vander P las, Skye Wanderman- M ilne, and Qiao Zhang. JAX : composable transformations of P ython+ N um P y programs, 2018. URL http://github.com/jax-ml/jax

    2018

Show all 59 references
  1. [9]

    Openai gym

    Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. Openai gym. ArXiv, abs/1606.01540, 2016. URL https://api.semanticscholar.org/CorpusID:16099293

    2016 arXiv

  2. [10]

    Explore, discover and learn: Unsupervised discovery of state-covering skills

    Victor Campos, Alexander Trott, Caiming Xiong, Richard Socher, Xavier Giro-I-Nieto, and Jordi Torres. Explore, discover and learn: Unsupervised discovery of state-covering skills. In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference on Mach...

    2020

  3. [11]

    Multi-task hierarchical adversarial inverse reinforcement learning

    Jiayu Chen, Dipesh Tamboli, Tian Lan, and Vaneet Aggarwal. Multi-task hierarchical adversarial inverse reinforcement learning. In Proceedings of the 40th International Conference on Machine Learning, ICML'23. JMLR.org, 2023

    2023

  4. [12]

    Diffusion policy: Visuomotor policy learning via action diffusion

    Cheng Chi, Siyuan Feng, Yilun Du, Zhenjia Xu, Eric Cousineau, Benjamin Burchfiel, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. In Proceedings of Robotics: Science and Systems (RSS), 2023

    2023

  5. [13]

    Diffusion policy: Visuomotor policy learning via action diffusion

    Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. The International Journal of Robotics Research, 2024

    2024

  6. [14]

    Dragan, Kenton C.T

    Anca D. Dragan, Kenton C.T. Lee, and Siddhartha S. Srinivasa. Legibility and predictability of robot motion. In Proceedings of the 8th ACM/IEEE International Conference on Human-Robot Interaction, HRI '13, pp.\ 301–308. IEEE Press, 2013. ISBN 9781467330558

    2013

  7. [15]

    Rewriting history with inverse rl: hindsight inference for policy improvement

    Benjamin Eysenbach, Xinyang Geng, Sergey Levine, and Ruslan Salakhutdinov. Rewriting history with inverse rl: hindsight inference for policy improvement. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS '20, Red Hook, NY, USA, ...

    2020

  8. [16]

    Contrastive learning as goal-conditioned reinforcement learning

    Benjamin Eysenbach, Tianjun Zhang, Sergey Levine, and Ruslan Salakhutdinov. Contrastive learning as goal-conditioned reinforcement learning. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS '22, Red Hook, NY, USA, 2022. Curran ...

    2022

  9. [17]

    Chelsea Finn, Sergey Levine, and P. Abbeel. Guided cost learning: Deep inverse optimal control via policy optimization. In International Conference on Machine Learning, 2016. URL https://api.semanticscholar.org/CorpusID:8121626

    2016

  10. [18]

    One-shot visual imitation learning via meta-learning

    Chelsea Finn, Tianhe Yu, Tianhao Zhang, Pieter Abbeel, and Sergey Levine. One-shot visual imitation learning via meta-learning. In Sergey Levine, Vincent Vanhoucke, and Ken Goldberg (eds.), Proceedings of the 1st Annual Conference on Robot Learning, volume 78 of Proceedings of...

    2017

  11. [19]

    Automatic goal generation for reinforcement learning agents

    Carlos Florensa, David Held, Xinyang Geng, and Pieter Abbeel. Automatic goal generation for reinforcement learning agents. In Jennifer Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learni...

    2018

  12. [20]

    Brax -- a differentiable physics engine for large scale rigid body simulation

    C Daniel Freeman, Erik Frey, Anton Raichuk, Sertan Girgin, Igor Mordatch, and Olivier Bachem. Brax -- a differentiable physics engine for large scale rigid body simulation. arXiv [cs.RO], June 2021

    2021

  13. [21]

    Learning robust rewards with adverserial inverse reinforcement learning

    Justin Fu, Katie Luo, and Sergey Levine. Learning robust rewards with adverserial inverse reinforcement learning. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings . OpenReview.ne...

    2018

  14. [22]

    Childhood as a solution to explore–exploit tensions

    Alison Gopnik. Childhood as a solution to explore–exploit tensions. Philosophical Transactions of the Royal Society B: Biological Sciences, 375 0 (1803): 0 20190502, 2020. doi:10.1098/rstb.2019.0502. URL https://royalsocietypublishing.org/doi/abs/10.1098/rstb.2019.0502

    2020

  15. [23]

    From exploration to instruction: Children learn from exploration and tailor their demonstrations to observers’ goals and competence

    Hyowon Gweon and Laura Schulz. From exploration to instruction: Children learn from exploration and tailor their demonstrations to observers’ goals and competence. Child Development, 90 0 (1): 0 e148--e164, 2019. doi:https://doi.org/10.1111/cdev.13059. URL https://srcd.onlinel...

    2019 doi

  16. [24]

    Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor

    Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International conference on machine learning, pp.\ 1861--1870. Pmlr, 2018

    2018

  17. [25]

    Generative adversarial imitation learning

    Jonathan Ho and Stefano Ermon. Generative adversarial imitation learning. In Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS'16, pp.\ 4572–4580, Red Hook, NY, USA, 2016. Curran Associates Inc. ISBN 9781510838819

    2016

  18. [26]

    BC -z: Zero-shot task generalization with robotic imitation learning

    Eric Jang, Alex Irpan, Mohi Khansari, Daniel Kappler, Frederik Ebert, Corey Lynch, Sergey Levine, and Chelsea Finn. BC -z: Zero-shot task generalization with robotic imitation learning. In 5th Annual Conference on Robot Learning, 2021. URL https://openreview.net/forum?id=8kbp23tSGYv

    2021

  19. [27]

    Learning to achieve goals

    Leslie Pack Kaelbling. Learning to achieve goals. In International Joint Conference on Artificial Intelligence, 1993. URL https://api.semanticscholar.org/CorpusID:5538688

    1993

  20. [28]

    Variational curriculum reinforcement learning for unsupervised discovery of skills

    Seongun Kim, Kyowoon Lee, and Jaesik Choi. Variational curriculum reinforcement learning for unsupervised discovery of skills. In Proceedings of the 40th International Conference on Machine Learning, ICML'23. JMLR.org, 2023

    2023

  21. [29]

    Urlb: Unsupervised reinforcement learning benchmark

    Misha Laskin, Denis Yarats, Hao Liu, Kimin Lee, Albert Zhan, Kevin Lu, Catherine Cang, Lerrel Pinto, and Pieter Abbeel. Urlb: Unsupervised reinforcement learning benchmark. In J. Vanschoren and S. Yeung (eds.), Proceedings of the Neural Information Processing Systems Track on ...

    2021

  22. [30]

    Learning Temporal Distances : Contrastive Successor Features Can Provide a Metric Structure for Decision-Making

    Vivek Myers, Chongyi Zheng, Anca Dragan, Sergey Levine, and Benjamin Eysenbach. Learning Temporal Distances : Contrastive Successor Features Can Provide a Metric Structure for Decision-Making . In International Conference on Machine Learning , 2024. URL https://proceedings.mlr...

    2024

  23. [31]

    Visual reinforcement learning with imagined goals

    Ashvin V Nair, Vitchyr Pong, Murtaza Dalal, Shikhar Bahl, Steven Lin, and Sergey Levine. Visual reinforcement learning with imagined goals. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (eds.), Advances in Neural Information Processing Sy...

    2018

  24. [32]

    Ng and Stuart J

    Andrew Y. Ng and Stuart J. Russell. Algorithms for inverse reinforcement learning. In Proceedings of the Seventeenth International Conference on Machine Learning, ICML '00, pp.\ 663–670, San Francisco, CA, USA, 2000. Morgan Kaufmann Publishers Inc. ISBN 1558607072

    2000

  25. [33]

    Octo: An open-source generalist robot policy

    Octo Model Team , Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Charles Xu, Jianlan Luo, Tobias Kreiman, You Liang Tan, Pannag Sanketi, Quan Vuong, Ted Xiao, Dorsa Sadigh, Chelsea Finn, and Sergey Levine. Octo: An open-source genera...

    2024

  26. [34]

    Representation Learning with Contrastive Predictive Coding

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation Learning with Contrastive Predictive Coding . arXiv:1807.03748 [cs, stat], July 2018. URL http://arxiv.org/abs/1807.03748. arXiv: 1807.03748

    2018 arXiv

  27. [35]

    Asymmetric self-play for automatic goal discovery in robotic manipulation

    OpenAI OpenAI, Matthias Plappert, Raul Sampedro, Tao Xu, Ilge Akkaya, Vineet Kosaraju, Peter Welinder, Ruben D'Sa, Arthur Petron, Henrique Pond \'e de Oliveira Pinto, Alex Paino, Hyeonwoo Noh, Lilian Weng, Qiming Yuan, Casey Chu, and Wojciech Zaremba. Asymmetric self-play for ...

    2021 arXiv

  28. [36]

    Efros, and Trevor Darrell

    Deepak Pathak, Parsa Mahmoudieh, Guanghao Luo, Pulkit Agrawal, Dian Chen, Fred Shentu, Evan Shelhamer, Jitendra Malik, Alexei A. Efros, and Trevor Darrell. Zero-shot visual imitation. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp....

    2018

  29. [37]

    Fast imitation via behavior foundation models

    Matteo Pirotta, Andrea Tirinzoni, Ahmed Touati, Alessandro Lazaric, and Yann Ollivier. Fast imitation via behavior foundation models. In B. Kim, Y. Yue, S. Chaudhuri, K. Fragkiadaki, M. Khan, and Y. Sun (eds.), International Conference on Representation Learning, volume 2024, ...

    2024

  30. [38]

    Maximum entropy gain exploration for long horizon multi-goal reinforcement learning

    Silviu Pitis, Harris Chan, Stephen Zhao, Bradly Stadie, and Jimmy Ba. Maximum entropy gain exploration for long horizon multi-goal reinforcement learning. In Proceedings of the 37th International Conference on Machine Learning, ICML'20. JMLR.org, 2020

    2020

  31. [39]

    Mars, and Sabine Hunnius

    Francesco Poli, Marlene Meyer, Rogier B. Mars, and Sabine Hunnius. Exploration in 4-year-old children is guided by learning progress and novelty. Child Development, 96 0 (1): 0 192--202, 2025. doi:https://doi.org/10.1111/cdev.14158. URL https://srcd.onlinelibrary.wiley.com/doi...

    2025 doi

  32. [40]

    Pomerleau

    Dean A. Pomerleau. Alvinn: an autonomous land vehicle in a neural network. In Proceedings of the 2nd International Conference on Neural Information Processing Systems, NIPS'88, pp.\ 305–313, Cambridge, MA, USA, 1988. MIT Press

    1988

  33. [41]

    Pong, Murtaza Dalal, Steven Lin, Ashvin Nair, Shikhar Bahl, and Sergey Levine

    Vitchyr H. Pong, Murtaza Dalal, Steven Lin, Ashvin Nair, Shikhar Bahl, and Sergey Levine. Skew-fit: state-covering self-supervised reinforcement learning. In Proceedings of the 37th International Conference on Machine Learning, ICML'20. JMLR.org, 2020

    2020

  34. [42]

    A generalist agent

    Scott Reed, Konrad Zolna, Emilio Parisotto, Sergio Gomez Colmenarejo, Alexander Novikov, Gabriel Barth-Maron, Mai Gimenez, Yury Sulsky, Jackie Kay, Jost Tobias Springenberg, Tom Eccles, Jake Bruce, Ali Razavi, Ashley Edwards, Nicolas Heess, Yutian Chen, Raia Hadsell, Oriol Vin...

    2022

  35. [43]

    A reduction of imitation learning and structured prediction to no-regret online learning

    Stephane Ross, Geoffrey Gordon, and Drew Bagnell. A reduction of imitation learning and structured prediction to no-regret online learning. In Geoffrey Gordon, David Dunson, and Miroslav Dudík (eds.), Proceedings of the Fourteenth International Conference on Artificial Intelli...

    2011

  36. [44]

    Universal value function approximators

    Tom Schaul, Daniel Horgan, Karol Gregor, and David Silver. Universal value function approximators. In Francis Bach and David Blei (eds.), Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of Machine Learning Research, pp.\ 1312--132...

    2015

  37. [45]

    Smile: Scalable meta inverse reinforcement learning through context-conditional policies

    Seyed Kamyar Seyed Ghasemipour, Shixiang (Shane) Gu, and Richard Zemel. Smile: Scalable meta inverse reinforcement learning through context-conditional policies. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d Alch\' e -Buc, E. Fox, and R. Garnett (eds.), Advances in Neural...

    2019

  38. [46]

    Mastering the game of go with deep neural networks and tree search

    David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Lillicrap, Madelei...

    2016

  39. [47]

    Stahl and Lisa Feigenson

    Aimee E. Stahl and Lisa Feigenson. Observing the unexpected enhances infants’ learning and exploration. Science, 348 0 (6230): 0 91--94, 2015. doi:10.1126/science.aaa3799. URL https://www.science.org/doi/abs/10.1126/science.aaa3799

    2015 doi

  40. [48]

    Learning one representation to optimize all rewards

    Ahmed Touati and Yann Ollivier. Learning one representation to optimize all rewards. In Proceedings of the 35th International Conference on Neural Information Processing Systems, NIPS '21, Red Hook, NY, USA, 2021. Curran Associates Inc. ISBN 9781713845393

    2021

  41. [49]

    Optimal goal-reaching reinforcement learning via quasimetric learning

    Tongzhou Wang, Antonio Torralba, Phillip Isola, and Amy Zhang. Optimal goal-reaching reinforcement learning via quasimetric learning. In International Conference on Machine Learning. PMLR, 2023

    2023

  42. [50]

    Unsupervised control through non-parametric discriminative rewards

    David Warde-Farley, Tom Van de Wiele, Tejas Kulkarni, Catalin Ionescu, Steven Hansen, and Volodymyr Mnih. Unsupervised control through non-parametric discriminative rewards. arXiv [cs.LG], November 2018

    2018

  43. [51]

    Outracing champion gran turismo drivers with deep reinforcement learning

    Peter R Wurman, Samuel Barrett, Kenta Kawamoto, James MacGlashan, Kaushik Subramanian, Thomas J Walsh, Roberto Capobianco, Alisa Devlic, Franziska Eckert, Florian Fuchs, Leilani Gilpin, Piyush Khandelwal, Varun Kompella, Haochih Lin, Patrick MacAlpine, Declan Oller, Takuma Sen...

    2022

  44. [52]

    Meta-inverse reinforcement learning with probabilistic context variables

    Lantao Yu, Tianhe Yu, Chelsea Finn, and Stefano Ermon. Meta-inverse reinforcement learning with probabilistic context variables. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d Alch\' e -Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, ...

    2019

  45. [53]

    One-shot imitation from observing humans via domain-adaptive meta-learning

    Tianhe Yu, Chelsea Finn, Annie Xie, Sudeep Dasari, Tianhao Zhang, Pieter Abbeel, and Sergey Levine. One-shot imitation from observing humans via domain-adaptive meta-learning. arXiv [cs.LG], February 2018

    2018

  46. [54]

    Maximum entropy inverse reinforcement learning

    Brian D Ziebart, Andrew L Maas, J Bagnell, and A Dey. Maximum entropy inverse reinforcement learning. National Conference on Artificial Intelligence, pp.\ 1433--1438, July 2008

    2008

  47. [55]

    Brown, and Anca D

    Matthew Zurek, Andreea Bobu, Daniel S. Brown, and Anca D. Dragan. Situational confidence assistance for lifelong shared autonomy. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pp.\ 2783–2789. IEEE Press, 2021. doi:10.1109/ICRA48506.2021.9561839. URL ...

    2021

  48. [56]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  49. [57]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  50. [58]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  51. [59]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.