Pith. sign in

REVIEW 3 major objections 5 minor 39 references

Decoupling slow skill planning from fast action control stops codebook collapse and improves multi-step language-conditioned robot learning.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 10:22 UTC pith:X6RR6VMI

load-bearing objection Solid hierarchical IL systems paper with a clean train-only diffusion + async VQ/DT split; gains look real, but causal credit for “dual-process” is overstated relative to per-env tuning. the 3 major comments →

arxiv 2607.10625 v1 pith:X6RR6VMI submitted 2026-07-12 cs.RO

Dual-Process Atomic Skill Learning: Decoupling Semantic Reasoning and Real-Time Control

Jun Chen , Erdent Bao , Wenlong Dong , Jierui Liu , Qi Cai , Hao Wan , Shaopeng Li , Weijun Qin
show 2 more authors
Jing Liang Huiping Zhuang
This is my paper
classification cs.RO
keywords language-conditioned imitation learninghierarchical skill learningvector quantizationlatent diffusionDecision Transformerasynchronous dual-process controlcompositional generalizationrobotic manipulation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Robots that follow natural language still struggle when a single instruction chains several sub-goals. Hierarchical methods try to learn reusable atomic skills, but jointly training the skill planner and the action generator often destabilizes the discrete skill library, so many codes go unused. DASL treats skill reasoning and motion control as two different timescales: a slow policy maps language and sparse observations into discrete skills via vector quantization, while a high-frequency Decision Transformer produces continuous actions conditioned on those skills. During training only, a skill-conditioned latent diffusion model regularizes the fused embeddings so that distinct behaviors occupy distinct codes. Asynchronous coordination plus this training-only regularizer is claimed to yield more stable skill acquisition and stronger compositional generalization on simulation benchmarks and a real-arm experiment than prior hierarchical baselines.

Core claim

An asynchronous dual-process hierarchy—slow-frequency Vector-Quantized skill planning plus a high-frequency Decision Transformer whose latent space is structured by skill-conditioned diffusion only during training—mitigates the skill-codebook interference that arises when high-level reasoning and low-level action generation are tightly coupled, producing more usable atomic skills and better generalization to unseen multi-step language instructions.

What carries the argument

DASL's asynchronous dual-process loop: a Slow-Frequency Policy that predicts continuous latents then quantizes them into a discrete skill codebook, fused by repetition into a High-Frequency Decision Transformer whose trajectory embeddings are regularized by a training-only latent diffusion loss.

Load-bearing premise

That a fixed update interval between skill planning and action generation, together with a training-only diffusion regularizer on fused embeddings, is enough to discover reusable discrete atomic skills from offline language-trajectory data and is the main reason codebook collapse decreases.

What would settle it

Train DASL and a synchronous joint-training counterpart on the same LOReL or Kitchen trajectories with matched encoders and codebook size; if the asynchronous version still shows severe codebook collapse (few codes used) or fails to improve success rates on held-out compositional instructions, the central claim fails.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes DASL, an asynchronous hierarchical language-conditioned imitation learning framework that decouples a slow-frequency Option Transformer with VQ skill quantization from a high-frequency Decision Transformer, regularized during training by a skill-conditioned latent diffusion model on fused observation–skill embeddings (and removed at inference). The central claim is that this multi-timescale decoupling plus train-only diffusion mitigates skill codebook interference/collapse common in joint hierarchical training (e.g., LISA) and improves atomic skill acquisition and compositional generalization. Evidence includes multi-seed results on LOReL (rephrasals and compositional tasks), Franka Kitchen (N/K-rates), CALVIN, BabyAI (data regimes), component ablations (Table 5; Appendix F), efficiency comparisons (Table 6), skill utilization visualizations (Fig. 13), and a small Realman real-robot study versus LISA.

Significance. If the results hold under fair controls, DASL is a useful systems contribution to hierarchical language-conditioned IL: multi-rate skill planning with a train-only latent diffusion regularizer and a fast DT executor is a practical design pattern that improves codebook utilization and compositional success while preserving real-time inference (Table 6). Strengths include multi-benchmark evaluation with seed stds, progressive ablations, skill heatmaps against LISA collapse, efficiency reporting, a real-robot deployment, and a public GitHub. The dual-process framing is mainly motivational; the technical value lies in the async schedule, VQ skills, and diffusion-as-regularizer combination relative to LISA, SkillDiffuser, and LADS.

major comments (3)
  1. Causal attribution of gains to asynchronous dual-process decoupling is only partially isolated. Table 5 (LOReL) shows large jumps when adding async then diffusion, and Fig. 13 shows better codebook use than LISA, but Table 11 reveals strongly environment-specific free parameters (I ∈ {2,5,20}, λ_diff ∈ {0.5,0.01,0.1}, K, horizon, batch size, iterations). Appendix F ablations remain mostly LOReL-centric and do not re-tune LISA/LADS under a matched (I, K, λ_vq, λ_diff, encoders, horizon) suite while swapping only the async schedule or the diffusion term. Please either (i) report controlled re-runs of the main baselines with the same I/K/loss weights/encoders, or (ii) clearly qualify that SOTA margins (Tables 1–4) may partly reflect hybrid capacity and per-environment tuning, not dual-process decoupling alone.
  2. Baseline fairness and reporting consistency need tightening. Several LOReL/Kitchen/CALVIN numbers are mixed between original papers and re-evaluations (Table 1 notes; Appendix C). The claim of matching LISA’s vision/language encoders is stated for DASL, but it is unclear whether SkillDiffuser, LADS, and LCSD re-runs used identical observation pipelines, context length L, and evaluation protocols (50 episodes). Please add a single comparison protocol table (encoders, state dim, horizon, eval episodes, seeds) and mark every cited vs. re-run entry explicitly so the SOTA claim is auditable.
  3. The real-robot experiment (Appendix G; Fig. 5; Table 21) supports practical viability but is too small and imbalanced (60/40/20 atomic demos; 10 compositional; 10–20 trials) to independently confirm the mechanism. Success rates of 88% vs LISA 56% are encouraging, yet without variance across seeds/policies, camera/viewpoint variation, or a stronger hierarchical baseline (e.g., LADS/SkillDiffuser-style), the experiment mainly illustrates deployment. Please report trial-level variance, clarify whether the same hyperparameters as simulation were used, and temper claims that the real-robot results “demonstrate” the dual-process mechanism.
minor comments (5)
  1. Multiple typos and formatting issues: “expermentt/experment”, “A verage”, “V ector Quantization”, “word-skill” heatmaps; standardize “rephrasal” vs “rewriting” and fix Table 1 asterisk explanation placement.
  2. Notation: clarify 1-based strided timeline T_slow, the exact gradient accumulation over I steps into the slow policy, and whether STE gradients from L_action reach the codebook only every I steps (Method §§3.3–3.5).
  3. Figure 1 and Figure 2 are dense; label which modules are train-only (diffusion U-Net) vs. inference-time more prominently. Fig. 13 is informative but hard to read at print size—consider a simplified utilization bar chart.
  4. Related work: position more sharply against LADS (already decouples high-level planning from latent plans) and dual-system VLAs; state what is new beyond multi-rate options + train-only diffusion + DT.
  5. Appendix F.1 frequency-ratio sensitivity is valuable; consider promoting a short summary of the I heuristic into the main text so readers know I is not an unrestricted free parameter.

Circularity Check

0 steps flagged

No significant circularity: empirical hierarchical IL architecture with standard losses and external task-success metrics.

full rationale

DASL is a systems/empirical paper proposing an asynchronous hierarchical policy (slow VQ skill planner + high-frequency Decision Transformer regularized by training-only skill-conditioned latent diffusion). The training objectives (action MSE L_action, VQ commitment L_vq, diffusion noise-prediction L_diff) are standard and do not algebraically redefine the reported success rates, N-rates, or K-rates. Those metrics are external environment outcomes measured on held-out instructions and real-robot trials. There is no self-definitional loop (skills are not defined via the success metric), no fitted free parameter re-labeled as a prediction of a closely related quantity, no load-bearing uniqueness theorem imported from overlapping authors, and no ansatz smuggled via self-citation. Citations to LISA, SkillDiffuser, LADS, Dual-Process Theory, etc., serve as baselines or inspiration and are not used to force the central claim by construction. Ablations and hyper-parameter tables are ordinary empirical practice; they do not create circular derivation. The paper is self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 2 invented entities

The central claim rests on standard imitation-learning and VQ/diffusion machinery plus several design choices treated as free knobs (skill interval, codebook size, loss weights) and domain assumptions that language tasks decompose into reusable discrete skills discoverable without subgoal labels. No new physical entity is postulated; the invented pieces are architectural modules whose value is argued empirically.

free parameters (5)
  • slow-frequency interval I
    Fixed skill-update stride (e.g., 2 on LOReL, 20 on Kitchen); ablated in Table 15 and treated as a key design choice that defines asynchrony.
  • skill codebook size K / number of options
    Discrete library size (typically 20; 30 on Kitchen image); sensitivity shown in Table 19; chosen by validation rather than derived.
  • λ_vq and λ_diff loss weights
    Trade off action cloning, VQ commitment, and diffusion regularization; diffusion weight varies by dataset (0.5 to 0.01 in Table 11).
  • diffusion timesteps, blocks, horizon H
    Training-only diffusion schedule and trajectory prefix length set per environment; not predicted from theory.
  • option/codebook embedding dimensions
    Option dim 128 and codebook dim 16 are architectural free choices listed in Table 11.
axioms (5)
  • domain assumption Multi-step language-conditioned tasks admit reusable atomic skills that can be discovered unsupervised from offline language-trajectory pairs without sub-task labels.
    Stated in Problem Setup §3.2 and Introduction; load-bearing for the skill-library framing.
  • domain assumption Vector quantization of continuous skill pre-activations yields an interpretable discrete skill library suitable for composition.
    §3.3 Discrete Skill Vector Quantization; inherited from LISA-style hierarchical IL.
  • ad hoc to paper Decoupling high-level semantic updates from high-frequency action gradients reduces mutual interference and codebook collapse relative to synchronous joint training.
    Core hypothesis from Dual-Process inspiration (§1, §3.1); supported by ablations but not independently proven.
  • ad hoc to paper A latent diffusion objective on fused skill-observation embeddings regularizes the trajectory manifold without needing iterative denoising at inference.
    §3.4–3.5; key design that distinguishes DASL from SkillDiffuser-style inference diffusion.
  • standard math Standard behavioral cloning / Decision Transformer / VQ / DDPM math and STE gradients are valid optimization tools for offline IL.
    Used throughout Method and Algorithm 1 without novel proof obligations.
invented entities (2)
  • DASL asynchronous dual-process hierarchical policy (Slow-Frequency Option Transformer + High-Frequency DT) no independent evidence
    purpose: Separate semantic skill selection from real-time action generation on different timescales.
    Primary architectural contribution; evidence is empirical benchmark gains, not external independent measurement of a new natural object.
  • Skill-conditioned latent trajectory diffusion regularizer (train-only) no independent evidence
    purpose: Structure latent skill-trajectory space to improve codebook utilization and skill-action alignment without inference latency.
    Introduced as auxiliary module removed at rollout; validated by ablations vs AE/VAE and dual-diffusion variants.

pith-pipeline@v1.1.0-grok45 · 28613 in / 3725 out tokens · 35482 ms · 2026-07-14T10:22:29.074742+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Dual-Process Atomic Skill Learning: Decoupling Semantic Reasoning and Real-Time Control." pith.science (2026). https://pith.science/paper/X6RR6VMI

@misc{pith2026260710625,
  author       = {Pith},
  title        = {Pith review of: Dual-Process Atomic Skill Learning: Decoupling Semantic Reasoning and Real-Time Control},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/X6RR6VMI}},
  note         = {Machine review of arXiv:2607.10625}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Language-conditioned Imitation Learning (IL) is essential for enabling robots to perform complex tasks following natural language instructions. However, generalizing to multi-step compositional tasks remains a significant challenge. While hierarchical approaches attempt to address this by decomposing tasks into atomic skills, existing methods often suffer from training instability and codebook collapse due to the tight coupling between high-level skill reasoning and low-level action generation in joint training paradigms. Inspired by the Dual-Process Theory of cognition, we propose Dual-Process Atomic Skill Learning (DASL), a novel asynchronous hierarchical imitation learning framework that decouples slow semantic reasoning from fast, real-time motion control. DASL comprises a Slow-Frequency Policy that predicts interpretable, discrete skills via Vector Quantization, and a High-Frequency Policy that leverages a latent diffusion model and a Decision Transformer to generate precise actions conditioned on these latent skills. By asynchronously coordinating these modules and utilizing diffusion to structure the latent space, our framework mitigates the skill codebook interference problem common in joint training paradigms. Evaluations across simulation benchmarks and experiment demonstrate that DASL significantly outperforms state-of-the-art baselines, excelling in skill acquisition and compositional generalization to unseen instructions. GitHub page: https://github.com/Hatakekaka/DASL

Figures

Figures reproduced from arXiv: 2607.10625 by Erdent Bao, Hao Wan, Huiping Zhuang, Jierui Liu, Jing Liang, Jun Chen, Qi Cai, Shaopeng Li, Weijun Qin, Wenlong Dong.

Figure 1
Figure 1. Figure 1: Baselines vs. our dual-process control. (a) Baselines: synchronous high-frequency skill and action generation. (b) Ours: asynchronous decoupling, with slow-frequency skill reasoning and high-frequency action execution. Language-conditioned imitation learning (IL) aims to train robots to execute tasks specified by natural lan￾guage instructions using paired language-trajectory demonstrations. However, this … view at source ↗
Figure 2
Figure 2. Figure 2: Overview of DASL, an asynchronous hierarchical imitation learning framework. The [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: K-rates on seen and unseen tasks in Kitchen (image). 4 Experiments In this section, we evaluate DASL on robotic manipulation and grid-world navigation tasks, comparing it against strong hierarchical and non-hierarchical baselines. We further analyze the generalization of the framework to compositional instructions, the contributions of the architectural components, the efficiency of inference, and the prac… view at source ↗
Figure 4
Figure 4. Figure 4: Success rates (%) on BabyAI GoToSeq task with vary￾ing numbers of demonstrations. to other Decision Transformer-based methods (e.g., Lang DT and LISA), our results underscore the necessity of modeling decision-making within a latent vector space. This approach effectively mitigates inherent skill learning instability and resolves LISA’s index collapse. Compared to SkillDif￾fuser, we validate that skill-con… view at source ↗
Figure 5
Figure 5. Figure 5: Real-world quantitative and qualitative results for DASL. (a) compares average success [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Simulation environments for manipulation and navigation tasks. [PITH_FULL_IMAGE:figures/full_fig_p016_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: LOReL State: open drawer and move black mug right [PITH_FULL_IMAGE:figures/full_fig_p019_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: LOReL Image: turn faucet right and close drawer [PITH_FULL_IMAGE:figures/full_fig_p019_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Kitchen State: activate bottom burner and activate top burner and turn on light switch and [PITH_FULL_IMAGE:figures/full_fig_p020_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Kitchen Image: activate bottom burner and activate top burner and turn on light switch [PITH_FULL_IMAGE:figures/full_fig_p020_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: CALVIN: push the sliding door to the left side [PITH_FULL_IMAGE:figures/full_fig_p021_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: CALVIN: use the switch to turn on the light bulb [PITH_FULL_IMAGE:figures/full_fig_p021_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Qualitative comparison of learned skill representations for LISA and DASL on LOReL [PITH_FULL_IMAGE:figures/full_fig_p022_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Word cloud of skills learned in LOReL Sawyer (state) compositional tasks. [PITH_FULL_IMAGE:figures/full_fig_p022_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Visualize two LOReL episodes with aligned keyframes and the discrete skill index selected [PITH_FULL_IMAGE:figures/full_fig_p023_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: Evolution of latent space representations across different stages. t-SNE visualizations [PITH_FULL_IMAGE:figures/full_fig_p023_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: Visualization of Latent Skill Distributions. Comparing skill codebook usage between [PITH_FULL_IMAGE:figures/full_fig_p024_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Real-world experimental setup. We evaluate the proposed DASL framework on a Realman robot. The setup involves various interactive objects, such as a drawer and a banana on a plate, to validate the performance of the model across atomic and composite manipulation tasks. To assess the robustness of the proposed method, we execute each task for 20 trials to compute the average success rate. As shown in [PIT… view at source ↗
Figure 19
Figure 19. Figure 19: Qualitative comparison on compositional tasks. While LISA fails via incomplete closing [PITH_FULL_IMAGE:figures/full_fig_p028_19.png] view at source ↗
Figure 20
Figure 20. Figure 20: Qualitative visualizations of the proposed DASL framework executing atomic and [PITH_FULL_IMAGE:figures/full_fig_p028_20.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

39 extracted references · 11 linked inside Pith

  1. [1]

    Do as i can, not as i say: Grounding language in robotic affordances.arXiv preprint arXiv:2204.01691, 2022

    Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Chuyuan Fu, Keerthana Gopalakrishnan, Karol Hausman, et al. Do as i can, not as i say: Grounding language in robotic affordances.arXiv preprint arXiv:2204.01691, 2022

    Pith/arXiv arXiv 2022

  2. [2]

    Gr00t n1: An open foundation model for generalist humanoid robots.arXiv preprint arXiv:2503.14734, 2025

    Johan Bjorck, Fernando Castañeda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi Fan, Yu Fang, Dieter Fox, Fengyuan Hu, Spencer Huang, et al. Gr00t n1: An open foundation model for generalist humanoid robots.arXiv preprint arXiv:2503.14734, 2025

    Pith/arXiv arXiv 2025

  3. [3]

    Language models are few-shot learners.Advances in neural information processing systems, 33:1877–1901, 2020

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners.Advances in neural information processing systems, 33:1877–1901, 2020

    1901

  4. [4]

    Fast-in-slow: A dual-system vla model unifying fast manipulation within slow reasoning

    Hao Chen, Jiaming Liu, Chenyang Gu, Zhuoyang Liu, Renrui Zhang, Xiaoqi Li, Xiao He, Yandong Guo, Chi-Wing Fu, Shanghang Zhang, et al. Fast-in-slow: A dual-system vla model unifying fast manipulation within slow reasoning. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems, 2024

    2024

  5. [5]

    Babyai: A platform to study the sample efficiency of grounded language learning.arXiv preprint arXiv:1810.08272, 2018

    Maxime Chevalier-Boisvert, Dzmitry Bahdanau, Salem Lahlou, Lucas Willems, Chitwan Saharia, Thien Huu Nguyen, and Yoshua Bengio. Babyai: A platform to study the sample efficiency of grounded language learning.arXiv preprint arXiv:1810.08272, 2018

    Pith/arXiv arXiv 2018

  6. [6]

    Diversity is all you need: Learning skills without a reward function.arXiv preprint arXiv:1802.06070, 2018

    Benjamin Eysenbach, Abhishek Gupta, Julian Ibarz, and Sergey Levine. Diversity is all you need: Learning skills without a reward function.arXiv preprint arXiv:1802.06070, 2018

    Pith/arXiv arXiv 2018

  7. [7]

    One-shot visual imitation learning via meta-learning

    Chelsea Finn, Tianhe Yu, Tianhao Zhang, Pieter Abbeel, and Sergey Levine. One-shot visual imitation learning via meta-learning. InConference on robot learning, pages 357–368. PMLR, 2017

    2017

  8. [8]

    Lisa: Learning interpretable skill abstractions from language.Advances in Neural Information Processing Systems, 35:21711–21724, 2022

    Divyansh Garg, Skanda Vaidyanath, Kuno Kim, Jiaming Song, and Stefano Ermon. Lisa: Learning interpretable skill abstractions from language.Advances in Neural Information Processing Systems, 35:21711–21724, 2022

    2022

  9. [9]

    Language-driven multi-task manipulation with action-mask-enhanced multimodal learning.IEEE Robotics and Automation Letters, 2026

    Ningquan Gu, Yuquan Li, Kazuhiro Kosuge, and Mitsuhiro Hayashibe. Language-driven multi-task manipulation with action-mask-enhanced multimodal learning.IEEE Robotics and Automation Letters, 2026

    2026

  10. [10]

    Relay policy learning: Solving long-horizon tasks via imitation and reinforcement learning.arXiv preprint arXiv:1910.11956, 2019

    Abhishek Gupta, Vikash Kumar, Corey Lynch, Sergey Levine, and Karol Hausman. Relay policy learning: Solving long-horizon tasks via imitation and reinforcement learning.arXiv preprint arXiv:1910.11956, 2019

    Pith/arXiv arXiv 1910

  11. [11]

    Learning an embedding space for transferable robot skills

    Karol Hausman, Jost Tobias Springenberg, Ziyu Wang, Nicolas Heess, and Martin Riedmiller. Learning an embedding space for transferable robot skills. InInternational Conference on Learning Representations, 2018

    2018

  12. [12]

    Discrete latent plans via semantic skill abstractions

    Haobin Jiang, Jiangxing Wang, and Zongqing Lu. Discrete latent plans via semantic skill abstractions. InICLR, 2025

    2025

  13. [13]

    Rethinking mutual information for language conditioned skill discovery on imitation learning

    Zhaoxun Ju, Chao Yang, Fuchun Sun, Hongbo Wang, and Yu Qiao. Rethinking mutual information for language conditioned skill discovery on imitation learning. InProceedings of the International Conference on Automated Planning and Scheduling, volume 34, pages 301–309, 2024

    2024

  14. [14]

    Uniskill: Imitating human videos via cross-embodiment skill representations.arXiv preprint arXiv:2505.08787, 2025

    Hanjung Kim, Jaehyun Kang, Hyolim Kang, Meedeum Cho, Seon Joo Kim, and Youngwoon Lee. Uniskill: Imitating human videos via cross-embodiment skill representations.arXiv preprint arXiv:2505.08787, 2025

    arXiv 2025

  15. [15]

    Openvla: An open-source vision-language-action model.arXiv preprint arXiv:2406.09246, 2024

    Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. Openvla: An open-source vision-language-action model.arXiv preprint arXiv:2406.09246, 2024

    Pith/arXiv arXiv 2024

  16. [16]

    Skill discovery in continuous reinforcement learning domains using skill chaining.Advances in neural information processing systems, 22, 2009

    George Konidaris and Andrew Barto. Skill discovery in continuous reinforcement learning domains using skill chaining.Advances in neural information processing systems, 22, 2009. 10

    2009

  17. [17]

    Skilldiffuser: Interpretable hierarchical planning via skill abstractions in diffusion-based task execution

    Zhixuan Liang, Yao Mu, Hengbo Ma, Masayoshi Tomizuka, Mingyu Ding, and Ping Luo. Skilldiffuser: Interpretable hierarchical planning via skill abstractions in diffusion-based task execution. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16467–16476, 2024

    2024

  18. [18]

    Delta: Decomposed efficient long-term robot task planning using large language models

    Yuchen Liu, Luigi Palmieri, Sebastian Koch, Ilche Georgievski, and Marco Aiello. Delta: Decomposed efficient long-term robot task planning using large language models. In2025 IEEE International Conference on Robotics and Automation (ICRA), pages 10995–11001. IEEE, 2025

    2025

  19. [19]

    Learning latent plans from play

    Corey Lynch, Mohi Khansari, Ted Xiao, Vikash Kumar, Jonathan Tompson, Sergey Levine, and Pierre Sermanet. Learning latent plans from play. InConference on robot learning, pages 1113–1132. Pmlr, 2020

    2020

  20. [20]

    Walk the talk: Connecting language, knowledge, and action in route instructions.Def, 2(6):4, 2006

    Matt MacMahon, Brian Stankiewicz, and Benjamin Kuipers. Walk the talk: Connecting language, knowledge, and action in route instructions.Def, 2(6):4, 2006

    2006

  21. [21]

    Calvin: A benchmark for language-conditioned policy learning for long-horizon robot manipulation tasks.IEEE Robotics and Automation Letters, 7(3):7327–7334, 2022

    Oier Mees, Lukas Hermann, Erick Rosete-Beas, and Wolfram Burgard. Calvin: A benchmark for language-conditioned policy learning for long-horizon robot manipulation tasks.IEEE Robotics and Automation Letters, 7(3):7327–7334, 2022

    2022

  22. [22]

    Visual reinforcement learning with imagined goals.Advances in neural information processing systems, 31, 2018

    Ashvin V Nair, Vitchyr Pong, Murtaza Dalal, Shikhar Bahl, Steven Lin, and Sergey Levine. Visual reinforcement learning with imagined goals.Advances in neural information processing systems, 31, 2018

    2018

  23. [23]

    Learning language- conditioned robot behavior from offline data and crowd-sourced annotation

    Suraj Nair, Eric Mitchell, Kevin Chen, Silvio Savarese, Chelsea Finn, et al. Learning language- conditioned robot behavior from offline data and crowd-sourced annotation. InConference on Robot Learning, pages 1303–1315. PMLR, 2022

    2022

  24. [24]

    R3m: A universal visual representation for robot manipulation.arXiv preprint arXiv:2203.12601, 2022

    Suraj Nair, Aravind Rajeswaran, Vikash Kumar, Chelsea Finn, and Abhinav Gupta. R3m: A universal visual representation for robot manipulation.arXiv preprint arXiv:2203.12601, 2022

    Pith/arXiv arXiv 2022

  25. [25]

    Dinov2: Learning robust visual features without supervision.arXiv preprint arXiv:2304.07193, 2023

    Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision.arXiv preprint arXiv:2304.07193, 2023

    Pith/arXiv arXiv 2023

  26. [26]

    Latent plans for task-agnostic offline reinforcement learning

    Erick Rosete-Beas, Oier Mees, Gabriel Kalweit, Joschka Boedecker, and Wolfram Burgard. Latent plans for task-agnostic offline reinforcement learning. InConference on Robot Learning, pages 1838–1849. PMLR, 2023

    2023

  27. [27]

    From llms to actions: Latent codes as bridges in hierarchical robot control

    Yide Shentu, Philipp Wu, Aravind Rajeswaran, and Pieter Abbeel. From llms to actions: Latent codes as bridges in hierarchical robot control. In2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 8539–8546. IEEE, 2024

    2024

  28. [28]

    Cliport: What and where pathways for robotic manipulation

    Mohit Shridhar, Lucas Manuelli, and Dieter Fox. Cliport: What and where pathways for robotic manipulation. InConference on robot learning, pages 894–906. PMLR, 2022

    2022

  29. [29]

    Rationalvla: A rational vision-language- action model with dual system.arXiv preprint arXiv:2506.10826, 2025

    Wenxuan Song, Jiayi Chen, Wenxue Li, Xu He, Han Zhao, Can Cui, Pengxiang Ding Shiyan Su, Feilong Tang, Xuelian Cheng, Donglin Wang, et al. Rationalvla: A rational vision-language- action model with dual system.arXiv preprint arXiv:2506.10826, 2025

    Pith/arXiv arXiv 2025

  30. [30]

    Language-conditioned imitation learning for robot manipulation tasks.Advances in Neural Information Processing Systems, 33:13139–13150, 2020

    Simon Stepputtis, Joseph Campbell, Mariano Phielipp, Stefan Lee, Chitta Baral, and Heni Ben Amor. Language-conditioned imitation learning for robot manipulation tasks.Advances in Neural Information Processing Systems, 33:13139–13150, 2020

    2020

  31. [31]

    Rold: Robot latent diffusion for multi-task policy modeling

    Wenhui Tan, Bei Liu, Junbo Zhang, Ruihua Song, and Jianlong Fu. Rold: Robot latent diffusion for multi-task policy modeling. InInternational Conference on Multimedia Modeling, pages 340–353. Springer, 2024

    2024

  32. [32]

    Understanding natural language commands for robotic navigation and mobile manipulation

    Stefanie Tellex, Thomas Kollar, Steven Dickerson, Matthew Walter, Ashis Banerjee, Seth Teller, and Nicholas Roy. Understanding natural language commands for robotic navigation and mobile manipulation. InProceedings of the AAAI conference on artificial intelligence, volume 25, pages 1507–1514, 2011. 11

    2011

  33. [33]

    Mujoco: A physics engine for model-based control

    Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In2012 IEEE/RSJ international conference on intelligent robots and systems, pages 5026–5033. IEEE, 2012

    2012

  34. [34]

    Lotus: Continual imitation learning for robot manipulation through unsupervised skill discovery

    Weikang Wan, Yifeng Zhu, Rutav Shah, and Yuke Zhu. Lotus: Continual imitation learning for robot manipulation through unsupervised skill discovery. In2024 IEEE International Conference on Robotics and Automation (ICRA), pages 537–544. IEEE, 2024

    2024

  35. [35]

    Rethinking visual-language-action model scaling: Alignment, mixture, and regularization.arXiv preprint arXiv:2602.09722, 2026

    Ye Wang, Sipeng Zheng, Hao Luo, Wanpeng Zhang, Haoqi Yuan, Chaoyi Xu, Haiweng Xu, Yicheng Feng, Mingyang Yu, Zhiyu Kang, et al. Rethinking visual-language-action model scaling: Alignment, mixture, and regularization.arXiv preprint arXiv:2602.09722, 2026

    arXiv 2026

  36. [36]

    Dynamind: Reasoning over abstract video dynamics for embodied decision- making

    Ziru Wang, Mengmeng Wang, Jade Dai, Teli Ma, Guo-Jun Qi, Yong Liu, Guang Dai, and Jingdong Wang. Dynamind: Reasoning over abstract video dynamics for embodied decision- making. InForty-second International Conference on Machine Learning, 2025

    2025

  37. [37]

    Dexvla: Vision-language model with plug-in diffusion expert for general robot control.arXiv preprint arXiv:2502.05855, 2025

    Junjie Wen, Yichen Zhu, Jinming Li, Zhibin Tang, Chaomin Shen, and Feifei Feng. Dexvla: Vision-language model with plug-in diffusion expert for general robot control.arXiv preprint arXiv:2502.05855, 2025

    Pith/arXiv arXiv 2025

  38. [38]

    Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning

    Tianhe Yu, Deirdre Quillen, Zhanpeng He, Ryan Julian, Karol Hausman, Chelsea Finn, and Sergey Levine. Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning. InConference on robot learning, pages 1094–1100. PMLR, 2020

    2020

  39. [39]

    go to a box and go to the purple door, then go to the grey door

    Jesse Zhang, Minho Heo, Zuxin Liu, Erdem Biyik, Joseph J Lim, Yao Liu, and Rasool Fakoor. Extract: Efficient policy learning by extracting transferable robot skills from offline data.arXiv preprint arXiv:2406.17768, 2024. 12 Appendix A Pseudo-code of Training DASL Algorithm 1 illustrates the training pseudocode of the DASL algorithm, which describes the m...

    Pith/arXiv arXiv 2024