Pith. sign in

REVIEW 2 major objections 5 minor 175 references

Bimanual VLA recipes for smooth multi-arm control also fit drones, and dual-system designs are the practical path to real deployment.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

T0 review · grok-4.5

2026-07-10 22:59 UTC pith:CLSO7ZPK

load-bearing objection Solid cross-domain survey that actually maps bimanual VLA recipes onto aerial systems; transfer claim is useful analogy, not new experiment. the 2 major comments →

arxiv 2607.06706 v1 pith:CLSO7ZPK submitted 2026-07-07 cs.RO cs.AIcs.LG

Vision Language Action (VLA) Models for Unmanned Aerial Robotics and Bimanual Manipulation: A Review

classification cs.RO cs.AIcs.LG
keywords Vision–Language–Action modelsbimanual manipulationunmanned aerial roboticsflow matchingaction chunkingimitation learningdual-system architecturesworld models
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This review of 183 papers argues that Vision–Language–Action models, which turn camera images and plain-language commands into motor commands, have become the main learning framework for robot manipulation, with two-arm coordination as the hardest test. The same structural problem appears in unmanned aerial robotics: a drone must coordinate thrust, attitude, and often gripper commands under tight latency and payload limits. The authors show that the recipes already proven for bimanual arms—continuous chunked action generation (especially flow matching and hybrid designs), diverse cross-embodiment co-training, and reinforcement learning from autonomous practice—transfer to aerial systems. They further conclude that the field is converging on dual-system architectures that pair a slower reasoning module with a faster action module, rather than on a single monolithic end-to-end model. The paper maps fourteen open directions that span both domains, from standardized two-arm benchmarks and safety certification to end-to-end drone VLAs and continuous self-improvement pipelines that close the gap between laboratory scores and industrial reliability.

Core claim

The coordination strategies, training recipes, and action representations developed for bimanual VLAs transfer to unmanned aerial systems. Continuous, chunked action generation—especially flow-matching and hybrid designs—avoids the quantization and latency bottlenecks of earlier approaches and is the converging solution for tightly coordinated control in both domains; dual-system (slow reasoner + fast actor) architectures are the practical path to real-world deployment.

What carries the argument

Action chunking with continuous generative heads (flow matching and hybrids): instead of emitting one discretized action token at a time, the model predicts a short future trajectory of continuous multi-dimensional actions in a single forward pass, amortizing expensive vision–language inference while preserving inter-actuator correlations needed for two arms or for thrust–attitude–gripper coupling.

Load-bearing premise

The paper treats the structural analogy between two multi-joint arms and a single underactuated drone (plus optional gripper) as deep enough that the same chunk lengths, flow schedules, and co-training ratios remain near-optimal, yet the transfer is argued mainly by side-by-side literature rather than controlled cross-embodiment experiments.

What would settle it

A controlled experiment that trains the same continuous-chunk flow-matching VLA on matched bimanual and aerial-manipulation tasks (identical visual backbone, chunk horizon, and co-training mix) and measures whether the aerial policy needs systematically different horizons or denoising steps to match bimanual success rates under wind and latency constraints; large, systematic divergence would undermine the claimed transfer.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. This review unifies Vision–Language–Action (VLA) models for bimanual manipulation and unmanned aerial robotics. It surveys 183 works (2017–2026) along a seven-dimension taxonomy (architectures, training recipes, action representations, bimanual coordination, UAV navigation/control, language grounding, and cross-cutting concerns). The central claim is that continuous, chunked action generation—especially flow matching and hybrid designs—avoids autoregressive quantization and multi-step diffusion latency, that training strategy (cross-embodiment diversity, co-training, RECAP-style RL) matters as much as architecture, and that dual-system (slow reasoner + fast actor) designs are the practical path to deployment. The authors argue that bimanual coordination strategies, training recipes, and action representations transfer to aerial systems and list fourteen research directions spanning both domains.

Significance. If the synthesis holds, the paper supplies a timely, cross-embodiment map of a fast-moving field that has lacked a joint treatment of bimanual VLAs and aerial VLAs. Strengths include the breadth of coverage (~183 works), explicit comparison tables (Tables 2–6, 8–15), repeated caveats on self-reported industrial numbers and non-comparable success rates, and a concrete fourteen-direction roadmap. The dual-system and continuous-chunking conclusions are well supported by the cited corpus and align with recent industrial practice (Gemini Robotics, GR00T N1, Helix). As a review, the contribution is organizational and transfer-by-analogy rather than new controlled experiments; that is appropriate to the genre and still useful for both communities.

major comments (2)
  1. The load-bearing transfer claim (Abstract; §8–9; Findings 1, 3, 11–13) rests on structural analogy—joint high-dimensional action chunks for coupled actuators under shared visual–language conditioning—illustrated by Flying Hand’s ACT reuse, dual-arm aerial harvesting as leader–follower, and multi-drone joint spaces. The analogy is coherent and repeatedly flagged, but the manuscript does not report (and does not claim) controlled cross-embodiment ablations that hold flow-matching schedule, chunk horizon H, and co-training ratio fixed across a bimanual arm and a quadrotor. For a review this is a scope limitation rather than an internal error; still, the claim would be stronger if §9 or §12.2 stated more explicitly which hyperparameters are expected to transfer unchanged versus which must be re-tuned for underactuation, ≥100 Hz flight, and outdoor dynamics, and if the corresponding open expe
  2. Table 8 and Figure 10 present approximate success rates and trend lines for bimanual tasks (laundry folding, box assembly, etc.) drawn from original publications under varying protocols. The caption of Table 8 correctly warns that values are not directly comparable, and Figure 10 is labeled “approximate.” Nonetheless, the Discussion (§12.1) and Highlights still treat the rise from ~30% (ACT) to >90% (π*_0) as a primary narrative of progress. A short additional paragraph quantifying protocol differences (object sets, success criteria, number of trials) or restricting the figure to methods evaluated on a common subset would keep the narrative from being read as a strict ranking.
minor comments (5)
  1. Notation: Table 1 and §2.1 introduce o_t, ℓ, q_t, A_t, H, and the flow-matching symbols consistently; a few later sections re-use t for both discrete control steps and continuous flow time without restating the distinction already made in §2.3.
  2. Table 4 and Table 8: bold “best in column” is useful but the captions already note non-comparable conditions; consider adding a footnote that bold is only within the subset of methods that report that column.
  3. Industrial numbers in Table 15 and §12.1 (e.g., DYNA-1 99.4%, Covariant 99%+) are correctly labeled self-reported; a single sentence in the Highlights or Abstract reminding readers of that caveat would match the care already taken in the body.
  4. Figure 1 taxonomy and Figure 2 timeline are clear; ensure that every leaf method cited in the figures appears in the reference list with a consistent year (a few 2025–2026 arXiv entries may shift between submission and publication).
  5. Minor typographical consistency: “UA V” vs “UAV”, “π*_0” vs “π*_0.6”, and occasional missing spaces around em-dashes in the PDF.

Circularity Check

0 steps flagged

No significant circularity: literature review synthesizes external corpus without self-definitional predictions or load-bearing self-citation chains.

full rationale

This is a survey of 183 external contributions (2017–2026) that organizes architectures, training recipes, action representations, bimanual coordination, and aerial systems into a taxonomy and fourteen research directions. The central transfer claim (bimanual strategies apply to UAVs) is advanced by juxtaposition of independently published systems (e.g., Flying Hand reusing ACT, dual-arm aerial harvesting as leader–follower, multi-drone joint action spaces) rather than by fitting parameters to data and then “predicting” related quantities, or by defining X in terms of Y. Formalisms (policy πθ, action chunks At, flow-matching loss LFM, bimanual joint action abi_t) are standard definitions imported from the cited literature, not circular constructions. Tables report success rates and latencies taken from original publications under varying conditions; no new quantitative prediction is forced by construction. Author self-citations, if any, are not load-bearing for the transfer thesis or the fourteen directions. As a review the paper is self-contained against the external corpus it surveys; the structural analogy between bimanual and aerial coordination is an organizational claim, not a derivation that reduces to its inputs.

Axiom & Free-Parameter Ledger

0 free parameters · 3 axioms · 1 invented entities

A review paper rests on domain conventions rather than free parameters or invented physical entities. The main load-bearing assumptions are definitional (what counts as a VLA) and analogical (bimanual coordination ≈ multi-actuator aerial control). No numerical constants are fitted; the taxonomy itself is an organizing construct, not a postulated particle or force.

axioms (3)
  • domain assumption A VLA is defined as a single foundation model that maps camera images + language (+ optional proprioception) to actions via a pre-trained VLM backbone plus an action head.
    Stated in Section 2.1 and used throughout to decide inclusion; classical planners and pure PID controllers are thereby excluded.
  • ad hoc to paper Bimanual joint-action spaces and multi-drone or aerial-manipulation action spaces are sufficiently analogous that the same chunking, flow-matching, and hierarchical recipes remain near-optimal.
    Core transfer claim of Sections 8–9 and Finding 11; not independently measured by a controlled cross-embodiment experiment inside the paper.
  • domain assumption Data diversity across embodiments matters more than raw dataset size for generalization.
    Repeatedly asserted from the surveyed literature (OpenVLA, π0, OXE) and treated as established fact for the training-recipe recommendations.
invented entities (1)
  • Seven-dimension taxonomy (architectures, training, actions, bimanual, aerial, language, cross-cutting) no independent evidence
    purpose: Organizes the 183 papers and structures the transfer argument.
    A useful organizing device introduced by the authors; not claimed to be a physical or mathematical object with independent existence.

pith-pipeline@v1.1.0-grok45 · 55800 in / 2623 out tokens · 40580 ms · 2026-07-10T22:59:38.096660+00:00 · methodology

0 comments
read the original abstract

Vision Language Action (VLA) models unify visual perception, natural-language understanding, and action generation within a single foundation model, allowing a robot to follow instructions such as fold the towel or fly to the red building directly from camera images. Because VLAs inherit world knowledge from internet-scale pre-training, they have become the dominant framework for learning-based manipulation, with bimanual coordination serving as the most demanding testbed: two arms with 7 degrees of freedom each must move in concert to fold, assemble, and reorient objects. Unmanned aerial robotics faces a structurally similar challenge: a drone must coordinate thrust, attitude, and increasingly gripper commands from visual observations under strict latency and payload constraints. This review covers 183 contributions spanning 2017-2026 and organized along seven dimensions: VLA architectures, training recipes, action representations, bimanual coordination (2022-2026), unmanned aerial vehicle (UAV) navigation and control (2017-2026), language grounding, and cross-cutting concerns including memory and world models. We show that the coordination strategies, training recipes, and action representations developed for bimanual VLAs transfer to unmanned aerial systems and identify fourteen research directions across both domains.

Figures

Figures reproduced from arXiv: 2607.06706 by Chanoh Park, Donghee Noh, Hea-Min Lee, Ho Seok Ahn, Inkyu Sa.

Figure 1
Figure 1. Figure 1: Taxonomy of VLA models for bimanual manipulation and unmanned aerial robotics. This review is organized along five major dimensions: architectural foundations (autoregressive, flow￾based, diffusion-based, hybrid), training recipes (pre-training, post-training, reinforcement learning), action representations (discrete tokenization, continuous generation), bimanual-specific concerns (coordination strategies,… view at source ↗
Figure 2
Figure 2. Figure 2: Timeline of key VLA and bimanual manipulation milestones (2022–2026). Colors indicate the architectural family: autoregressive (blue), flow-based (red), diffusion-based (green), hardware platforms (orange), and hybrid/efficient methods (purple). The field has accelerated rapidly, with the majority of VLA contributions appearing in 2024–2026. (a) Autoregressive (RT-2) Image Tokens Language Tokens VLM Backbo… view at source ↗
Figure 3
Figure 3. Figure 3: Architectural comparison of the four VLA families. (a) Autoregressive VLAs (RT-2, OpenVLA) discretize actions and generate them as language tokens. (b) Flow-based VLAs (π0) use a flow-matching head that iteratively denoises a noise sample conditioned on VLM features. (c) Diffusion VLAs (RDT-1B) use a Diffusion Transformer to denoise action chunks. (d) Hybrid VLAs (HybridVLA) combine autoregressive and flow… view at source ↗
Figure 4
Figure 4. Figure 4: The ALOHA bimanual teleoperation platform and representative tasks. A human operator controls two follower arms via leader arms for intuitive demonstration collection. ALOHA and its ACT policy established the standard platform for bimanual VLA research. Reprinted with permission from Ref. [16]. Copyright 2023, Zhao et al [PITH_FULL_IMAGE:figures/full_fig_p025_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Real-world deployment of π0.5 in homes. A hierarchical VLA decomposes high-level instructions into subgoals, with high success rates on household tasks such as table clearing and laundry folding. Reprinted with permission from Ref. [3]. Copyright 2025, Black et al. https://doi.org/10.3390/drones10060412 [PITH_FULL_IMAGE:figures/full_fig_p029_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Timeline of unmanned aerial robotics milestones for learning-based drone control (2017– 2026). Colors indicate the research area: RL-based control (blue), vision–language navigation (green), aerial manipulation (red), language-guided planning (orange), and simulation platforms (purple). Early work focused on RL for agile flight and simulators; 2023–2024 saw the emergence of language￾guided navigation; 2025… view at source ↗
Figure 7
Figure 7. Figure 7: An RL-trained quadrotor recovering from an inverted throw at 5 m/s. The policy maps state to motor commands at 7 µs per step, establishing the viability of learned end-to-end drone control. Reprinted with permission from Ref. [131]. Copyright 2017, Hwangbo et al., IEEE. Subsequent learned systems pushed the performance frontier to superhuman lev￾els. The landmark result is an RL-trained autonomous racing p… view at source ↗
Figure 8
Figure 8. Figure 8: Flying Hand: a fully actuated hexarotor with a 4-DOF arm performing writing, peg-in-hole, and pick-and-place via ACT, demonstrating that action chunking transfers from manipulation to aerial systems. The numbers 1–4 along each row index successive video frames of the same task sequence (1: approach, 2: contact, 3: execution, 4: completion). Reprinted with permission from Ref. [12]. Copyright 2025, He et al… view at source ↗
Figure 9
Figure 9. Figure 9: summarizes the complete VLA training and deployment pipeline that ties together the architectural choices (Section 5), training recipes (Section 6), and deployment considerations discussed above. The interplay among these architectural, training, and deployment considerations shapes the current state of the art, which we synthesize next. VLM Pre-training Web-scale Image-Text Robot Pre-training OXE / DROID … view at source ↗
Figure 10
Figure 10. Figure 10: Approximate evolution of VLA performance on bimanual manipulation tasks (2023– 2025). Values are approximate trend values synthesized by the authors from reported results across different evaluation setups and task definitions; they illustrate general trends rather than exact comparable benchmarks. Bimanual task success rates have improved dramatically, from ∼30% with early methods such as ACT [16] to >90… view at source ↗
Figure 11
Figure 11. Figure 11: Industrial VLA-powered humanoid robot systems. (a, top left) Boston Dynamics Atlas with TRI Large Behavior Model performing warehouse manipulation. Reprinted with permission from Ref. [178]. Copyright 2025, Boston Dynamics and Toyota Research Institute. (b, top right) Unitree humanoid executing dynamic whole-body control (image courtesy of Unitree Robotics). (c, bottom left) Tesla Optimus humanoid with de… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

175 extracted references · 175 canonical work pages · 34 internal anchors

  1. [1]

    RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control

    Brohan, A.; Brown, N.; Carbajal, J.; Chebotar, Y.; Chen, X.; Choromanski, K.; Ding, T.; Driess, D.; Dubey, A.; Finn, C.; et al. RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control. InProceedings of the 7th Conference on Robot Learning (CoRL); PMLR: Atlanta, GA, USA, 6–9 November 2023; Volume 229, pp. 2165–2183

  2. [2]

    π0: A Vision-Language-Action Flow Model for General Robot Control

    Black, K.; Brown, N.; Driess, D.; Esmail, A.; Equi, M.; Finn, C.; Fusai, N.; Groom, L.; Hausman, K.; Ichter, B.; et al. π0: A Vision-Language-Action Flow Model for General Robot Control. InProceedings of Robotics: Science and Systems (RSS), Los Angeles, CA, USA, 21–25 June 2025

  3. [3]

    π0.5: A Vision-Language-Action Model with Open-World Generalization

    Black, K.; Brown, N.; Darpinian, J.; Dhabalia, K.; Driess, D.; Esmail, A.; Equi, M.; Finn, C.; Fusai, N.; Galliker, M.Y.; et al. π0.5: A Vision-Language-Action Model with Open-World Generalization. InProceedings of the 9th Conference on Robot Learning (CoRL); PMLR: Seoul, Republic of Korea, 27–30 September 2025; Volume 305, pp. 17–40

  4. [4]

    $\pi^{*}_{0.6}$: a VLA That Learns From Experience

    Amin, A.; Aniceto, R.; Balakrishna, A.; Black, K.; Conley, K.; Connors, G.; Darpinian, J.; Dhabalia, K.; DiCarlo, J.; Driess, D.; et al. π∗ 0.6: A VLA That Learns From Experience.arXiv2025, arXiv:2511.14759

  5. [6]

    Octo: An Open-Source Generalist Robot Policy

    Octo Model Team; Ghosh, D.; Walke, H.; Pertsch, K.; Black, K.; Mees, O.; Dasari, S.; Hejna, J.; Kreiman, T.; Xu, C.; et al. Octo: An Open-Source Generalist Robot Policy. InProceedings of Robotics: Science and Systems (RSS), Delft, The Netherlands, 15–19 July 2024. https://doi.org/10.15607/RSS.2024.XX.090

  6. [7]

    TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulation.IEEE Robot

    Wen, J.; Zhu, Y.; Li, J.; Zhu, M.; Wu, K.; Xu, Z.; Liu, N.; Cheng, R.; Shen, C.; Peng, Y.; et al. TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulation.IEEE Robot. Autom. Lett. (RA-L)2025,10, 3988–3995

  7. [8]

    FAST: Efficient Action Tokenization for Vision-Language-Action Models

    Pertsch, K.; Stachowicz, K.; Ichter, B.; Driess, D.; Nair, S.; Vuong, Q.; Mees, O.; Finn, C.; Levine, S. FAST: Efficient Action Tokenization for Vision-Language-Action Models. InProceedings of Robotics: Science and Systems (RSS), Los Angeles, CA, USA, 21–25 June 2025. https://doi.org/10.15607/RSS.2025.XXI.012

  8. [9]

    CognitiveDrone: A VLA Model and Evaluation Benchmark for Real-Time Cognitive Task Solving and Reasoning in UAVs

    Lykov, A.; Serpiva, V .; Khan, M.H.; Sautenkov, O.; Myshlyaev, A.; Tadevosyan, G.; Yaqoot, Y.; Tsetserukou, D. Cognitive- Drone: A VLA Model and Evaluation Benchmark for Real-Time Cognitive Task Solving and Reasoning in UAVs.arXiv2025, arXiv:2503.01378

  9. [10]

    DroneVLA: VLA Based Aerial Manipulation

    Mehboob, F.; James, M.; Habel, A.; Sam, J.; Altamirano Cabrera, M.; Tsetserukou, D. DroneVLA: VLA Based Aerial Manipulation. arXiv2026, arXiv:2601.13809

  10. [11]

    AIR-VLA: Vision-Language-Action Systems for Aerial Manipulation.arXiv2026, arXiv:2601.21602

    Sun, J.; Tian, B.; Zhang, Q.; Li, C.; Song, Z.; Cui, Z.; Lv, Y.; Tian, Y. AIR-VLA: Vision-Language-Action Systems for Aerial Manipulation.arXiv2026, arXiv:2601.21602

  11. [12]

    Flying Hand: End-Effector-Centric Framework for Versatile Aerial Manipulation Teleoperation and Policy Learning

    He, G.; Guo, X.; Tang, L.; Zhang, Y.; Mousaei, M.; Xu, J.; Geng, J.; Scherer, S.; Shi, G. Flying Hand: End-Effector-Centric Framework for Versatile Aerial Manipulation Teleoperation and Policy Learning.arXiv2025, arXiv:2504.10334

  12. [13]

    Foundation Models in Robotics: Applications, Challenges, and the Future.Int

    Firoozi, R.; Tucker, J.; Tian, S.; Majumdar, A.; Sun, J.; Liu, W.; Zhu, Y.; Song, S.; Kapoor, A.; Hausman, K.; et al. Foundation Models in Robotics: Applications, Challenges, and the Future.Int. J. Robot. Res.2025,44, 701–739

  13. [14]

    Diffusion Models for Robotic Manipulation: A Survey.Front

    Wolf, R.; Shi, Y.; Liu, S.; Rayyes, R. Diffusion Models for Robotic Manipulation: A Survey.Front. Robot. AI2025,12, 1606247

  14. [15]

    A Systematic Review on Cooperative Dual-Arm Manipulators: Modeling, Planning, Control, and Vision Strategies.Int

    Abbas, M.; Narayan, J.; Dwivedy, S.K. A Systematic Review on Cooperative Dual-Arm Manipulators: Modeling, Planning, Control, and Vision Strategies.Int. J. Intell. Robot. Appl.2023,7, 683–707

  15. [16]

    Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware

    Zhao, T.Z.; Kumar, V .; Levine, S.; Finn, C. Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware. InProceedings of Robotics: Science and Systems (RSS), Daegu, Republic of Korea, 10–14 July 2023. https://doi.org/10.15607/RSS.2023.XIX.016

  16. [17]

    Flow Matching for Generative Modeling

    Lipman, Y.; Chen, R.T.Q.; Ben-Hamu, H.; Nickel, M.; Le, M. Flow Matching for Generative Modeling. InProceedings of the International Conference on Learning Representations (ICLR), Kigali, Rwanda, 1–5 May 2023

  17. [18]

    Multirotor Aerial Vehicles: Modeling, Estimation, and Control of Quadrotor.IEEE Robot

    Mahony, R.; Kumar, V .; Corke, P . Multirotor Aerial Vehicles: Modeling, Estimation, and Control of Quadrotor.IEEE Robot. Autom. Mag.2012,19, 20–32

  18. [19]

    Attention Is All You Need

    Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, Ł.; Polosukhin, I. Attention Is All You Need. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), Long Beach, CA, USA, 4–9 December 2017

  19. [20]

    An Image is Worth 16 × 16 Words: Transformers for Image Recognition at Scale

    Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; et al. An Image is Worth 16 × 16 Words: Transformers for Image Recognition at Scale. InProceedings of the International Conference on Learning Representations (ICLR), Virtual, 3–7 May 2021

  20. [21]

    Learning Transferable Visual Models From Natural Language Supervision

    Radford, A.; Kim, J.W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P .; Clark, J.; et al. Learning Transferable Visual Models From Natural Language Supervision. InProceedings of the 38th International Conference on Machine Learning (ICML); PMLR: Virtual, 18–24 July 2021; Volume 139

  21. [22]

    Language Models Are Few-Shot Learners

    Brown, T.B.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J.; Dhariwal, P .; Neelakantan, A.; Shyam, P .; Sastry, G.; Askell, A.; et al. Language Models Are Few-Shot Learners. InProceedings of the Advances in Neural Information Processing Systems (NeurIPS), Virtual, 6–12 December 2020

  22. [23]

    Training Language Models to Follow Instructions with Human Feedback

    Ouyang, L.; Wu, J.; Jiang, X.; Almeida, D.; Wainwright, C.L.; Mishkin, P .; Zhang, C.; Agarwal, S.; Slama, K.; Ray, A.; et al. Training Language Models to Follow Instructions with Human Feedback. InProceedings of the Advances in Neural Information Processing Systems (NeurIPS), New Orleans, LA, USA, 28 November–9 December 2022

  23. [24]

    PaLM-E: An Embodied Multimodal Language Model

    Driess, D.; Xia, F.; Sajjadi, M.S.M.; Lynch, C.; Chowdhery, A.; Ichter, B.; Wahid, A.; Tompson, J.; Vuong, Q.; Yu, T.; et al. PaLM-E: An Embodied Multimodal Language Model. InProceedings of the 40th International Conference on Machine Learning (ICML); PMLR: Honolulu, HI, USA, 23–29 July 2023; Volume 202

  24. [25]

    PaliGemma: A versatile 3B VLM for transfer

    Beyer, L.; Steiner, A.; Pinto, A.S.; Kolesnikov, A.; Wang, X.; Salz, D.; Neumann, M.; Alabdulmohsin, I.; Tschannen, M.; Bugliarello, E.; et al. PaliGemma: A Versatile 3B VLM for Transfer.arXiv2024, arXiv:2407.07726

  25. [26]

    Gemma: Open Models Based on Gemini Research and Technology

    Gemma Team. Gemma: Open Models Based on Gemini Research and Technology.arXiv2024, arXiv:2403.08295

  26. [27]

    LLaMA: Open and Efficient Foundation Language Models

    Touvron, H.; Lavril, T.; Izacard, G.; Martinet, X.; Lachaux, M.A.; Lacroix, T.; Rozière, B.; Goyal, N.; Hambro, E.; Azhar, F.; et al. LLaMA: Open and Efficient Foundation Language Models.arXiv2023, arXiv:2302.13971

  27. [28]

    Visual Instruction Tuning

    Liu, H.; Li, C.; Wu, Q.; Lee, Y.J. Visual Instruction Tuning. InProceedings of the Advances in Neural Information Processing Systems (NeurIPS), New Orleans, LA, USA, 10–16 December 2023

  28. [30]

    ALVINN: An Autonomous Land Vehicle in a Neural Network

    Pomerleau, D.A. ALVINN: An Autonomous Land Vehicle in a Neural Network. InProceedings of the Advances in Neural Information Processing Systems (NeurIPS), Denver, CO, USA, 27–30 November 1989

  29. [31]

    A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning

    Ross, S.; Gordon, G.J.; Bagnell, D. A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning. In Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS), Fort Lauderdale, FL, USA, 11–13 April 2011

  30. [32]

    Language-Conditioned Imitation Learning for Robot Manipulation Tasks

    Stepputtis, S.; Campbell, J.; Phielipp, M.; Lee, S.; Baral, C.; Ben Amor, H. Language-Conditioned Imitation Learning for Robot Manipulation Tasks. InProceedings of the Advances in Neural Information Processing Systems (NeurIPS), Virtual, 6–12 December 2020

  31. [33]

    Auto-Encoding Variational Bayes

    Kingma, D.P .; Welling, M. Auto-Encoding Variational Bayes. InProceedings of the International Conference on Learning Representations (ICLR), Banff, AB, Canada, 14–16 April 2014

  32. [34]

    Generative Adversarial Nets

    Goodfellow, I.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; Bengio, Y. Generative Adversarial Nets. InProceedings of the Advances in Neural Information Processing Systems (NeurIPS), Montreal, QC, Canada, 8–13 December 2014

  33. [35]

    Denoising Diffusion Probabilistic Models

    Ho, J.; Jain, A.; Abbeel, P . Denoising Diffusion Probabilistic Models. InProceedings of the Advances in Neural Information Processing Systems (NeurIPS), Virtual, 6–12 December 2020

  34. [36]

    Score-Based Generative Modeling through Stochastic Differential Equations

    Song, Y.; Sohl-Dickstein, J.; Kingma, D.P .; Kumar, A.; Ermon, S.; Poole, B. Score-Based Generative Modeling through Stochastic Differential Equations. InProceedings of the International Conference on Learning Representations (ICLR), Virtual, 3–7 May 2021

  35. [37]

    High-Resolution Image Synthesis with Latent Diffusion Models

    Rombach, R.; Blattmann, A.; Lorenz, D.; Esser, P .; Ommer, B. High-Resolution Image Synthesis with Latent Diffusion Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 19–24 June 2022

  36. [38]

    Decision Transformer: Reinforcement Learning via Sequence Modeling

    Chen, L.; Lu, K.; Rajeswaran, A.; Lee, K.; Grover, A.; Laskin, M.; Abbeel, P .; Srinivas, A.; Mordatch, I. Decision Transformer: Reinforcement Learning via Sequence Modeling. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), 2021

  37. [39]

    A Generalist Agent.Trans

    Reed, S.; Zolna, K.; Parisotto, E.; Colmenarejo, S.G.; Novikov, A.; Barth-Maron, G.; Giménez, M.; Sulsky, Y.; Kay, J.; Springenberg, J.T.; et al. A Generalist Agent.Trans. Mach. Learn. Res. (TMLR)2022. Available online: https://openreview.net/forum?id=1ikK0 kHjvj (accessed on 21 May 2026)

  38. [40]

    Diffusion Policy: Visuomotor Policy Learning via Action Diffusion.Int

    Chi, C.; Xu, Z.; Feng, S.; Cousineau, E.; Du, Y.; Burchfiel, B.; Tedrake, R.; Song, S. Diffusion Policy: Visuomotor Policy Learning via Action Diffusion.Int. J. Robot. Res. (IJRR)2024,44, 1684–1704. https://doi.org/10.1177/02783649241273668

  39. [41]

    Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow

    Liu, X.; Gong, C.; Liu, Q. Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow. InProceedings of the International Conference on Learning Representations (ICLR), Kigali, Rwanda, 1–5 May 2023

  40. [42]

    Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation

    Fu, Z.; Zhao, T.Z.; Finn, C. Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation. InProceedings of the 8th Conference on Robot Learning (CoRL); PMLR: Munich, Germany, 6–9 November 2024; Volume 270, pp. 4066–4083

  41. [43]

    Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots

    Chi, C.; Xu, Z.; Pan, C.; Cousineau, E.; Burchfiel, B.; Feng, S.; Tedrake, R.; Song, S. Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots. InProceedings of Robotics: Science and Systems (RSS), Delft, The Netherlands, 15–19 July 2024. https://doi.org/10.15607/RSS.2024.XX.045

  42. [44]

    RDT-1B: A Diffusion Foundation Model for Bimanual Manipulation

    Liu, S.; Wu, L.; Li, B.; Tan, H.; Chen, H.; Wang, Z.; Xu, K.; Su, H.; Zhu, J. RDT-1B: A Diffusion Foundation Model for Bimanual Manipulation. InProceedings of the International Conference on Learning Representations (ICLR), Singapore, 24–28 April 2025

  43. [45]

    AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles

    Shah, S.; Dey, D.; Lovett, C.; Kapoor, A. AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles. InField and Service Robotics (FSR), Zurich, Switzerland, 12–15 September 2017; Springer Proceedings in Advanced Robotics, Volume 5, pp. 621–635, published 2018

  44. [46]

    Flightmare: A Flexible Quadrotor Simulator

    Song, Y.; Naji, S.; Kaufmann, E.; Loquercio, A.; Scaramuzza, D. Flightmare: A Flexible Quadrotor Simulator. InProceedings of the 4th Conference on Robot Learning (CoRL); PMLR: Virtual, 16–18 November 2020; Volume 155, pp. 1147–1157

  45. [47]

    LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning

    Liu, B.; Zhu, Y.; Gao, C.; Feng, Y.; Liu, Q.; Zhu, Y.; Stone, P . LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning. InProceedings of the Advances in Neural Information Processing Systems (NeurIPS), New Orleans, LA, USA, 10–16 December 2023

  46. [48]

    Evaluating Real-World Robot Manipulation Policies in Simulation

    Li, X.; Hsu, K.; Gu, J.; Pertsch, K.; Mees, O.; Walke, H.R.; Fu, C.; Lunawat, I.; Sieh, I.; Kirmani, S.; et al. Evaluating Real-World Robot Manipulation Policies in Simulation. InProceedings of the 8th Conference on Robot Learning (CoRL); PMLR: Munich, Germany, 6–9 November 2024; Volume 270, pp. 3705–3728

  47. [49]

    Open X-Embodiment: Robotic Learning Datasets and RT-X Models

    Open X-Embodiment Collaboration. Open X-Embodiment: Robotic Learning Datasets and RT-X Models. InProceedings of the IEEE International Conference on Robotics and Automation (ICRA), Yokohama, Japan, 13–17 May 2024; pp. 6892–6903

  48. [50]

    DROID: A Large-Scale In-the-Wild Robot Manipulation Dataset

    Khazatsky, A.; Pertsch, K.; Nair, S.; Balakrishna, A.; Dasari, S.; Karamcheti, S.; Nasiriany, S.; Srirama, M.K.; Chen, L.Y.; Ellis, K.; et al. DROID: A Large-Scale In-the-Wild Robot Manipulation Dataset. InProceedings of Robotics: Science and Systems (RSS), Delft, The Netherlands, 15–19 July 2024. https://doi.org/10.15607/RSS.2024.XX.120

  49. [52]

    Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasets

    Ebert, F.; Yang, Y.; Schmeckpeper, K.; Bucher, B.; Georgakis, G.; Daniilidis, K.; Finn, C.; Levine, S. Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasets. InProceedings of Robotics: Science and Systems (RSS), New York, NY, USA, 27 June–1 July 2022

  50. [53]

    GigaBrain-0.5M∗: A VLA That Learns From World Model-Based Reinforcement Learning.arXiv2026, arXiv:2602.12099

    Wang, B.; Li, B.; Ni, C.; Huang, G.; Zhao, G.; Li, H.; Li, J.; Lv, J.; Liu, J.; Feng, L.; et al. GigaBrain-0.5M∗: A VLA That Learns From World Model-Based Reinforcement Learning.arXiv2026, arXiv:2602.12099

  51. [54]

    RLBench: The Robot Learning Benchmark and Learning Environment.IEEE Robot

    James, S.; Ma, Z.; Arrojo, D.R.; Davison, A.J. RLBench: The Robot Learning Benchmark and Learning Environment.IEEE Robot. Autom. Lett. (RA-L)2020,5, 3019–3026

  52. [55]

    Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning

    Yu, T.; Quillen, D.; He, Z.; Julian, R.; Hausman, K.; Finn, C.; Levine, S. Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning. In Proceedings of the Conference on Robot Learning (CoRL), 2020

  53. [56]

    robosuite: A Modular Simulation Framework and Benchmark for Robot Learning

    Zhu, Y.; Wong, J.; Mandlekar, A.; Martín-Martín, R.; Joshi, A.; Lin, K.; Maddukuri, A.; Nasiriany, S.; Zhu, Y. robosuite: A Modular Simulation Framework and Benchmark for Robot Learning.arXiv2020, arXiv:2009.12293

  54. [57]

    ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skills

    Gu, J.; Xiang, F.; Li, X.; Ling, Z.; Liu, X.; Mu, T.; Tang, Y.; Tao, S.; Wei, X.; Yao, Y.; et al. ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skills. InProceedings of the International Conference on Learning Representations (ICLR), Kigali, Rwanda, 1–5 May 2023

  55. [58]

    BEHAVIOR-1K: A Human-Centered, Embodied AI Benchmark with 1,000 Everyday Activities and Realistic Simulation

    Li, C.; Zhang, R.; Wong, J.; Gokmen, C.; Srivastava, S.; Martín-Martín, R.; Wang, C.; Levine, G.; Ai, W.; Martinez, B.; et al. BEHAVIOR-1K: A Human-Centered, Embodied AI Benchmark with 1,000 Everyday Activities and Realistic Simulation.arXiv 2024, arXiv:2403.09227

  56. [59]

    RT-1: Robotics Transformer for Real-World Control at Scale

    Brohan, A.; Brown, N.; Carbajal, J.; Chebotar, Y.; Dabis, J.; Finn, C.; Gopalakrishnan, K.; Hausman, K.; Herzog, A.; Hsu, J.; et al. RT-1: Robotics Transformer for Real-World Control at Scale. InProceedings of Robotics: Science and Systems (RSS), Daegu, Republic of Korea, 10–14 July 2023. https://doi.org/10.15607/RSS.2023.XIX.025

  57. [60]

    Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success

    Kim, M.J.; Finn, C.; Liang, P . Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success. InProceedings of Robotics: Science and Systems (RSS), Los Angeles, CA, USA, 21–25 June 2025

  58. [61]

    Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulation

    Wu, H.; Jing, Y.; Cheang, C.; Chen, G.; Xu, J.; Li, X.; Liu, M.; Li, H.; Kong, T. Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulation. InProceedings of the International Conference on Learning Representations (ICLR), Vienna, Austria, 7–11 May 2024

  59. [62]

    HAMSTER: Hierarchical Action Models For Open-World Robot Manipulation

    Li, Y.; Deng, Y.; Zhang, J.; Jang, J.; Memmel, M.; Yu, R.; Garrett, C.R.; Ramos, F.; Fox, D.; Li, A.; et al. HAMSTER: Hierarchical Action Models for Open-World Robot Manipulation.arXiv2025, arXiv:2502.05485

  60. [63]

    SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Model

    Qu, D.; Song, H.; Chen, Q.; Yao, Y.; Ye, X.; Ding, Y.; Wang, Z.; Gu, J.; Zhao, B.; Wang, D.; et al. SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Model. InProceedings of Robotics: Science and Systems (RSS), Los Angeles, CA, USA, 21–25 June 2025

  61. [64]

    BAKU: An Efficient Transformer for Multi-Task Policy Learning

    Haldar, S.; Peng, Z.; Pinto, L. BAKU: An Efficient Transformer for Multi-Task Policy Learning. InProceedings of the Advances in Neural Information Processing Systems (NeurIPS), Vancouver, BC, Canada, 9–15 December 2024

  62. [65]

    Keypoint Action Tokens Enable In-Context Imitation Learning in Robotics

    Di Palo, N.; Johns, E. Keypoint Action Tokens Enable In-Context Imitation Learning in Robotics. InProceedings of Robotics: Science and Systems (RSS), Delft, The Netherlands, 15–19 July 2024

  63. [66]

    SimpleVLA-RL: Scaling VLA Training via Reinforcement Learning

    Li, H.; Zuo, Y.; Yu, J.; Zhang, Y.; Yang, Z.; Zhang, K.; Zhu, X.; Zhang, Y.; Chen, T.; Cui, G.; et al. SimpleVLA-RL: Scaling VLA Training via Reinforcement Learning. InProceedings of the International Conference on Learning Representations (ICLR), Available online: https://iclr.cc/Conferences/2026 (accessed on 21 May 2026)

  64. [67]

    CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulation

    Li, Q.; Liang, Y.; Wang, Z.; Luo, L.; Chen, X.; Liao, M.; Wei, F.; Deng, Y.; Xu, S.; Zhang, Y.; et al. CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulation.arXiv2024, arXiv:2411.19650

  65. [68]

    Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation

    Shridhar, M.; Manuelli, L.; Fox, D. Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation. InProceedings of the Conference on Robot Learning (CoRL), Atlanta, GA, USA, 6–9 November 2023

  66. [69]

    RVT: Robotic View Transformer for 3D Object Manipulation

    Goyal, A.; Xu, J.; Guo, Y.; Blukis, V .; Chao, Y.W.; Fox, D. RVT: Robotic View Transformer for 3D Object Manipulation. In Proceedings of the Conference on Robot Learning (CoRL), Atlanta, GA, USA, 6–9 November 2023

  67. [70]

    3D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representations

    Ze, Y.; Zhang, G.; Zhang, K.; Hu, C.; Wang, M.; Xu, H. 3D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representations. InProceedings of Robotics: Science and Systems (RSS), Delft, The Netherlands, 15–19 July 2024

  68. [71]

    Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model

    Zhou, C.; Yu, L.; Babu, A.; Tirumala, K.; Yasunaga, M.; Shamis, L.; Kahn, J.; Ma, X.; Zettlemoyer, L.; Levy, O. Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model. InProceedings of the 13th International Conference on Learning Representations (ICLR), Singapore, 24–28 April 2025

  69. [72]

    HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Model

    Liu, J.; Chen, H.; An, P .; Liu, Z.; Zhang, R.; Gu, C.; Li, X.; Guo, Z.; Chen, S.; Liu, M.; et al. HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Model.arXiv2025, arXiv:2503.10631

  70. [73]

    MiniVLA: A Better VLA with a Smaller Footprint

    Belkhale, S.; Sadigh, D. MiniVLA: A Better VLA with a Smaller Footprint. 2024. Stanford AI Lab Blog. Available online: https://ai.stanford.edu/blog/minivla/ (accessed on 21 May 2026)

  71. [75]

    MemoryVLA: Perceptual-Cognitive Memory in Vision-Language-Action Models for Robotic Manipulation

    Shi, H.; Xie, B.; Liu, Y.; Sun, L.; Liu, F.; Wang, T.; Zhou, E.; Fan, H.; Zhang, X.; Huang, G. MemoryVLA: Perceptual-Cognitive Memory in Vision-Language-Action Models for Robotic Manipulation.arXiv2025, arXiv:2508.19236

  72. [76]

    ContextVLA: Vision-Language-Action Model with Amortized Multi-Frame Context.arXiv2025, arXiv:2510.04246

    Jang, H.; Yu, S.; Kwon, H.; Jeon, H.; Seo, Y.; Shin, J. ContextVLA: Vision-Language-Action Model with Amortized Multi-Frame Context.arXiv2025, arXiv:2510.04246

  73. [77]

    GigaBrain-0: A World Model-Powered Vision-Language-Action Model.arXiv2025, arXiv:2510.19430

    Ye, A.; Wang, B.; Ni, C.; Huang, G.; Zhao, G.; Li, H.; Li, J.; Zhu, J.; Feng, L.; Li, P .; et al. GigaBrain-0: A World Model-Powered Vision-Language-Action Model.arXiv2025, arXiv:2510.19430

  74. [78]

    Causal Video Models Are Data-Efficient Robot Policy Learners

    Rhoda AI. Causal Video Models Are Data-Efficient Robot Policy Learners. 2026 Available online: https://www.rhoda.ai/ research/direct-video-action (accessed on 21 May 2026)

  75. [79]

    WorldVLA: Towards Autoregressive Action World Model

    Cen, J.; Yu, C.; Yuan, H.; Jiang, Y.; Huang, S.; Guo, J.; Li, X.; Song, Y.; Luo, H.; Wang, F.; et al. WorldVLA: Towards Autoregressive Action World Model.arXiv2025, arXiv:2506.21539

  76. [80]

    GigaWorld-0: World Models as Data Engine to Empower Embodied AI.arXiv2025, arXiv:2511.19861

    Ye, A.; Wang, B.; Ni, C.; Huang, G.; Zhao, G.; Li, H.; Zhu, J.; Li, K.; Xu, M.; Deng, Q.; et al. GigaWorld-0: World Models as Data Engine to Empower Embodied AI.arXiv2025, arXiv:2511.19861

  77. [81]

    LoRA: Low-Rank Adaptation of Large Language Models

    Hu, E.J.; Shen, Y.; Wallis, P .; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; Chen, W. LoRA: Low-Rank Adaptation of Large Language Models. In Proceedings of the International Conference on Learning Representations (ICLR), 2022

  78. [82]

    Direct Preference Optimization: Your Language Model is Secretly a Reward Model

    Rafailov, R.; Sharma, A.; Mitchell, E.; Ermon, S.; Manning, C.D.; Finn, C. Direct Preference Optimization: Your Language Model is Secretly a Reward Model. InProceedings of the Advances in Neural Information Processing Systems (NeurIPS), New Orleans, LA, USA, 10–16 December 2023

  79. [83]

    Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Better

    Driess, D.; Springenberg, J.T.; Ichter, B.; Yu, L.; Li-Bell, A.; Pertsch, K.; Ren, A.Z.; Walke, H.; Vuong, Q.; Shi, L.X.; et al. Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Better.arXiv2025, arXiv:2505.23705

  80. [84]

    Align-Then-stEer: Adapting the Vision-Language Action Models through Unified Latent Guidance

    Zhang, Y.; Wang, C.; Lu, O.; Zhao, Y.; Ge, Y.; Sun, Z.; Li, X.; Zhang, C.; Bai, C.; Li, X. Align-Then-stEer: Adapting the Vision-Language Action Models through Unified Latent Guidance.arXiv2025, arXiv:2509.02055

Showing first 80 references.