Pith. sign in

REVIEW 4 major objections 4 minor 4 cited by

From Intention to Execution: Probing the Generalization Boundaries of Vision-Language-Action Models

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Current vision-language-action models inherit strong semantic grounding from their VLM backbones, but under distribution shift that understanding fails to become reliable motor execution—an 'intention-action gap' exposed by a new 50-task…

desk verdict Useful benchmark with a plausible but under-validated headline metric; deserves peer review with requests for sensitivity analysis and error bars. read the letter →

arxiv 2506.09930 v1 pith:57BKKR5L submitted 2025-06-11 cs.RO cs.CV

Irving Fang , Juexiao Zhang , Shengbang Tong , Chen Feng This is my paper · ORCID
classification cs.ROcs.CV
keywords vision-language-actionmodelsrobotgeneralizationintention-actiongapbenchmarkout-of-distributionimitationlearningmanipulationsimulation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to measure how much the pretrained vision-language knowledge inside Vision-Language-Action (VLA) models actually survives fine-tuning on robot data and contributes to out-of-distribution generalization. To do this, the authors build INT-ACT, a 50-task simulation suite that systematically varies objects, language instructions, and visual distractors relative to the BridgeV2 fine-tuning dataset, and they evaluate π0, SpatialVLA, Magma, and Octo on it. The central finding is an 'intention-action gap': the models almost always look at and move toward the correct object, with 80-100% intention correctness, but grasp and task success rates drop sharply, so semantic understanding does not translate into precise control. A second finding is that fine-tuning on action data erodes the VLM's generalist language and reasoning abilities, with models degrading under simple paraphrases, negation, and appearance descriptions. If these results hold, closing the perception-to-action gap, rather than improving semantic grounding, becomes the central design problem for generalist robot policies.

What carries the argument

The load-bearing object is INT-ACT itself, a 50-task simulation suite built on SimplerEnv that sweeps three generalization axes: object diversity, with out-of-distribution sources, targets, and relations; language complexity, with action paraphrases, negation, appearance descriptions, and commonsense; and vision-language thinking, with distractors and compounding shifts. Its distinctive mechanism is the metric decomposition: alongside the native Grasp Success and Task Success rates, it introduces Intention Correct Rate, defined as whether the gripper ever moves within a 5 cm radius of the correct source object. Separating intention from execution is what makes the central claim visible, because without that decomposition, near-perfect targeting and near-zero completion would be conflated into a single failure number.

What would settle it

Audit whether the 'OOD' objects and language patterns actually appear in each model's pretraining corpus; for open models like Octo this is directly checkable, and for π0 a surrogate is probing the frozen PaliGemma backbone's recognition of the probe objects. If the objects are already familiar to the backbone, the measured drops reflect fine-tuning distribution mismatch rather than a true generalization boundary. A second check is to fine-tune a VLA on the OOD objects themselves: if task success then matches intention correctness, the gap is a training-data artifact, not a structural property of VLA architectures.

Watch

Extended reading notes

Core claim

The central claim is that current state-of-the-art VLA architectures decouple semantic grounding from low-level control: the VLM backbone supplies what the paper calls 'good intentions,' a generalizable notion of what to do, but this reliably fails to become reliable execution. Across all generalization categories, VLM-initialized VLAs achieve near-perfect Intention Correctness, 80-100%, while Task Success Rates collapse, often below 30%, and the gap persists even when compared against non-VLM baselines like Octo, indicating that the bottleneck is not understanding but its translation into robust low-level action. The paper further claims that this fragility is compounded under multimodal ambiguity: when commonsense paraphrases and visual distractors are combined, such as 'rabbit's favorite vegetable' with a toy bunny in the scene or 'orange juice' with an orange nearby, policies systematically grasp the wrong object. Finally, end-to-end VLA training erodes linguistic capability: simple language variations cause consistent performance drops, and even adding paraphrased instructions during fine-tuning restores intention but not execution.

Load-bearing premise

The entire generalization-boundary interpretation depends on the probe tasks being genuinely out-of-distribution, but OOD status is defined only against the BridgeV2 fine-tuning dataset, because the pretraining data of models like π0 is proprietary and cannot be audited.

Editorial extensions

If this is right

  • Improving VLM backbones alone will not yield more generalist robot policies, because the bottleneck sits in the action head or the perception-to-action interface rather than in semantic understanding.
  • VLA fine-tuning actively damages the VLM's pretrained reasoning, so linguistic robustness must be treated as an explicit training objective rather than assumed to carry over.
  • Benchmarking VLAs should report intention and execution separately, since single success-rate numbers obscure where generalization actually fails.
  • Evaluation protocols that only use in-distribution tasks, like the original four SimplerEnv tasks, overestimate deployment readiness because the gap widens precisely under out-of-distribution observations.
  • The suite provides a standardized, simulation-only testbed that makes cross-institution comparison of VLA generalization feasible without real-robot setups.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension of the paper's claim is that freezing the VLM backbone or routing actions through a separate, grounded control module would shrink the intention-action gap; the paper hints at modular designs but does not test them.
  • The out-of-distribution labels in INT-ACT are defined only against BridgeV2, so the measured 'generalization boundary' may partly reflect what the models already saw during web-scale pretraining; auditing pretraining corpora for the probe objects and phrases would sharpen the interpretation.
  • The intention metric is a proxy for semantics, but it may also capture grasp-priming behavior; sweeping the 5 cm threshold could separate 'knows the right object' from 'commits to the right object.'
  • If the gap is architectural rather than data-driven, scaling up demonstration data for OOD objects will not close it; a cheap check is fine-tuning on those objects and measuring whether intention and success converge.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces INT-ACT, a simulation-based probing benchmark for Vision-Language-Action (VLA) models, comprising 50 tasks across object diversity, language complexity, and vision-language thinking categories. The authors evaluate six model variants (π0-finetune, π0-scratch, Magma, SpatialVLA, Octo Small/Base) and report three metrics: Grasp Success Rate, a new Intention Correct Rate, and Task Success Rate. The central empirical claim is an 'intention-action gap': VLA policies exhibit substantially higher intention correctness than task success, especially under out-of-distribution (OOD) conditions, indicating that VLM pretraining provides semantic grounding that does not reliably transfer to precise motor execution. A second claim is that fine-tuning erodes the linguistic and multimodal reasoning capabilities of the underlying VLM. The authors release their code and task suite.

Significance. If the main claims hold, the paper makes a useful contribution by providing a reproducible, open benchmark for VLA generalization and by decomposing policy failures into perceptual/semantic ('intention') and motor ('execution') components. The observation that intention and execution can decouple across multiple architectures is a potentially important design signal for the robotics community. However, the headline 'intention-action gap' currently rests on a single, unvalidated metric (Intention Correct Rate) without a chance-level baseline or threshold sensitivity analysis, and the OOD labels are defined relative to the BridgeV2 fine-tuning set rather than each model's pretraining data. The paper does ship its code and task definitions publicly, which strengthens reproducibility, and the external PaliGemma VQA probe is a good design choice for checking whether the base VLM understands the language variations.

major comments (4)
  1. [§4.1, Appendix A] The Intention Correct Rate metric is load-bearing for the paper's central claim, but it is not validated as a measure of 'intention.' The metric counts an episode as intention-correct if the gripper ever comes within a 5 cm radius of the correct source object at any frame; this threshold is described only as 'empirically determined' (Appendix A), with no sensitivity analysis and no chance-level baseline. Because 'at any frame' is extremely lenient, a policy that moves toward any salient object or wanders near the source can be counted as having the correct intention, especially in scenes where the source is the only manipulable object. I request the authors to report (a) the intention rate of a random or language-ignoring policy on the same tasks, (b) how the reported intention rates change with the radius threshold (e.g., 2 cm, 5 cm, 10 cm), and (c) the distribution of first-contact times relative to episode length. Without these, the '80–100% intention' values in Table 1 cannot be interpreted as evidence of semantic grounding.
  2. [§3.1, §4.2] The OOD status is defined solely relative to the BridgeV2 fine-tuning dataset, not relative to each model's pretraining data. The paper acknowledges in §3.1 that auditing π0's proprietary pretraining corpus is impossible, but this is not merely a minor caveat: it directly affects the interpretation of 'generalization boundaries.' If OOD objects (e.g., coke can, nut, wheel) appeared in web-scale pretraining, then the measured performance drops reflect adaptation to a new fine-tuning distribution rather than failure to generalize to truly unseen concepts. The authors should either soften the claim to 'OOD relative to the fine-tuning dataset' throughout the abstract and conclusions, or provide control tasks using objects that are verifiably absent from the pretraining corpora of all evaluated models. As written, the headline statement about 'out-of-distribution observations' overstates what the experiment can establish.
  3. [§4.2, Tables 1–2, Figure 4] All reported metrics are averages over 24 episodes and 3 seeds, but no error bars, confidence intervals, or significance tests are provided anywhere in the paper. This is a serious issue for a benchmark paper that draws strong cross-model and cross-category conclusions. For example, Table 2 reports deltas such as SpatialVLA's success rate changing by +0.0 under language negation, and many per-task differences are below 5 percentage points; without variance estimates these differences are impossible to distinguish from noise. The authors have the raw per-seed data in hand, so reporting standard errors or bootstrap intervals across the 3 seeds is straightforward. At minimum, the main gap between intention and success should be accompanied by a paired test (e.g., Wilcoxon signed-rank across tasks) to confirm it is not an artifact of a few outlier tasks.
  4. [§4.2, Figure 4] The paper's interpretation that grasping varies 'more by object identity than by OOD status' is confounded with physical graspability. For instance, Figure 4b shows the coke can achieves higher grasping success than the carrot even for models where the carrot is in-distribution and the coke can is OOD; the authors note this but do not control for object-level difficulty. Consequently, the observed intention-action gap could partly reflect simple manipulation difficulty (small or awkward objects) rather than a genuine decoupling of semantic intention from motor control. I recommend adding a per-object difficulty calibration, such as measuring each model's grasp success on the same object when it is in-distribution, or reporting a ratio of OOD-to-in-distribution grasp success within matched object pairs. The Wrong Object Attempt Rate used in the case studies (Figure 5) is a more semantically meaningful measure and should be incorporated into the headline aggregate analysis, not only the case studies.
minor comments (4)
  1. [§1 and various] There are several typos and grammatical errors: 'mathch' in §3, 'adavanced' in §3.2, 'the Task Success Radar on the left, Intention Correctness Radar on the left' in the caption of Figure 3 (one of these should be 'right'), and inconsistent use of 'illustraion' in Figure 2's caption. These do not affect the technical content but should be cleaned up.
  2. [Figure 4] The caption says 'Out-of-Distribution objects are painted in orange,' but the figure appears to use text labels rather than color-coding; the authors should clarify how the colors map to OOD status or update the figure to use a visible visual marker.
  3. [Appendix C] The full results tables report deltas as both absolute and percentage values, but the percentage columns are sometimes ambiguous when the baseline is zero (e.g., entries with 'N/A' or values like '199.76'). The authors should define how deltas are computed for zero baselines and consider omitting percentages when the denominator is zero.
  4. [§4.1] The text states that Octo is evaluated in zero-shot mode because preliminary trials showed zero-shot performs better than fine-tuning, but no data are provided to support this claim. Either include the comparison results in an appendix or remove the claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the intention–action gap is a direct empirical measurement from rollouts, not a quantity derived from fitted parameters or self-citations.

full rationale

This paper is an evaluation/benchmarking study rather than a derivation from first principles. Its central claims—persistent intention–action gap, fragile language generalization, and brittleness under compound visual–language shifts—are computed directly from measured policy rollouts under fixed task definitions and fixed metrics (§4.1). The Intention Correct Rate is a new author-defined metric with an empirically chosen 5 cm threshold (Appendix A), but the threshold is applied uniformly across all models and conditions and is not fitted to any target conclusion or to the headline gap; it is an operationalization, not a fitted parameter renamed as a prediction. The OOD labels in §3.1 are explicitly defined relative to the BridgeV2 fine-tuning dataset because the proprietary π0 pretraining data cannot be audited; the paper states this limitation openly. That is a construct-validity boundary, not a circular reduction, and it does not make the measured drops equal to the definition of OOD by construction. The references to the authors' own prior work ([39], [40], [41]) appear as general VLM pointers and as one speculative suggestion about modularized architectures; they are not load-bearing evidence for any headline result. No equation in the paper defines a predicted quantity in terms of the outcome it is supposed to explain, and no benchmark metric is inferred from the models' own outputs in a way that would make the conclusion tautological. The comparison against PaliGemma VQA (§B) is an independent external probe. Therefore, no significant circularity is present.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The benchmark rests on three unproven premises: the simulation's sim2real match extends to new tasks, OOD relative to BridgeV2 is the right yardstick for every model, and the proximity-based intention metric truly captures intent. The authors acknowledge the second premise and mention the sim2real limitation in Section 5, but none of these premises are experimentally verified within the paper.

free parameters (1)
  • Intention Correct Rate threshold = 5 cm
    Appendix A states the threshold is empirically determined but close to existing SimplerEnv thresholds. The headline intention-action gap is computed from this hand-chosen radius, and no sensitivity analysis is provided.
assumptions (3)
  • domain assumption SimplerEnv's sim-to-real fidelity extends to the newly added OOD tasks and language variations
    The paper builds on SimplerEnv, but the new objects, distractors, and language variants are not validated against real-world robot performance. Sections 3 and 5 acknowledge the sim2real gap but do not verify the extension.
  • domain assumption OOD status relative to BridgeV2 is the correct reference for all evaluated models
    Section 3.1 notes that proprietary pretraining corpora for models like pi-zero cannot be audited, yet all non-BridgeV2 objects and instructions are treated as out-of-distribution for every model.
  • domain assumption Gripper proximity to the source object is a valid proxy for policy intention
    Section 4.1 defines Intention Correct Rate as the gripper moving within a small radius of the correct source object. This conflates reaching near an object with actually intending to grasp it, and the threshold is set empirically in Appendix A.

how reviews work

0 comments
Cite this review

Pith. "Pith review of From Intention to Execution: Probing the Generalization Boundaries of Vision-Language-Action Models." pith.science (2026). https://pith.science/paper/57BKKR5L

@misc{pith2026250609930,
  author       = {Pith},
  title        = {Pith review of: From Intention to Execution: Probing the Generalization Boundaries of Vision-Language-Action Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/57BKKR5L}},
  note         = {Machine review of arXiv:2506.09930}
}
read the original abstract

One promise that Vision-Language-Action (VLA) models hold over traditional imitation learning for robotics is to leverage the broad generalization capabilities of large Vision-Language Models (VLMs) to produce versatile, "generalist" robot policies. However, current evaluations of VLAs remain insufficient. Traditional imitation learning benchmarks are unsuitable due to the lack of language instructions. Emerging benchmarks for VLAs that incorporate language often come with limited evaluation tasks and do not intend to investigate how much VLM pretraining truly contributes to the generalization capabilities of the downstream robotic policy. Meanwhile, much research relies on real-world robot setups designed in isolation by different institutions, which creates a barrier for reproducibility and accessibility. To address this gap, we introduce a unified probing suite of 50 simulation-based tasks across 10 subcategories spanning language instruction, vision, and objects. We systematically evaluate several state-of-the-art VLA architectures on this suite to understand their generalization capability. Our results show that while VLM backbones endow VLAs with robust perceptual understanding and high level planning, which we refer to as good intentions, this does not reliably translate into precise motor execution: when faced with out-of-distribution observations, policies often exhibit coherent intentions, but falter in action execution. Moreover, finetuning on action data can erode the original VLM's generalist reasoning abilities. We release our task suite and evaluation code to serve as a standardized benchmark for future VLAs and to drive research on closing the perception-to-action gap. More information, including the source code, can be found at https://ai4ce.github.io/INT-ACT/

Figures

Figures reproduced from arXiv: 2506.09930 by the authors.

Figure 1
Figure 1. Left: Examples of tasks with out-of-distribution objects. Right: Examples of tasks with commonsense reasoning, distractors, and commonsense reasoning + distractors. We begin by auditing all objects present in the BridgeV2 dataset. To introduce out-of-distribution (OOD) diversity, we first include additional household items inspired by other robotic benchmarks such as LIBERO [22] and Fractal [3], which feature simila… view at source ↗
Figure 2
Figure 2. Illustration of the INT-ACT probing suite. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. The Intention-Action Gap illustrated by comparing the two radar maps. Task Success Radar on the left, Intention Correctness Radar on the left. Best viewed in color. Question 1: How do VLAs generalize to OOD objects? VLAs show robust intention to OOD objects, but execution falls short. Changing the target can affect grasping of the source. We break down per-object OOD performance in [PITH_FULL_IMAGE:figures/full_fig… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: OOD generalization results. Out-of-Distribution objects are painted in [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Case studies showing the impact of visual distractions and language commonsense varia [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Prompting PaliGemma with the task and the first frame of visual observations. Prompts are [PITH_FULL_IMAGE:figures/full_fig_p018_6.png]
Figure 7
Figure 7. Figure 7: Prompting PaliGemma to do VQA on the task instructions and the first frame of visual [PITH_FULL_IMAGE:figures/full_fig_p019_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. RL$^2$-VLA: Adaptive RL Latent Compositional Steering with Test-Time Scaling for Vision-Language-Action Models

    cs.RO 2026-07 conditional novelty 6.0 of 10

    RL² improves VLA robot success rates by conditionally composing an offline RL policy's actions with the frozen VLA only when a failure detector flags impending failure.

  2. Robots Acquire Manipulation Skills in Seconds from a Single Human Video

    cs.RO 2026-07 conditional novelty 6.0 of 10

    A single human video is enough for a robot to acquire a new manipulation skill at inference time, with no parameter updates, reaching 62% success on 50 novel tasks.

  3. Genie Sim 3.0 : A High-Fidelity Comprehensive Simulation Platform for Humanoid Robot

    cs.RO 2026-01 unverdicted novelty 6.0 of 10

    Genie Sim 3.0 introduces an LLM-powered scene generator, the first LLM-based automated evaluation benchmark, and a large open synthetic dataset that demonstrates zero-shot sim-to-real transfer for robotic manipulation...

  4. LIBERO-PRO: Towards Robust and Fair Evaluation of Vision-Language-Action Models Beyond Memorization

    cs.CV 2025-10 conditional novelty 6.0 of 10

    SOTA VLA models like OpenVLA and pi0 collapse when object positions are perturbed, indicating that standard LIBERO scores reward memorization; LIBERO-PRO provides a systematic perturbed evaluation suite.

Reference graph

Works this paper leans on

47 extracted references · 23 canonical work pages · cited by 4 Pith papers

  1. [1]

    Paligemma: A versatile 3b vlm for transfer, 2024

    Lucas Beyer, Andreas Steiner, André Susano Pinto, Alexander Kolesnikov, Xiao Wang, Daniel Salz, Maxim Neumann, Ibrahim Alabdulmohsin, Michael Tschannen, Emanuele Bugliarello, Thomas Unterthiner, Daniel Keysers, Skanda Koppula, Fangyu Liu, Adam Grycner, Alexey Gritsenko, Neil Houlsby, Manoj Kumar, Keran Rong, Julian Eisenschlos, Rishabh Kabra, Matthias Bau...

    arXiv 2024

  2. [2]

    π0: A vision-language-action flow model for general robot control, 2024

    Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Sergey Levine, Adrian Li-Bell, Mohith Mothukuri, Suraj Nair, Karl Pertsch, Lucy Xiaoyang Shi, James Tanner, Quan Vuong, Anna Walling, Haohuan Wang, and Ury Zhilinsky. π0: A visi...

    arXiv 2024

  3. [3]

    Rt-1: Robotics transformer for real-world control at scale

    Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, Julian Ibarz, Brian Ichter, Alex Irpan, Tomas Jackson, Sally Jesmonth, Nikhil Joshi, Ryan Julian, Dmitry Kalashnikov, Yuheng Kuang, Isabel Leal, Kuang-Huei Lee, Sergey Levine, Yao Lu, Utsav Malla, De...

    arXiv 2022

  4. [4]

    Rt-2: Vision-language-action models transfer web knowledge to robotic control

    Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choro- manski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, Pete Florence, Chuyuan Fu, Montse Gonzalez Arenas, Keerthana Gopalakrishnan, Kehang Han, Karol Hausman, Alex Herzog, Jasmine Hsu, Brian Ichter, Alex Irpan, Nikhil Joshi, Ryan Julian, Dmitry Kalashnikov, Y...

    arXiv 2023

  5. [5]

    Language models are few-shot learn- ers

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agar- wal, Ariel Herbert-V oss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Sco...

    work page 1901

  6. [6]

    Pali-x: On scaling up a multilingual vision and language model, 2023

    Xi Chen, Josip Djolonga, Piotr Padlewski, Basil Mustafa, Soravit Changpinyo, Jialin Wu, Carlos Riquelme Ruiz, Sebastian Goodman, Xiao Wang, Yi Tay, Siamak Shakeri, Mostafa Dehghani, Daniel Salz, Mario Lucic, Michael Tschannen, Arsha Nagrani, Hexiang Hu, Mandar Joshi, Bo Pang, Ceslee Montgomery, Paulina Pietrzyk, Marvin Ritter, AJ Piergiovanni, Matthias Mi...

    arXiv 2023

  7. [7]

    Pali: A jointly-scaled multilingual language-image model

    Xi Chen, Xiao Wang, Soravit Changpinyo, AJ Piergiovanni, Piotr Padlewski, Daniel Salz, Sebastian Goodman, Adam Grycner, Basil Mustafa, Lucas Beyer, et al. Pali: A jointly-scaled multilingual language-image model. InICLR, 2023. 1, 2

    work page 2023

  8. [8]

    Diffusion policy: Visuomotor policy learning via action diffusion

    Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. The International Journal of Robotics Research, 2024. 2

    work page 2024

Show all 47 references
  1. [9]

    Open X-Embodiment Collaboration, Abby O’Neill, Abdul Rehman, Abhinav Gupta, Abhi- ram Maddukuri, Abhishek Gupta, Abhishek Padalkar, Abraham Lee, Acorn Pooley, Agrim Gupta, Ajay Mandlekar, Ajinkya Jain, Albert Tung, Alex Bewley, Alex Herzog, Alex Irpan, Alexander Khazatsky, Ana...

    arXiv

  2. [10]

    BERT: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.),Proceedings of the 2019 Conference of the North American Chapter of th...

    2019 doi

  3. [11]

    Danny Driess, Fei Xia, Mehdi S. M. Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, Wenlong Huang, Yevgen Chebotar, Pierre Sermanet, Daniel Duckworth, Sergey Levine, Vincent Vanhoucke, Karol Hausman, Marc Toussaint...

    2023 arXiv

  4. [12]

    Poor performance of openvla on bridge

    Github Issue from user: abadithela. Poor performance of openvla on bridge. https://github. com/simpler-env/SimplerEnv/issues/78, March 2025. URL https://github.com/ simpler-env/SimplerEnv/issues/78. GitHub Issue #78 in simpler-env/SimplerEnv repos- itory. 5

    2025

  5. [13]

    Gemini, 2023

    Google. Gemini, 2023. URL https://blog.google/technology/ai/ google-gemini-ai/. 1

    2023

  6. [14]

    Maniskill2: A unified benchmark for generalizable manipulation skills

    Jiayuan Gu, Fanbo Xiang, Xuanlin Li, Zhan Ling, Xiqiang Liu, Tongzhou Mu, Yihe Tang, Stone Tao, Xinyue Wei, Yunchao Yao, Xiaodi Yuan, Pengwei Xie, Zhiao Huang, Rui Chen, and Hao Su. Maniskill2: A unified benchmark for generalizable manipulation skills. InInternational Conferen...

    2023

  7. [15]

    Rekep: Spatio- temporal reasoning of relational keypoint constraints for robotic manipulation.arXiv preprint arXiv:2409.01652, 2024

    Wenlong Huang, Chen Wang, Yunzhu Li, Ruohan Zhang, and Li Fei-Fei. Rekep: Spatio- temporal reasoning of relational keypoint constraints for robotic manipulation.arXiv preprint arXiv:2409.01652, 2024. 7 12

    2024 arXiv

  8. [16]

    Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Manuel Y . Galliker, Dibya Ghosh, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Devin L...

    2025 arXiv

  9. [17]

    Stephen James, Zicong Ma, David Rovick Arrojo, and Andrew J. Davison. Rlbench: The robot learning benchmark & learning environment.IEEE Robotics and Automation Letters, 2020. 2, 3

    2020

  10. [18]

    Prismatic vlms: Investigating the design space of visually-conditioned language models

    Siddharth Karamcheti, Suraj Nair, Ashwin Balakrishna, Percy Liang, Thomas Kollar, and Dorsa Sadigh. Prismatic vlms: Investigating the design space of visually-conditioned language models. InInternational Conference on Machine Learning (ICML), 2024. 2

    2024

  11. [19]

    Open- vla: An open-source vision-language-action model

    Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan P Foster, Pannag R Sanketi, Quan Vuong, Thomas Kollar, Benjamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, and Chelsea Finn. Open- vla: An ...

    2025

  12. [20]

    Evaluating real-world robot manipulation policies in simulation

    Xuanlin Li, Kyle Hsu, Jiayuan Gu, Karl Pertsch, Oier Mees, Homer Rich Walke, Chuyuan Fu, Ishikaa Lunawat, Isabel Sieh, Sean Kirmani, Sergey Levine, Jiajun Wu, Chelsea Finn, Hao Su, Quan Vuong, and Ted Xiao. Evaluating real-world robot manipulation policies in simulation. arXiv...

    2024 arXiv

  13. [21]

    Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling, 2023. URLhttps://arxiv.org/abs/2210.02747. 2

    2023 arXiv

  14. [22]

    Libero: Benchmarking knowledge transfer for lifelong robot learning.arXiv preprint arXiv:2306.03310,

    Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. Libero: Benchmarking knowledge transfer for lifelong robot learning.arXiv preprint arXiv:2306.03310,

    arXiv

  15. [23]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. In NeurIPS, 2023. 1

    2023

  16. [24]

    Rectified flow: A marginal preserving approach to optimal transport, 2022

    Qiang Liu. Rectified flow: A marginal preserving approach to optimal transport, 2022. URL https://arxiv.org/abs/2209.14577. 2

    2022 arXiv

  17. [25]

    Rdt-1b: a diffusion foundation model for bimanual manipulation.arXiv preprint arXiv:2410.07864, 2024

    Songming Liu, Lingxuan Wu, Bangguo Li, Hengkai Tan, Huayu Chen, Zhengyi Wang, Ke Xu, Hang Su, and Jun Zhu. Rdt-1b: a diffusion foundation model for bimanual manipulation.arXiv preprint arXiv:2410.07864, 2024. 2

    2024 arXiv

  18. [26]

    What matters in learning from offline human demonstrations for robot manipulation

    Ajay Mandlekar, Danfei Xu, Josiah Wong, Soroush Nasiriany, Chen Wang, Rohun Kulkarni, Li Fei-Fei, Silvio Savarese, Yuke Zhu, and Roberto Martín-Martín. What matters in learning from offline human demonstrations for robot manipulation. InarXiv preprint arXiv:2108.03298,

    arXiv

  19. [27]

    Calvin: A benchmark for language-conditioned policy learning for long-horizon robot manipulation tasks.IEEE Robotics and Automation Letters (RA-L), 7(3):7327–7334, 2022

    Oier Mees, Lukas Hermann, Erick Rosete-Beas, and Wolfram Burgard. Calvin: A benchmark for language-conditioned policy learning for long-horizon robot manipulation tasks.IEEE Robotics and Automation Letters (RA-L), 7(3):7327–7334, 2022. 2, 3

    2022

  20. [28]

    RoboManipBaselines, December

    Masaki Murooka, Tomohiro Motoda, and Ryoichi Nakajo. RoboManipBaselines, December

  21. [29]

    Octo: An open-source generalist robot policy

    Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Charles Xu, Jianlan Luo, Tobias Kreiman, You Liang Tan, Lawrence Yun- liang Chen, Pannag Sanketi, Quan Vuong, Ted Xiao, Dorsa Sadigh, Chelsea Finn, and Sergey Levine. Oc...

    2024

  22. [30]

    gpt4o, 2024

    OpenAI. gpt4o, 2024. URLhttps://openai.com/index/hello-gpt-4o/. 1, 2

    2024

  23. [31]

    OpenAI, :, Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander M ˛ adry, Alex Baker-Whitcomb, Alex Beutel, Alex Borzunov, Alex Carney, Alex Chow, Alex Kirillov, Alex Nichol, Alex ...

    2024 arXiv

  24. [32]

    Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V . V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel HAZIZA, Francisco Massa, Alaaeldin El-Nouby, Mido Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael Rab...

    2024

  25. [33]

    Transfer between modalities with metaqueries.arXiv preprint arXiv:2504.06256, 2025

    Xichen Pan, Satya Narayan Shukla, Aashu Singh, Zhuokai Zhao, Shlok Kumar Mishra, Jialiang Wang, Zhiyang Xu, Jiuhai Chen, Kunpeng Li, Felix Juefei-Xu, et al. Transfer between modalities with metaqueries.arXiv preprint arXiv:2504.06256, 2025. 9

    2025 arXiv

  26. [34]

    Fast: Efficient action tokenization for vision-language-action models, 2025

    Karl Pertsch, Kyle Stachowicz, Brian Ichter, Danny Driess, Suraj Nair, Quan Vuong, Oier Mees, Chelsea Finn, and Sergey Levine. Fast: Efficient action tokenization for vision-language-action models, 2025. URLhttps://arxiv.org/abs/2501.09747. 2

    2025 arXiv

  27. [35]

    Spatialvla: Exploring spatial representations for visual-language-action model, 2025

    Delin Qu, Haoming Song, Qizhi Chen, Yuanqi Yao, Xinyi Ye, Yan Ding, Zhigang Wang, JiaYuan Gu, Bin Zhao, Dong Wang, and Xuelong Li. Spatialvla: Exploring spatial representations for visual-language-action model, 2025. URLhttps://arxiv.org/abs/2501.15830. 2, 5, 7

    2025 arXiv

  28. [36]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. InInterna- tional ...

    2021

  29. [37]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer.Journal of Machine Learning Research, 21(140):1–67, 2020. URL http://...

    2020

  30. [38]

    Gemini robotics: Bringing ai into the physical world.arXiv preprint arXiv:2503.20020, 2025

    Gemini Robotics Team, Saminda Abeyruwan, Joshua Ainslie, Jean-Baptiste Alayrac, Montser- rat Gonzalez Arenas, Travis Armstrong, Ashwin Balakrishna, Robert Baruch, Maria Bauza, Michiel Blokzijl, et al. Gemini robotics: Bringing ai into the physical world.arXiv preprint arXiv:25...

    2025 arXiv

  31. [39]

    Cambrian-1: A fully open, vision-centric exploration of multimodal llms

    Shengbang Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Manoj Middepogu, Sai Charitha Akula, Jihan Yang, Shusheng Yang, Adithya Iyer, Xichen Pan, et al. Cambrian-1: A fully open, vision-centric exploration of multimodal llms. InNeurIPS, 2024. 2

    2024

  32. [40]

    Metamorph: Multimodal under- standing and generation via instruction tuning, 2024

    Shengbang Tong, David Fan, Jiachen Zhu, Yunyang Xiong, Xinlei Chen, Koustuv Sinha, Michael Rabbat, Yann LeCun, Saining Xie, and Zhuang Liu. Metamorph: Multimodal under- standing and generation via instruction tuning, 2024. URL https://arxiv.org/abs/2412. 14164. 4

    2024

  33. [41]

    Vlm see, robot do: Human demo video to robot action plan via vision language model.arXiv preprint arXiv:2410.08792, 2024

    Beichen Wang, Juexiao Zhang, Shuwen Dong, Irving Fang, and Chen Feng. Vlm see, robot do: Human demo video to robot action plan via vision language model.arXiv preprint arXiv:2410.08792, 2024. 7 15

    2024

  34. [42]

    Decomposing the generalization gap in imitation learning for visual robotic manipulation

    Annie Xie, Lisa Lee, Ted Xiao, and Chelsea Finn. Decomposing the generalization gap in imitation learning for visual robotic manipulation. In2024 IEEE International Conference on Robotics and Automation (ICRA), pp. 3153–3160. IEEE, 2024. 3, 5

    2024

  35. [43]

    Magma: A foundation model for multimodal ai agents

    Jianwei Yang, Reuben Tan, Qianhui Wu, Ruijie Zheng, Baolin Peng, Yongyuan Liang, Yu Gu, Mu Cai, Seonghyeon Ye, Joel Jang, et al. Magma: A foundation model for multimodal ai agents. arXiv preprint arXiv:2502.13130, 2025. 5, 7

    2025 arXiv

  36. [44]

    Sigmoid loss for language image pre-training

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. InIEEE/CVF International Conference on Computer Vision, ICCV 2023, Paris, France, October 1-6, 2023, pp. 11941–11952. IEEE, 2023. doi: 10.1109/ ICCV51070.2023.0110...

    2023

  37. [45]

    Transfusion: Predict the next token and diffuse images with one multi-modal model.arXiv preprint arXiv:2408.11039, 2024

    Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke Zettlemoyer, and Omer Levy. Transfusion: Predict the next token and diffuse images with one multi-modal model.arXiv preprint arXiv:2408.11039, 2024. 2

    2024 arXiv

  38. [46]

    carrot on plate

    Orr Zohar, Kuan-Chieh Wang, and Serena Yeung. Prob: Probabilistic objectness for open world object detection. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11444–11453, June 2023. 3 16 A Experiment details Training π0 on Bridge...

    2023

  39. [2024]

    URLhttps://github.com/isri-aist/RoboManipBaselines. 3

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.