Pith. sign in

REVIEW 4 major objections 6 minor 2 cited by

ACE-Brain-0.5: A Unified Embodied Foundational Model for Physical Agentic AI

T0 review · 4 major / 6 minor · reviewed 2026-07-11 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read One 8B robot model can run perception, planning, action, and progress monitoring as a single closed loop.

desk verdict Solid four-function closed-loop 8B robot brain with a practical merge recipe; the fifth function (self-improvement) is mostly harness and oracle flywheel, not backbone self-evolution. read the letter →

arxiv 2607.04426 v1 pith:EHOD6VFY submitted 2026-07-05 cs.RO

ACE-Brain Team: Ziyang Gong , Haoming Gu , Zehang Luo , Tianyi Zhang , Tao Tao , Yixiao Chi , Zhe Liu , Lingsi Zhu
show 22 more authors
Jingyuan Liu Anke Tang Songze Li Yilun Kong Ningjing Liu Tianyu Zhu Yunpeng Qing Shuang Luo Xiang Liu Shi Fu Dawei Nie Sixiang Liu Zhexi Wen Feng Pan Xiaofeng Wang Zhi Hou Chunxiao Liu Xue Yang Junchi Yan Hengshuang Zhao Dacheng Tao Xiaogang Wang
This is my paper · ORCID
classification cs.RO
keywords embodiedfoundationmodelphysicalagenticAIspatialintelligencevision-language-actiontask-vectormergingprogressestimationself-improvingrobotsSSR+
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that general physical agentic AI needs one shared robot brain, not separate perception, planning, action, and evaluation modules. ACE-Brain-0.5 builds on a prior spatial-intelligence model and puts four of five core functions—spatial perception, decision making, embodied interaction, and self-monitoring—into a single 8B backbone that grounds objects, plans subgoals, navigates and manipulates, and scores execution progress. A fifth function, self-improvement, updates external task memory and recovery cases from rollouts rather than retraining the whole network. The authors introduce SSR+, a train-specialize-merge-then-reactivate recipe that fuses heterogeneous task interfaces without them cancelling each other out. Across more than fifteen benchmarks, the unified model beats its predecessor on most spatial and grounding tests, stays competitive on navigation and manipulation, and estimates progress well in both in-distribution and out-of-distribution settings.

What carries the argument

SSR+ (Scaffold–Specialize–Reconcile–Reactivate): start from a spatial scaffold, train specialist checkpoints for QA, grounding, navigation, and progress, merge them by task-vector fusion that preserves intermediate representations, then run a short mixed fine-tune to re-align output formats so one model can switch interfaces without cross-task collapse.

What would settle it

Train the same data with plain mixed supervised fine-tuning versus SSR+; if the merged-then-reactivated model loses navigation success, manipulation success, or progress correlation relative to specialists or mixed SFT, or if oracle-free closed-loop rollouts show no gain after external-state updates, the unification claim fails.

Watch

Extended reading notes

Core claim

Spatial perception, decision making, embodied interaction, self-monitoring, and self-improvement can be unified inside one robot foundation model: a single 8B backbone realizes the first four as a Perception–Planning–Action–Evaluation loop, while self-improvement is supplied by a companion framework that revises external execution state from experience, and joint training under SSR+ yields gains over the prior understanding-only model and competitive results against specialist systems.

Load-bearing premise

That after merging specialists, the model still holds the real task knowledge and only needs a brief fine-tune to fix output formats, and that updating external memory and recovery cases from rollouts counts as self-improvement without changing the model’s own weights.

Editorial extensions

If this is right

  • Robot stacks can drop multi-model orchestration for grounding, planning, control, and progress scoring and run them from one backbone.
  • Spatial pretraining becomes the shared initialization for navigation, manipulation, and reward-style progress estimation.
  • Progress scores from the same model can serve as rewards or recovery triggers in closed-loop training.
  • Self-improvement can start as lightweight updates to task schemas and failure cases rather than full weight retraining.
  • Cross-task interference in multi-interface robot models can be managed by merge-then-reactivate instead of only data mixing.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If Reactivate only restores formats, similar merge-and-calibrate pipelines may scale to more action interfaces (mobile base, dual-arm, whole-body) without full multi-task retraining.
  • Strong OOD progress estimation suggests the same backbone could grade other agents’ trajectories as a shared critic, not only its own rollouts.
  • Keeping self-improvement outside the weights may trade long-term policy evolution for deployability; a natural next test is whether external H updates eventually demand model-level fine-tuning to keep improving.
  • Driving-benchmark dips while spatial and interaction gains rise imply a capacity trade-off: unified brains may need explicit load balancing across domains as more interfaces are added.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. ACE-Brain-0.5 claims a Unified Embodied Foundation Model that organizes robot intelligence into five functions—spatial perception, decision making, embodied interaction, self-monitoring, and self-improvement—on a single 8B backbone built from ACE-Brain-0. The first four are realized as a Perception–Planning–Action–Evaluation loop via an omni-vision encoder, LLM decoder, Fast Vision pathway, and flow-matching Action Expert; heterogeneous interfaces are unified by SSR+ (Scaffold–Specialize–Reconcile plus Reactivate after task-vector merging). Self-improvement is a companion framework that updates external execution state H and a navigation-evolving flywheel that uses oracle-recovered corrective trajectories. Across 15+ benchmarks the model improves ACE-Brain-0 on most spatial/grounding tasks, is competitive on VLN-CE and LIBERO/SimplerEnv, and leads on RBM-EVAL progress estimation including reverse-trajectory refined splits.

Significance. If the results hold, the paper is a substantial systems contribution: it shows that spatial QA, 3D grounding, navigation, continuous manipulation, and progress estimation can share one backbone without catastrophic interference, with SSR+ as a practical multi-interface recipe and RBM-EVAL-Refined as a useful temporal-direction control. Strengths include broad external benchmarks (Tables 2–6), competitive or SOTA manipulation (Tables 4–5), strong progress VOC including OOD and refined reverse trajectories (Table 6, Fig. 3), and an explicit architecture for low-latency control (Fast Vision + Action Expert). The five-function framing and Table 1 Self-Improving checkmark overstate what the single model learns relative to the external/oracle self-improvement path, but the four-function closed loop is still a clear step beyond fragmented VLA or agent stacks.

major comments (4)
  1. Abstract, §3.3 Eq. (10), and Table 1: Self-improvement is defined as updates to external H (schemas, spatial memory, failure cases) without weight updates, and §4.5 Navigation Evolving relies on an oracle teacher that detects deviations and produces Devo recovery trajectories before re-imitation (Table 7). That is a closed-loop data flywheel plus harness adaptation, not model-level self-evolution of the backbone that implements the other four functions. Table 1’s full ✓ for Self-Improving and the five-function “unified in one model” claim therefore rest on a broader definition than the four-function loop. Either restrict Self-Improving to partial support / companion framework, or add ablations that improve without oracle recovery and with model-weight adaptation from self-generated experience.
  2. §3.2 Stages 3–4 and Appendix A.5: The load-bearing claim that task-vector merging preserves specialist semantic knowledge so that only output-format conventions need short Reactivate SFT is asserted empirically (“central empirical finding”) and via transfer bounds adapted from ACE-Brain-0, but the manuscript does not report pre- vs post-merge vs post-Reactivate metrics per interface, nor Reactivate step budget / Dmix composition. Without those ablations, it is hard to separate true multi-task unification from residual specialist capacity plus light mixed SFT. A compact table of θ_i / θ_merge / θ_0.5 on grounding, nav, progress, and spatial QA would make the SSR+ claim falsifiable.
  3. Table 2 driving block and §4.2: ACE-Brain-0.5 drops relative to ACE-Brain-0 on several decision-centric driving metrics (e.g., MAPLM 71.3 vs 77.8, NuScenesQA 44.7 vs 58.8, LingoQA 55.6 vs 65.8, MME-RealWorld 66.6 vs 71.2). The text acknowledges moderate drops but still markets unified decision making. Either quantify the multi-task trade-off (SSR+ vs single-task specialists on the same backbone) or narrow the decision-making claim to planning/subgoal decomposition rather than driving-specialized QA.
  4. Tables 3–5: Navigation and manipulation results mix the unified model with ACE-Brain-0.5-Specialist (VLN) and ACE-Brain-0.5-VLA (SimplerEnv; backbone fully fine-tuned, no FastVision, and without ACE-Brain-0.5 manipulation pretraining weights). The central “single unified architecture without task-specific architectural modifications” claim (§4 intro) is therefore only partially supported for embodied interaction. Report the frozen-backbone unified model on SimplerEnv and the non-specialist unified model as the primary numbers; treat specialist/VLA variants as ablations.
minor comments (6)
  1. Table 1 legend and Self-Improving row: align the ✓ definition with the external-H / oracle description in §3.3–§4.5 so the comparison table does not over-claim relative to peers.
  2. Eq. (4)–(5): clarify the relationship between the layer-wise output-discrepancy objective and the Frobenius task-vector surrogate; a one-line statement of what is optimized in FusionBench would help reproducibility.
  3. Progress supervision (§5.1): scaling labels by 1000 and T=8 sampling are free design choices; a short sensitivity note would strengthen the reward-model claim.
  4. Appendix A.4 / Table 8: RBM-EVAL-Refined is a useful control; state selection criteria and whether reverse trajectories are used only at eval or also in training.
  5. Presentation: long nested pie charts in Fig. 5 and dense multi-model Table 2 would benefit from clearer ranking highlights and consistent asterisks for numbers taken from other papers.
  6. Date line “July 7, 2026” / arXiv 2607.04426: ensure consistency with camera-ready metadata.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: empirical multi-task systems paper evaluated on external benchmarks; self-improvement is explicitly external/oracle-aided rather than a definitional self-prediction.

full rationale

ACE-Brain-0.5 is an empirical foundation-model paper. The central claims (single 8B backbone for spatial perception, decision making, embodied interaction, and self-monitoring via SSR+; competitive results on VSI, MMSI, MindCube, Scan*, VLN-CE, LIBERO, SimplerEnv-Bridge, RBM-EVAL, etc.) are supported by training on public and in-house corpora and evaluation against external or held-out benchmarks and baselines. SSR+ is a training recipe (scaffold from ACE-Brain-0, specialize, task-vector merge, short Reactivate SFT); the Reactivate recovery claim is an empirical observation about merged weights, not a quantity fitted then re-predicted. Self-improvement is stated as a companion framework that updates external execution state H (Eq. 10) and a navigation flywheel that uses an oracle teacher to build corrective Devo trajectories (Table 7)—transparent closed-loop data collection, not a self-definitional prediction of the model’s own success. RBM-EVAL-Refined adds reversed trajectories as negative controls; that is a controlled test design, not fitting the VOC metric to the model. Self-citation of ACE-Brain-0 (scaffold, SSR, Appendix A.5 theorems) and Robometer-style data is normal prior-work scaffolding and does not force the multi-benchmark results by construction. No step reduces a claimed first-principles prediction to its own fitted input or to an unverified uniqueness theorem. Score 1 reflects only routine self-citation of the prior scaffold, not load-bearing circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 3 invented entities

Empirical robotics/ML systems paper. Load-bearing content is architectural choices, multi-task merge recipe, and large heterogeneous datasets rather than free physical constants. Domain assumptions (spatial scaffold transfers to action; flow-matching action expert; external H as self-improvement) are stated or implied; free parameters are standard training/merge knobs.

free parameters (4)
  • SSR+ merge iterations and layer-wise objective weights = 1000 iterations
    1,000 data-free Adam iterations via FusionBench; layer-wise residual interference objective is a design choice that shapes the merged checkpoint.
  • Reactivate mixed-SFT step budget and Dmix composition
    Lightweight post-merge fine-tune size and per-interface sampling ratios are chosen to restore formats without re-specializing.
  • Action chunk length / flow-matching schedule and FastVision injection = 10-step action sequence (2B variant detail)
    10-step chunks, DINOv3 FastVision path, frozen VLM during action training are engineering hyperparameters controlling control latency and performance.
  • Progress token scaling (×1000) and T=8 frame sampling = T=8; scores scaled to [0,1000]
    Autoregressive progress representation and frame sampling strategy for RBM-1M are design choices affecting supervision density.
assumptions (4)
  • domain assumption Spatial intelligence is a transferable shared scaffold across heterogeneous robot platforms and tasks.
    Inherited from ACE-Brain-0 and used as Stage-1 initialization for all specialists (§1, §3.2).
  • ad hoc to paper Task-vector merging approximately preserves specialist intermediate representations; residual loss is mainly output-format desynchronization.
    Central empirical premise of Reactivate (§3.2 Stage 4); supported by recovery observations but not proved for all interfaces.
  • domain assumption Flow-matching action experts conditioned on cached LLM state plus fast vision features suffice for closed-loop manipulation/navigation control.
    Follows π0-style design (§3.1); standard in recent VLA literature.
  • ad hoc to paper External execution-state updates (schemas, spatial memory, failure cases) plus oracle-recovered trajectories count as Self Improvement for Physical Agentic AI.
    Defines the fifth function without requiring continual weight updates (§3.3, §4.5).
invented entities (3)
  • SSR+ (Scaffold–Specialize–Reconcile–Reactivate)
    purpose: Unify heterogeneous robot interfaces in one parameter space without cross-task interference.
    Extends authors’ SSR with a post-merge Reactivate stage; independent evidence is the reported multi-benchmark recovery, not external theory.
  • Five-function Unified Embodied Foundation Model paradigm
    purpose: Organize robot intelligence as spatial perception, decision making, embodied interaction, self-monitoring, self-improvement.
    Framing device for architecture and evaluation; not a physical entity with external measurement.
  • RBM-EVAL-Refined reverse-trajectory control set
    purpose: Test temporal/action-direction understanding beyond monotonic progress shortcuts.
    Author-constructed complement to Robometer splits; useful control but paper-defined.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ACE-Brain-0.5: A Unified Embodied Foundational Model for Physical Agentic AI." pith.science (2026). https://pith.science/paper/EHOD6VFY

@misc{pith2026260704426,
  author       = {Pith},
  title        = {Pith review of: ACE-Brain-0.5: A Unified Embodied Foundational Model for Physical Agentic AI},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EHOD6VFY}},
  note         = {Machine review of arXiv:2607.04426}
}
read the original abstract

Embodied AI is moving from isolated perception or action modules toward physical agents that understand, plan under goals, act through robot bodies, monitor progress, and improve from experience. Existing systems address this loop only in parts: end-to-end policies generate actions but often lack spatial reasoning, planning, and execution assessment, while robot-agent systems orchestrate tools or specialists but do not learn a shared representation. This fragmentation limits general Physical Agentic AI. We present ACE-Brain-0.5, a unified embodied foundation model that organizes robot intelligence into five coupled functions: spatial perception, decision making, embodied interaction, self-monitoring, and self-improvement. Built on ACE-Brain-0, which established spatial intelligence as a shared scaffold across robot platforms, ACE-Brain-0.5 extends an understanding-centric model into a closed-loop foundation model. A single 8B backbone instantiates the first four functions: grounding objects and affordances, reasoning over 3D and egocentric spatial relations, decomposing instructions into subgoals, generating navigation and manipulation actions, and estimating progress for verification and recovery. To unify these capabilities without cross-task interference, we introduce SSR+, which extends Scaffold-Specialize-Reconcile with a Reactivate stage after task-vector merging. The fifth function, self-improvement, is realized by a companion framework that updates external execution state, including task schemas, spatial memory, and failure-recovery cases, from rollouts. Across fifteen benchmarks, ACE-Brain-0.5 improves over ACE-Brain-0 on 14 of 18 spatial perception and grounding benchmarks, achieves competitive navigation and manipulation performance, and provides strong progress estimation in ID and OOD settings. Together, these results mark an early step toward general Physical Agentic AI.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Capek 0.5: An Execution-Centric Vision-Language Model for Embodied Intelligence

    cs.AI 2026-08 conditional novelty 6.0 of 10

    An execution-centric training recipe that specializes a vision-language model into four capability experts and consolidates them via weight merging and distillation yields a single embodied model with broad benchmark gains.

  2. OC-VLA++: Monocular Geometry-Guided Cross-View Consistency for Viewpoint-Robust Robotic Manipulation

    cs.RO 2026-08 conditional novelty 6.0 of 10

    OC-VLA++ augments camera-space action grounding with synthesized nearby views and a cross-view action-equivariance loss, improving robot manipulation success under unseen camera poses.

Reference graph

Works this paper leans on

169 extracted references · 78 linked inside Pith · cited by 2 Pith papers

  1. [1]

    A survey of embodied ai: From simulators to research tasks

    Jiafei Duan, Samson Yu, Hui Li Tan, Hongyuan Zhu, and Cheston Tan. A survey of embodied ai: From simulators to research tasks. IEEE Transactions on Emerging Topics in Computational Intelligence, 6(2):230–244, 2022

    2022

  2. [2]

    A survey on robotics with foundation models: Toward embodied ai

    Zhiyuan Xu, Kun Wu, Junjie Wen, Jinming Li, Ning Liu, Zhengping Che, and Jian Tang. A survey on robotics with foundation models: Toward embodied ai. arXiv preprint arXiv:2402.02385, 2024

    arXiv 2024

  3. [3]

    Robotwin 2.0: A scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipulation

    Tianxing Chen, Zanxin Chen, Baijun Chen, Zijian Cai, Yibin Liu, Zixuan Li, Qiwei Liang, Xianliang Lin, Yiheng Ge, Zhenyu Gu, et al. Robotwin 2.0: A scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipulation. arXiv preprint arXiv:2506.18088, 2025

    arXiv 2025

  4. [4]

    Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models

    Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504.10479, 2025

    arXiv 2025

  5. [5]

    Qwen3-vl technical report

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Junyang Lin, Xuejing Liu, Jiawei Liu, Chenglong Liu, Yang Liu, Dayiheng Liu, Shixuan ...

    arXiv 2025

  6. [6]

    Qwen3 technical report

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025

    arXiv 2025

  7. [7]

    Gemini: a family of highly capable multimodal models

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023

    arXiv 2023

  8. [8]

    Qwen-robotnav technical report: A scalable navigation model designed for an agentic navigation system

    Jiazhao Zhang, Gengze Zhou, Hale Yin, Yiyang Huang, Zixing Lei, Qihang Peng, Haoqi Yuan, Jie Zhang, Xudong Guo, Xiaoyue Chen, et al. Qwen-robotnav technical report: A scalable navigation model designed for an agentic navigation system. arXiv preprint arXiv:2606.18112, 2026

    arXiv 2026

Show all 169 references
  1. [9]

    Embodied navigation foundation model

    Jiazhao Zhang, Anqi Li, Yunpeng Qi, Minghan Li, Jiahang Liu, Shaoan Wang, Haoran Liu, Gengze Zhou, Yuze Wu, Xingxing Li, et al. Embodied navigation foundation model. arXiv preprint arXiv:2509.12129, 2025

    2025

  2. [10]

    π0: A vision-language-action flow model for general robot control

    Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al. π0: A vision-language-action flow model for general robot control. arXiv preprint arXiv:2410.24164, 2024

    2024 arXiv

  3. [11]

    World action models are zero-shot policies

    Seonghyeon Ye, Yunhao Ge, Kaiyuan Zheng, Shenyuan Gao, Sihyun Yu, George Kurian, Suneel Indupuru, You Liang Tan, Chuning Zhu, Jiannan Xiang, et al. World action models are zero-shot policies. arXiv preprint arXiv:2602.15922, 2026

    2026 arXiv

  4. [12]

    RoboReward: General-purpose vision-language reward models for robotics

    Tony Lee et al. RoboReward: General-purpose vision-language reward models for robotics. arXiv preprint arXiv:2601.00675, 2026

    2026

  5. [13]

    RoboMeter: Scaling general-purpose robotic reward models via trajectory comparisons

    Anthony Liang et al. RoboMeter: Scaling general-purpose robotic reward models via trajectory comparisons. arXiv preprint arXiv:2603.02115, 2026

    2026 arXiv

  6. [14]

    Nils J. Nilsson. Shakey the robot. In SRI International Technical Note 323. SRI International, 1984

    1984

  7. [15]

    Rodney A. Brooks. A robust layered control system for a mobile robot. IEEE Journal on Robotics and Automation, 2(1):14–23, 1986

    1986

  8. [16]

    Robin R. Murphy. Introduction to AI Robotics. MIT Press, 2000

    2000

  9. [17]

    π0.5: A vision-language-action model with open-world generalization

    Physical Intelligence, Kevin Black, et al. π0.5: A vision-language-action model with open-world generalization. arXiv preprint arXiv:2504.16054, 2025. 24

    2025 arXiv

  10. [18]

    Interleave-vla: Enhancing robot manipulation with interleaved image-text instructions, 2025

    Cunxin Fan, Xiaosong Jia, Yihang Sun, Yixiao Wang, Jianglan Wei, Ziyang Gong, Xiangyu Zhao, Masayoshi Tomizuka, Xue Yang, Junchi Yan, and Mingyu Ding. Interleave-vla: Enhancing robot manipulation with interleaved image-text instructions, 2025. URLhttps://arxiv.org/abs/2505.02152

    2025

  11. [19]

    Do as i can, not as i say: Grounding language in robotic affordances

    Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Chuyuan Fu, Karol Gopalakrishnan, Karol Hausman, et al. Do as i can, not as i say: Grounding language in robotic affordances. In Conference on Robot Learning (CoRL), 2022

    2022

  12. [20]

    PaLM-E: An embodied multimodal language model

    Danny Driess et al. PaLM-E: An embodied multimodal language model. In International Conference on Machine Learning (ICML), 2023

    2023

  13. [21]

    Code as policies: Language model programs for embodied control

    Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng. Code as policies: Language model programs for embodied control. In IEEE International Conference on Robotics and Automation (ICRA), pages 9493–9500, 2023

    2023

  14. [22]

    Voxposer: Composable 3d value maps for robotic manipulation with language models

    Wenlong Huang, Chen Wang, Ruohan Zhang, Yunzhu Li, Jiajun Wu, and Li Fei-Fei. Voxposer: Composable 3d value maps for robotic manipulation with language models. In Conference on Robot Learning (CoRL), 2023

    2023

  15. [23]

    RoboCodeX: Multimodal code generation for robotic behavior synthesis

    Yao Mu et al. RoboCodeX: Multimodal code generation for robotic behavior synthesis. In International Conference on Machine Learning (ICML), 2024

    2024

  16. [24]

    RoboAgent: Chaining basic capabilities for embodied task planning

    Peiran Xu, Jiaqi Zheng, and Yadong Mu. RoboAgent: Chaining basic capabilities for embodied task planning. arXiv preprint arXiv:2604.07774, 2026

    2026 arXiv

  17. [25]

    Gr00t n1: An open foundation model for generalist humanoid robots

    NVIDIA, Johan Bjorck, Fernando Casta˜ neda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi Fan, Yu Fang, Dieter Fox, Fengyuan Hu, Spencer Huang, Joel Jang, Zhenyu Jiang, Jan Kautz, Kaushil Kundalia, Lawrence Lao, Zhiqi Li, Zongyu Lin, Kevin Lin, Guilin Liu, Edith Llontop, Loi...

    2025 doi arXiv

  18. [26]

    Gr00t n1.5: An improved open foundation model for generalist humanoid robots

    NVIDIA. Gr00t n1.5: An improved open foundation model for generalist humanoid robots. https://research. nvidia.com/labs/gear/gr00t-n1_5/, 2025. NVIDIA GEAR Lab technical blog

    2025

  19. [27]

    Qwen-vla: Unifying vision-language-action modeling across tasks, environments, and robot embodiments

    Qiuyue Wang, Mingsheng Li, Jian Guan, Jinhui Ye, Sicheng Xie, Yitao Liu, Junhao Chen, Zhixuan Liang, Jie Zhang, Xintong Hu, et al. Qwen-vla: Unifying vision-language-action modeling across tasks, environments, and robot embodiments. arXiv preprint arXiv:2605.30280, 2026

    2026 arXiv

  20. [28]

    Qwen-robotworld technical report: Unifying embodied world modeling through language- conditioned video generation

    Jie Zhang, Xiaoyue Chen, Anzhe Chen, Chenxu Lv, Deqing Li, Gengze Zhou, Hang Yin, Haoqi Yuan, Haoyang Li, Jiahao Li, et al. Qwen-robotworld technical report: Unifying embodied world modeling through language- conditioned video generation. arXiv preprint arXiv:2606.17030, 2026

    2026 arXiv

  21. [29]

    Abot-m0: Vla foundation model for robotic manipulation with action manifold learning

    Yandan Yang, Shuang Zeng, Tong Lin, Xinyuan Chang, Dekang Qi, Junjin Xiao, Haoyun Liu, Ronghan Chen, Yuzhi Chen, Dongjie Huo, et al. Abot-m0: Vla foundation model for robotic manipulation with action manifold learning. arXiv preprint arXiv:2602.11236, 2026

    2026 arXiv

  22. [30]

    Rynnbrain: Open embodied foundation models

    Ronghao Dang, Jiayan Guo, Bohan Hou, Sicong Leng, Kehan Li, Xin Li, Jiangpin Liu, Yunxuan Mao, Zhikai Wang, Yuqian Yuan, et al. Rynnbrain: Open embodied foundation models. arXiv preprint arXiv:2602.14979, 2026

    2026

  23. [31]

    Cosmos 3: Omnimodal world models for physical ai

    Niket Agarwal, Arslan Ali, Jon Allen, Martin Antolini, Adeline Aubame, Alisson Azzolini, Junjie Bai, Maciej Bala, Yogesh Balaji, Josh Bapst, et al. Cosmos 3: Omnimodal world models for physical ai. arXiv preprint arXiv:2606.02800, 2026

    2026 arXiv

  24. [32]

    Ace-brain-0: Spatial intelligence as a shared scaffold for universal embodiments

    Ziyang Gong, Zehang Luo, Anke Tang, Zhe Liu, Shi Fu, Zhi Hou, Ganlin Yang, Weiyun Wang, Xiaofeng Wang, Jianbo Liu, Gen Luo, Haolan Kang, Shuang Luo, Yue Zhou, Yong Luo, Li Shen, Xiaosong Jia, Yao Mu, Xue Yang, Chunxiao Liu, Junchi Yan, Hengshuang Zhao, Dacheng Tao, and Xiaogan...

    2026

  25. [33]

    Pelican-unify 1.0: A unified embodied intelligence model for understanding, reasoning, imagination and action

    Yi Zhang, Yinda Chen, Che Liu, Zeyuan Ding, Jin Xu, Shilong Zou, Junwei Liao, Jiayu Hu, Xiancong Ren, Xiaopeng Zhang, et al. Pelican-unify 1.0: A unified embodied intelligence model for understanding, reasoning, imagination and action. arXiv preprint arXiv:2605.15153, 2026. 25

    2026 arXiv

  26. [34]

    Hy-embodied-0.5: Embodied foundation models for real-world agents

    HY Team, Xumin Yu, Zuyan Liu, Ziyi Wang, He Zhang, Yongming Rao, Fangfu Liu, Yani Zhang, Ruowen Zhao, Oran Wang, et al. Hy-embodied-0.5: Embodied foundation models for real-world agents. arXiv preprint arXiv:2604.07430, 2026

    2026 arXiv

  27. [35]

    Physical Intelligence, Bo Ai, Ali Amin, R Aniceto, A Balakrishna, G Balke, K Black, G Bokinsky, S Cao, T Charbonnier, et al. π0. 7: a steerable generalist robotic foundation model with emergent capabilities, 2026. URL https://arxiv. org/abs/2604.15483

    2026 arXiv

  28. [36]

    π∗ 0.6: a vla that learns from experience

    Physical Intelligence, Ali Amin, Raichelle Aniceto, Ashwin Balakrishna, Kevin Black, Ken Conley, Grace Connors, James Darpinian, Karan Dhabalia, Jared DiCarlo, et al. π∗ 0.6: a vla that learns from experience. arXiv preprint arXiv:2511.14759, 2025

    2025 arXiv

  29. [37]

    Embodied-r1

    Yifu Yuan, Yaoting Huang, Xianze Yao, Yutong Li, Shuoheng Zhang, Linqi Han, Pengyi Li, Jiangeng Sun, Wenting Jia, Zhao Zhang, et al. Embodied-r1. 5: Evolving physical intelligence via embodied foundation models. arXiv preprint arXiv:2606.11324, 2026

    2026 arXiv

  30. [38]

    Molmoact2: Action reasoning models for real-world deployment

    Haoquan Fang, Jiafei Duan, Donovan Clay, Sam Wang, Shuo Liu, Weikai Huang, Xiang Fan, Wei-Chuan Tsai, Shirui Chen, Yi Ru Wang, et al. Molmoact2: Action reasoning models for real-world deployment. arXiv preprint arXiv:2605.02881, 2026

    2026 arXiv

  31. [39]

    Qwen-robotmanip technical report: Alignment unlocks scale for robotic manipulation foundation models

    Haoqi Yuan, Zhixuan Liang, Anzhe Chen, Ye Wang, Haoyang Li, Pei Lin, Yiyang Huang, Zixing Lei, Tong Zhang, Jiazhao Zhang, et al. Qwen-robotmanip technical report: Alignment unlocks scale for robotic manipulation foundation models. arXiv preprint arXiv:2606.17846, 2026

    2026 arXiv

  32. [40]

    Abot-n0: Technical report on the vla foundation model for versatile embodied navigation

    Zedong Chu, Shichao Xie, Xiaolong Wu, Yanfen Shen, Minghua Luo, Zhengbo Wang, Fei Liu, Xiaoxu Leng, Junjun Hu, Mingyang Yin, et al. Abot-n0: Technical report on the vla foundation model for versatile embodied navigation. arXiv preprint arXiv:2602.11598, 2026

    2026

  33. [41]

    Gemini Robotics-ER 1.6: Powering real-world robotics tasks through enhanced em- bodied reasoning

    Laura Graesser and Peng Xu. Gemini Robotics-ER 1.6: Powering real-world robotics tasks through enhanced em- bodied reasoning. https://deepmind.google/blog/gemini-robotics-er-1-6/ , April 2026. Google DeepMind Blog, accessed June 26, 2026

    2026

  34. [42]

    Introducing helix 02: Full-body autonomy, 2026

    AI Figure. Introducing helix 02: Full-body autonomy, 2026

    2026

  35. [43]

    Model merging in llms, mllms, and beyond: Methods, theories, applications, and opportunities

    Enneng Yang, Li Shen, Guibing Guo, Xingwei Wang, Xiaochun Cao, Jie Zhang, and Dacheng Tao. Model merging in llms, mllms, and beyond: Methods, theories, applications, and opportunities. ACM Computing Surveys, 58(8):1–41, 2026

    2026

  36. [44]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025

    2025 arXiv

  37. [45]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 26296–26306, 2024

    2024

  38. [46]

    Gpt-4o system card.https://openai.com/index/gpt-4o-system-card/, 2025

    OpenAI. Gpt-4o system card.https://openai.com/index/gpt-4o-system-card/, 2025

    2025

  39. [47]

    Claude sonnet 4

    Anthropic. Claude sonnet 4. 2025

    2025

  40. [48]

    SpatialVLM: Endowing vision-language models with spatial reasoning capabilities

    Boyuan Chen et al. SpatialVLM: Endowing vision-language models with spatial reasoning capabilities. In CVPR, 2024

    2024

  41. [49]

    RoboPoint: A vision-language model for spatial affordance prediction for robotics

    Wentao Yuan et al. RoboPoint: A vision-language model for spatial affordance prediction for robotics. In Conference on Robot Learning (CoRL), 2024

    2024

  42. [50]

    RoboRefer: Towards spatial referring with reasoning in vision-language models for robotics

    Enshen Zhou et al. RoboRefer: Towards spatial referring with reasoning in vision-language models for robotics. arXiv preprint arXiv:2506.04308, 2025

    2025

  43. [51]

    Robobrain: A unified brain model for robotic manipulation from abstract to concrete

    Yuheng Ji, Huajie Tan, Jiayu Shi, Xiaoshuai Hao, Yuan Zhang, Hengyuan Zhang, Pengwei Wang, Mengdi Zhao, Yao Mu, Pengju An, et al. Robobrain: A unified brain model for robotic manipulation from abstract to concrete. In Proceedings of the Computer Vision and Pattern Recognition ...

    2025

  44. [52]

    Robobrain 2.5: Depth in sight, time in mind

    Huajie Tan, Enshen Zhou, Zhiyu Li, Yijie Xu, Yuheng Ji, Xiansheng Chen, Cheng Chi, Pengwei Wang, Huizhu Jia, Yulong Ao, et al. Robobrain 2.5: Depth in sight, time in mind. arXiv preprint arXiv:2601.14352, 2026. 26

    2026

  45. [53]

    Mimo-embodied: X-embodied foundation model technical report

    Xiaoshuai Hao, Lei Zhou, Zhijian Huang, Zhiwen Hou, Yingbo Tang, Lingfeng Zhang, Guang Li, Zheng Lu, Shuhuai Ren, Xianhui Meng, et al. Mimo-embodied: X-embodied foundation model technical report. arXiv preprint arXiv:2511.16518, 2025

    2025 arXiv

  46. [54]

    Visual embodied brain: Let multimodal large language models see, think, and control in spaces

    Gen Luo, Ganlin Yang, Ziyang Gong, Guanzhou Chen, Haonan Duan, Erfei Cui, Ronglei Tong, Zhi Hou, Tianyi Zhang, Zhe Chen, et al. Visual embodied brain: Let multimodal large language models see, think, and control in spaces. arXiv preprint arXiv:2506.00123, 2025

    2025 arXiv

  47. [55]

    Robobrain 2.0 technical report

    BAAI RoboBrain Team, Mingyu Cao, Huajie Tan, Yuheng Ji, Xiansheng Chen, Minglan Lin, Zhiyu Li, Zhou Cao, Pengwei Wang, Enshen Zhou, et al. Robobrain 2.0 technical report. arXiv preprint arXiv:2507.02029, 2025

    2025 arXiv

  48. [56]

    Vlaser: Vision-language-action model with synergistic embodied reasoning

    Ganlin Yang, Tianyi Zhang, Haoran Hao, Weiyun Wang, Yibin Liu, Dehui Wang, Guanzhou Chen, Zijian Cai, Junting Chen, Weijie Su, et al. Vlaser: Vision-language-action model with synergistic embodied reasoning. arXiv preprint arXiv:2510.11027, 2025

    2025

  49. [57]

    Pelican-vl 1.0: A foundation brain model for embodied intelligence

    Yi Zhang, Che Liu, Xiancong Ren, Hanchu Ni, Shuai Zhang, Zeyuan Ding, Jiayu Hu, Hanzhe Shan, Zhenwei Niu, Zhaoyang Liu, et al. Pelican-vl 1.0: A foundation brain model for embodied intelligence. arXiv preprint arXiv:2511.00108, 2025

    2025

  50. [58]

    RT-1: Robotics transformer for real-world control at scale

    Anthony Brohan et al. RT-1: Robotics transformer for real-world control at scale. In Robotics: Science and Systems (RSS), 2023

    2023

  51. [59]

    RT-2: Vision-language-action models transfer web knowledge to robotic control

    Brianna Zitkovich et al. RT-2: Vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning (CoRL), 2023

    2023

  52. [60]

    Octo: An open-source generalist robot policy

    Octo Model Team. Octo: An open-source generalist robot policy. In Robotics: Science and Systems (RSS), 2024

    2024

  53. [61]

    OpenVLA: An open-source vision-language-action model

    Moo Jin Kim et al. OpenVLA: An open-source vision-language-action model. In Conference on Robot Learning (CoRL), 2024

    2024

  54. [62]

    Open X-Embodiment: Robotic learning datasets and RT-X models

    Open X-Embodiment Collaboration, Abby O’Neill, Abdul Rehman, Abhinav Gupta, ..., Zipeng Lin, and Zubair Irshad. Open X-Embodiment: Robotic learning datasets and RT-X models. In 2024 IEEE International Conference on Robotics and Automation (ICRA), 2024

    2024

  55. [63]

    RT-H: Action hierarchies using language

    Suneel Belkhale, Tianli Ding, Ted Xiao, Pierre Sermanet, Quan Vuong, Jonathan Tompson, Yevgen Chebotar, Debidatta Dwibedi, and Dorsa Sadigh. RT-H: Action hierarchies using language. In Robotics: Science and Systems (RSS), 2024. arXiv:2403.01823

    2024 arXiv

  56. [64]

    Eo-1: Interleaved vision-text- action pretraining for general robot control

    Delin Qu, Haoming Song, Qizhi Chen, Zhaoqing Chen, Xianqiang Gao, Xinyi Ye, Qi Lv, Modi Shi, Guanghui Ren, Cheng Ruan, Maoqing Yao, Haoran Yang, Jiacheng Bao, Bin Zhao, and Dong Wang. Eo-1: Interleaved vision-text- action pretraining for general robot control. arXiv preprint, ...

    2025

  57. [65]

    CogACT: A foundational vision-language-action model for synergizing cognition and action in robotic manipulation

    Qixiu Li et al. CogACT: A foundational vision-language-action model for synergizing cognition and action in robotic manipulation. arXiv preprint arXiv:2411.19650, 2024

    2024 arXiv

  58. [66]

    Gr-2: A generative video-language-action model with web-scale knowledge for robot manipulation

    Chi-Lam Cheang, Guangzeng Chen, Ya Jing, Tao Kong, Hang Li, Yifeng Li, Yuxiao Liu, Hongtao Wu, Jiafeng Xu, Yichu Yang, et al. Gr-2: A generative video-language-action model with web-scale knowledge for robot manipulation. arXiv preprint arXiv:2410.06158, 2024

    2024 arXiv

  59. [67]

    GR-3 technical report

    Chilam Cheang et al. GR-3 technical report. arXiv preprint arXiv:2507.15493, 2025

    2025 arXiv

  60. [68]

    Gaze-regularized vision-language-action models for robotic manipulation

    Anupam Pani et al. Gaze-regularized vision-language-action models for robotic manipulation. arXiv preprint arXiv:2603.23202, 2026

    2026 arXiv

  61. [69]

    Vla-jepa: Enhancing vision-language-action model with latent world model, 2026

    Jingwen Sun, Wenyao Zhang, Zekun Qi, Shaojie Ren, Zezhi Liu, Hanxin Zhu, Guangzhong Sun, Xin Jin, and Zhibo Chen. Vla-jepa: Enhancing vision-language-action model with latent world model, 2026. URL https://arxiv.org/abs/2602.10098

    2026

  62. [70]

    Cosmos policy: Fine-tuning video models for visuomotor control and planning, 2026

    Moo Jin Kim, Yihuai Gao, Tsung-Yi Lin, Yen-Chen Lin, Yunhao Ge, Grace Lam, Percy Liang, Shuran Song, Ming-Yu Liu, Chelsea Finn, and Jinwei Gu. Cosmos policy: Fine-tuning video models for visuomotor control and planning, 2026. URLhttps://arxiv.org/abs/2601.16163

    2026 arXiv

  63. [71]

    Gigaworld-policy: An efficient action-centered world–action model

    Angen Ye, Boyuan Wang, Chaojun Ni, Guan Huang, Guosheng Zhao, Hao Li, Hengtao Li, Jie Li, Jindi Lv, Jingyu Liu, et al. Gigaworld-policy: An efficient action-centered world–action model. arXiv preprint arXiv:2603.17240, 2026. 27

    2026

  64. [72]

    Unified 4d world action modeling from video priors with asynchronous denoising

    Jun Guo, Qiwei Li, Peiyan Li, Zilong Chen, Nan Sun, Yifei Su, Heyun Wang, Yuan Zhang, Xinghang Li, and Huaping Liu. Unified 4d world action modeling from video priors with asynchronous denoising. arXiv preprint arXiv:2604.26694, 2026

    2026 arXiv

  65. [73]

    Fast-wam: Do world action models need test-time future imagination?, 2026

    Tianyuan Yuan, Zibin Dong, Yicheng Liu, and Hang Zhao. Fast-wam: Do world action models need test-time future imagination?, 2026. URLhttps://arxiv.org/abs/2603.16666

    2026 arXiv

  66. [74]

    Dreamvla: a vision-language-action model dreamed with comprehensive world knowledge

    Wenyao Zhang, Hongsi Liu, Zekun Qi, Yunnan Wang, Xinqiang Yu, Jiazhao Zhang, Runpei Dong, Jiawei He, He Wang, Zhizheng Zhang, et al. Dreamvla: a vision-language-action model dreamed with comprehensive world knowledge. Advances in Neural Information Processing Systems, 38:24195...

    2026

  67. [75]

    Being-h0

    Hao Luo, Wanpeng Zhang, Yicheng Feng, Sipeng Zheng, Haiweng Xu, Chaoyi Xu, Ziheng Xi, Yuhui Fu, and Zongqing Lu. Being-h0. 7: A latent world-action model from egocentric videos. arXiv preprint arXiv:2605.00078, 2026

    2026 arXiv

  68. [76]

    Rynnvla-002: A unified vision-language-action and world model

    Jun Cen, Siteng Huang, Yuqian Yuan, Kehan Li, Hangjie Yuan, Chaohui Yu, Yuming Jiang, Jiayan Guo, Xin Li, Hao Luo, et al. Rynnvla-002: A unified vision-language-action and world model. arXiv preprint arXiv:2511.17502, 2025

    2025 arXiv

  69. [77]

    Abot-m0.5: Unified mobility-and-manipulation world action model, 2026

    Ronghan Chen, Yandan Yang, Zuojin Tang, Dongjie Huo, Tong Lin, Haoning Wu, Haoyun Liu, Yuzhi Chen, Lulu Zheng, Botai Yuan, Tianlun Li, Mingxin Wang, Dekang Qi, Bin Hu, Wei Mei, Yuze Xuan, Haolong Yang, Yanqing Zhu, Mu Xu, Zhiheng Ma, and Xinyuan Chang. Abot-m0.5: Unified mobil...

    2026 arXiv

  70. [78]

    Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments

    Peter Anderson, Qi Wu, Damien Teney, Jake Bruce, Mark Johnson, Niko Sunderhauf, Ian Reid, Stephen Gould, and Anton van den Hengel. Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments. In Proceedings of the IEEE Conference...

    2018

  71. [79]

    Think global, act local: Dual-scale graph transformer for vision-and-language navigation

    Shizhe Chen, Pierre-Louis Guhur, Makarand Tapaswi, Cordelia Schmid, and Ivan Laptev. Think global, act local: Dual-scale graph transformer for vision-and-language navigation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16537–16547, 2022

    2022

  72. [80]

    Beyond the nav-graph: Vision- and-language navigation in continuous environments

    Jacob Krantz, Erik Wijmans, Arjun Majumdar, Dhruv Batra, and Stefan Lee. Beyond the nav-graph: Vision- and-language navigation in continuous environments. In European Conference on Computer Vision, 2020

    2020

  73. [81]

    Vision-and-language navigation with foundation models: A survey

    Yue Zhang et al. Vision-and-language navigation with foundation models: A survey. Transactions on Machine Learning Research (TMLR), 2024

    2024

  74. [82]

    Navgpt: Explicit reasoning in vision-and-language navigation with large language models

    Gengze Zhou, Yicong Hong, and Qi Wu. Navgpt: Explicit reasoning in vision-and-language navigation with large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 7641–7649, 2024

    2024

  75. [83]

    Navid: Video-based vlm plans the next step for vision-and-language navigation

    Jiazhao Zhang, Kunyu Wang, Rongtao Xu, Gengze Zhou, Yicong Hong, Xiaomeng Fang, Qi Wu, Zhizheng Zhang, and He Wang. Navid: Video-based vlm plans the next step for vision-and-language navigation. arXiv preprint arXiv:2402.15852, 2024

    2024 arXiv

  76. [84]

    Navila: Legged robot vision-language-action model for navigation

    An-Chieh Cheng, Yandong Ji, Zhaojing Yang, Zaitian Gongye, Xueyan Zou, Jan Kautz, Erdem Bıyık, Hongxu Yin, Sifei Liu, and Xiaolong Wang. Navila: Legged robot vision-language-action model for navigation. arXiv preprint arXiv:2412.04453, 2024

    2024 arXiv

  77. [85]

    Uni-NaVid: A video-based vision-language-action model for unifying embodied navigation tasks

    Jiazhao Zhang et al. Uni-NaVid: A video-based vision-language-action model for unifying embodied navigation tasks. arXiv preprint arXiv:2412.06224, 2024

    2024 arXiv

  78. [86]

    Streamvln: Streaming vision-and-language navigation via slowfast context modeling

    Meng Wei, Chenyang Wan, Xiqian Yu, Tai Wang, Yuqiang Yang, Xiaohan Mao, Chenming Zhu, Wenzhe Cai, Hanqing Wang, Yilun Chen, Xihui Liu, and Jiangmiao Pang. Streamvln: Streaming vision-and-language navigation via slowfast context modeling. arXiv preprint arXiv:2507.05240, 2025

    2025 arXiv

  79. [87]

    Octonav: Towards generalist embodied navigation

    Chen Gao, Liankai Jin, Xingyu Peng, Jiazhao Zhang, Yue Deng, Annan Li, He Wang, and Si Liu. Octonav: Towards generalist embodied navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 40074–40084, 2026

    2026

  80. [88]

    Agentvln: Towards agentic vision-and-language navigation

    Zihao Xin, Wentong Li, Yixuan Jiang, Ziyuan Huang, Bin Wang, Piji Li, Jianke Zhu, Jie Qin, and Shengjun Huang. Agentvln: Towards agentic vision-and-language navigation. arXiv preprint arXiv:2603.17670, 2026. 28

    2026

  81. [89]

    Learning goal-oriented language-guided navigation with self-improving demonstrations at scale

    Songze Li, Zun Wang, Gengze Zhou, Jialu Li, Xiangyu Zeng, Limin Wang, Yu Qiao, Qi Wu, Mohit Bansal, and Yi Wang. Learning goal-oriented language-guided navigation with self-improving demonstrations at scale. arXiv preprint arXiv:2509.24910, 2025

    2025

  82. [90]

    Endowing embodied agents with spatial reasoning capabilities for vision-and-language navigation

    Qianqian Bai, Zhongpu Chen, Ling Luo, Huaming Du, Yuqian Lei, and Ziyun Jiao. Endowing embodied agents with spatial reasoning capabilities for vision-and-language navigation. arXiv preprint arXiv:2504.08806, 2025

    2025

  83. [91]

    Modernising reinforcement learning-based navigation for embodied semantic scene graph generation

    Roman K¨ uble et al. Modernising reinforcement learning-based navigation for embodied semantic scene graph generation. arXiv preprint arXiv:2603.25415, 2026

    2026 arXiv

  84. [92]

    Sontakke, Jesse Zhang, S´ ebastien M

    Sumedh A. Sontakke, Jesse Zhang, S´ ebastien M. R. Arnold, Karl Pertsch, Erdem Biyik, Dorsa Sadigh, Chelsea Finn, and Laurent Itti. RoboCLIP: One demonstration is enough to learn robot policies. In Advances in Neural Information Processing Systems, 2023

    2023

  85. [93]

    VIP: Towards universal visual reward and representation via value-implicit pre-training

    Yecheng Jason Ma, Shagun Sodhani, Dinesh Jayaraman, Osbert Bastani, Vikash Kumar, and Amy Zhang. VIP: Towards universal visual reward and representation via value-implicit pre-training. In International Conference on Learning Representations, 2023

    2023

  86. [94]

    Vision language models are in-context value learners

    Yecheng Jason Ma, Joey Hejna, Ayzaan Wahid, Chuyuan Fu, Dhruv Shah, Jacky Liang, Zhuo Xu, Sean Kirmani, Peng Xu, Danny Driess, Ted Xiao, Jonathan Tompson, Osbert Bastani, Dinesh Jayaraman, Wenhao Yu, Tingnan Zhang, Dorsa Sadigh, and Fei Xia. Vision language models are in-conte...

    2025

  87. [95]

    Vision-language models are zero-shot reward models for reinforcement learning

    Juan Rocamonde, Victoriano Montesinos, Elvis Nava, Ethan Perez, and David Lindner. Vision-language models are zero-shot reward models for reinforcement learning. In International Conference on Learning Representations, 2024

    2024

  88. [96]

    A survey on explainable reinforcement learning: Concepts, algorithms, challenges

    Yunpeng Qing, Shunyu Liu, Jie Song, Yang Zhou, Kaixuan Chen, Huiqiong Wang, and Mingli Song. A survey on explainable reinforcement learning: Concepts, algorithms, challenges. arXiv preprint arXiv:2211.06665, 2022

    2022 arXiv

  89. [97]

    LIV: Language-image representations and rewards for robotic control

    Yecheng Jason Ma, Vikash Kumar, Amy Zhang, Osbert Bastani, and Dinesh Jayaraman. LIV: Language-image representations and rewards for robotic control. In Proceedings of the 40th International Conference on Machine Learning, pages 23301–23320, 2023

    2023

  90. [98]

    Rank2Reward: Learning shaped reward functions from passive video

    Daniel Yang, Davin Tjia, Jacob Berg, Dima Damen, Pulkit Agrawal, and Abhishek Gupta. Rank2Reward: Learning shaped reward functions from passive video. In Proceedings of the 2024 IEEE International Conference on Robotics and Automation, 2024

    2024

  91. [99]

    Lim, Jesse Thomason, Erdem Biyik, and Jesse Zhang

    Jiahui Zhang, Yusen Luo, Abrar Anwar, Sumedh Anand Sontakke, Joseph J. Lim, Jesse Thomason, Erdem Biyik, and Jesse Zhang. ReWiND: Language-guided rewards teach robot policies without new demonstrations. In Proceedings of The 9th Conference on Robot Learning, pages 460–488, 2025

    2025

  92. [100]

    SARM: Stage-aware reward modeling for long horizon robot manipulation

    Qianzhong Chen, Justin Yu, Mac Schwager, Pieter Abbeel, Fred Shentu, and Philipp Wu. SARM: Stage-aware reward modeling for long horizon robot manipulation. In International Conference on Learning Representations, 2026

    2026

  93. [101]

    Ratliff, Jiafei Duan, Dieter Fox, and Ranjay Krishna

    Shirui Chen, Cole Harrison, Ying-Chun Lee, Angela Jin Yang, Zhongzheng Ren, Lillian J. Ratliff, Jiafei Duan, Dieter Fox, and Ranjay Krishna. TOPReward: Token probabilities as hidden zero-shot rewards for robotics. arXiv preprint arXiv:2602.19313, 2026

    2026

  94. [102]

    A vision-language-action-critic model for robotic real-world reinforcement learning

    Shaopeng Zhai, Qi Zhang, Tianyi Zhang, Fuxian Huang, Haoran Zhang, Ming Zhou, Shengzhe Zhang, Litao Liu, Sixu Lin, and Jiangmiao Pang. A vision-language-action-critic model for robotic real-world reinforcement learning. arXiv preprint arXiv:2509.15937, 2025

    2025

  95. [103]

    Robo-Dopamine: General process reward modeling for high-precision robotic manipulation

    Huajie Tan, Sixiang Chen, Yijie Xu, Zixiao Wang, Yuheng Ji, Cheng Chi, Yaoxu Lyu, Zhongxia Zhao, Xiansheng Chen, Peterson Co, Shaoxuan Xie, Guocai Yao, Pengwei Wang, Zhongyuan Wang, and Shanghang Zhang. Robo-Dopamine: General process reward modeling for high-precision robotic ...

    2026

  96. [104]

    Reflexion: Language agents with verbal reinforcement learning

    Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems (NeurIPS),

  97. [105]

    Agent-R: Training language model agents to reflect via iterative self-training

    Siyu Yuan, Zehui Chen, Zhiheng Xi, Junjie Ye, Zhengyin Du, and Jiecao Chen. Agent-R: Training language model agents to reflect via iterative self-training. arXiv preprint arXiv:2501.11425, 2025. 29

    2025 arXiv

  98. [106]

    Skillopt: Executive strategy for self-evolving agent skills, 2026

    Yifan Yang, Ziyang Gong, Weiquan Huang, Qihao Yang, Ziwei Zhou, Zisu Huang, Yan Li, Xuemei Gao, Qi Dai, Bei Liu, Kai Qiu, Yuqing Yang, Dongdong Chen, Xue Yang, and Chong Luo. Skillopt: Executive strategy for self-evolving agent skills, 2026. URLhttps://arxiv.org/abs/2605.23904

    2026 arXiv

  99. [107]

    From raw experience to skill consumption: A systematic study of model-generated agent skills, 2026

    Zisu Huang, Jingwen Xu, Yifan Yang, Ziyang Gong, Qihao Yang, Muzhao Tian, Xiaohua Wang, Changze Lv, Xuemei Gao, Qi Dai, Bei Liu, Kai Qiu, Xue Yang, Dongdong Chen, Xiaoqing Zheng, and Chong Luo. From raw experience to skill consumption: A systematic study of model-generated age...

    2026 arXiv

  100. [108]

    Continual harness: Online adaptation for self-improving foundation agents, 2026

    Seth Karten, Joel Zhang, Tersoo Upaa Jr, Ruirong Feng, Wenzhe Li, Chengshuai Shi, Chi Jin, and Kiran Vodrahalli. Continual harness: Online adaptation for self-improving foundation agents, 2026. URL https: //arxiv.org/abs/2605.09998

    2026 arXiv

  101. [109]

    Tptu-v2: Boosting task planning and tool usage of large language model-based agents in real-world industry systems

    Yilun Kong, Jingqing Ruan, Yihong Chen, Bin Zhang, Tianpeng Bao, Shi Shiwei, Xiaoru Hu, Hangyu Mao, Ziyue Li, Xingyu Zeng, et al. Tptu-v2: Boosting task planning and tool usage of large language model-based agents in real-world industry systems. In Proceedings of the 2024 conf...

    2024

  102. [110]

    Qpo: Query-dependent prompt optimization via multi-loop offline reinforcement learning

    Yilun Kong, Hangyu Mao, Qi Zhao, Bin Zhang, Jingqing Ruan, Li Shen, Yongzhe Chang, Xueqian Wang, Rui Zhao, and Dacheng Tao. Qpo: Query-dependent prompt optimization via multi-loop offline reinforcement learning. arXiv preprint arXiv:2408.10504, 2024

    2024 arXiv

  103. [111]

    Geometrically-constrained agent for spatial reasoning, 2025

    Zeren Chen, Xiaoya Lu, Zhijie Zheng, Pengrui Li, Lehan He, Yijin Zhou, Jing Shao, Bohan Zhuang, and Lu Sheng. Geometrically-constrained agent for spatial reasoning, 2025. URLhttps://arxiv.org/abs/2511.22659

    2025

  104. [112]

    Robotic visual instruction

    Yanbang Li, Ziyang Gong, Haoyang Li, Xiaoqi Huang, Haolan Kang, Guangping Bai, and Xianzheng Ma. Robotic visual instruction. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 12155–12165, 2025

    2025

  105. [113]

    Cap-x: A framework for benchmarking and improving coding agents for robot manipulation

    Max Fu, Justin Yu, Karim El-Refai, Ethan Kou, Haoru Xue, Huang Huang, Wenli Xiao, Guanzhi Wang, Li Fei-Fei, Guanya Shi, Jiajun Wu, Shankar Sastry, Yuke Zhu, Ken Goldberg, and Linxi Fan. Cap-x: A framework for benchmarking and improving coding agents for robot manipulation. arX...

    2026 arXiv

  106. [114]

    Self-improving embodied foundation models

    Seyed Kamyar Seyed Ghasemipour, Ayzaan Wahid, Jonathan Tompson, Pannag Sanketi, and Igor Mordatch. Self-improving embodied foundation models. In Advances in Neural Information Processing Systems (NeurIPS),

  107. [115]

    SEEA-R1: Tree-structured reinforcement fine-tuning for self-evolving embodied agents

    Wanxin Tian, Shijie Zhang, Kevin Zhang, Xiaowei Chi, Yulin Luo, Junyu Lu, Chunkai Fan, Qiang Zhou, Yiming Zhao, Siyu Lin, Zhiyuan Qin, Xiaozhu Ju, Shanghang Zhang, and Jian Tang. SEEA-R1: Tree-structured reinforcement fine-tuning for self-evolving embodied agents. In Advances ...

    2025

  108. [116]

    RISE: Self-improving robot policy with compositional world model

    Jiazhi Yang, Kunyang Lin, Jinwei Li, Wencong Zhang, Tianwei Lin, Longyan Wu, Zhizhong Su, Hao Zhao, Ya-Qin Zhang, Li Chen, Ping Luo, Xiangyu Yue, and Hongyang Li. RISE: Self-improving robot policy with compositional world model. arXiv preprint arXiv:2602.11075, 2026

    2026 arXiv

  109. [117]

    Reflection-based task adaptation for self-improving VLA

    Baicheng Li et al. Reflection-based task adaptation for self-improving VLA. arXiv preprint arXiv:2510.12710, 2025

    2025 arXiv

  110. [118]

    A2po: Towards effective offline reinforcement learning from an advantage-aware perspective

    Yunpeng Qing, Shunyu Liu, Jingyuan Cong, Kaixuan Chen, Yihe Zhou, and Mingli Song. A2po: Towards effective offline reinforcement learning from an advantage-aware perspective. Advances in Neural Information Processing Systems, 37:29064–29090, 2024

    2024

  111. [119]

    Bitrajdiff: Bidirectional trajectory generation with diffusion models for offline reinforcement learning

    Yunpeng Qing, Yixiao Chi, Shuo Chen, Shunyu Liu, Kelu Yao, Sixu Lin, Litao Liu, and Changqing Zou. Bitrajdiff: Bidirectional trajectory generation with diffusion models for offline reinforcement learning. arXiv preprint arXiv:2506.05762, 2025

    2025 arXiv

  112. [120]

    Curricular subgoals for inverse reinforcement learning

    Shunyu Liu, Yunpeng Qing, Shuqi Xu, Hongyan Wu, Jiangtao Zhang, Jingyuan Cong, Tianhao Chen, Yun-Fu Liu, and Mingli Song. Curricular subgoals for inverse reinforcement learning. IEEE Transactions on Intelligent Transportation Systems, 26(3):3016–3027, 2025

    2025

  113. [121]

    Mastering massive multi-task reinforcement learning via mixture-of-expert decision transformer

    Yilun Kong, Guozheng Ma, Qi Zhao, Haoyu Wang, Li Shen, Xueqian Wang, and Dacheng Tao. Mastering massive multi-task reinforcement learning via mixture-of-expert decision transformer. arXiv preprint arXiv:2505.24378, 2025. 30

    2025 arXiv

  114. [122]

    Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. In International Conference on Learning Representations (ICLR), 2023

    2023

  115. [123]

    Vo, Maximilian Seitzer, et al

    Oriane Sim´ eoni, Huy V. Vo, Maximilian Seitzer, et al. Dinov3. arXiv preprint arXiv:2508.10104, 2025

    2025 arXiv

  116. [124]

    Efficient and effective weight-ensembling mixture of experts for multi-task model merging

    Li Shen, Anke Tang, Enneng Yang, Guibing Guo, Yong Luo, Lefei Zhang, Xiaochun Cao, Bo Du, and Dacheng Tao. Efficient and effective weight-ensembling mixture of experts for multi-task model merging. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025

    2025

  117. [125]

    Fusionbench: A unified library and comprehensive benchmark for deep model fusion

    Anke Tang, Li Shen, Yong Luo, Enneng Yang, Han Hu, Lefei Zhang, Bo Du, and Dacheng Tao. Fusionbench: A unified library and comprehensive benchmark for deep model fusion. arXiv preprint arXiv:2406.03280, 2025

    2025

  118. [126]

    Sparse upcycling: Training mixture-of-experts from dense checkpoints

    Aran Komatsuzaki, Joan Puigcerver, James Lee-Thorp, Carlos Riquelme Ruiz, Basil Mustafa, Joshua Ainslie, Yi Tay, Mostafa Dehghani, and Neil Houlsby. Sparse upcycling: Training mixture-of-experts from dense checkpoints. arXiv preprint arXiv:2212.05055, 2022

    2022 arXiv

  119. [127]

    Thinking in space: How multimodal large language models see, remember, and recall spaces

    Jihan Yang, Shusheng Yang, Anjali W Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. Thinking in space: How multimodal large language models see, remember, and recall spaces. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 10632–10643, 2025

    2025

  120. [128]

    Mmsi-bench: A benchmark for multi-image spatial intelligence

    Sihan Yang, Runsen Xu, Yiman Xie, Sizhe Yang, Mo Li, Jingli Lin, Chenming Zhu, Xiaochen Chen, Haodong Duan, Xiangyu Yue, et al. Mmsi-bench: A benchmark for multi-image spatial intelligence. arXiv preprint arXiv:2505.23764, 2025

    2025 arXiv

  121. [129]

    Mindcube: Spatial mental modeling from limited views, 2026

    Qineng Wang, Baiqiao Yin, Pingyue Zhang, Jianshu Zhang, Kangrui Wang, Zihan Wang, Jieyu Zhang, Keshigeyan Chandrasegaran, Han Liu, Ranjay Krishna, Saining Xie, Jiajun Wu, Li Fei-Fei, and Manling Li. Mindcube: Spatial mental modeling from limited views, 2026. URLhttps://arxiv.o...

    2026

  122. [130]

    Scanqa: 3d question answering for spatial scene understanding, 2022

    Daichi Azuma, Taiki Miyanishi, Shuhei Kurita, and Motoaki Kawanabe. Scanqa: 3d question answering for spatial scene understanding, 2022. URLhttps://arxiv.org/abs/2112.10482

    2022 arXiv

  123. [131]

    Sqa3d: Situated question answering in 3d scenes, 2023

    Xiaojian Ma, Silong Yong, Zilong Zheng, Qing Li, Yitao Liang, Song-Chun Zhu, and Siyuan Huang. Sqa3d: Situated question answering in 3d scenes, 2023. URLhttps://arxiv.org/abs/2210.07474

    2023 arXiv

  124. [132]

    Dave Zhenyu Chen, Ali Gholami, Matthias Nießner, and Angel X. Chang. Scan2cap: Context-aware dense captioning in rgb-d scans, 2020. URLhttps://arxiv.org/abs/2012.02206

    2020 arXiv

  125. [133]

    Chang, and Matthias Nießner

    Dave Zhenyu Chen, Angel X. Chang, and Matthias Nießner. Scanrefer: 3d object localization in rgb-d scans using natural language, 2020. URLhttps://arxiv.org/abs/1912.08830

    2020 arXiv

  126. [134]

    Multi3drefer: Grounding text description to multiple 3d objects

    Yiming Zhang, ZeMing Gong, and Angel X Chang. Multi3drefer: Grounding text description to multiple 3d objects. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15225–15236, 2023

    2023

  127. [135]

    Embspatial-bench: Benchmarking spatial understanding for embodied tasks with large vision-language models

    Mengfei Du, Binhao Wu, Zejun Li, Xuan-Jing Huang, and Zhongyu Wei. Embspatial-bench: Benchmarking spatial understanding for embodied tasks with large vision-language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: S...

    2024

  128. [136]

    Gemini robotics: Bringing ai into the physical world

    Gemini Robotics Team, Saminda Abeyruwan, Joshua Ainslie, Jean-Baptiste Alayrac, Montserrat Gonzalez Arenas, Travis Armstrong, Ashwin Balakrishna, Robert Baruch, Maria Bauza, Michiel Blokzijl, et al. Gemini robotics: Bringing ai into the physical world. arXiv preprint arXiv:250...

    2025 arXiv

  129. [137]

    Robospatial: Teaching spatial understanding to 2d and 3d vision-language models for robotics

    Chan Hee Song, Valts Blukis, Jonathan Tremblay, Stephen Tyree, Yu Su, and Stan Birchfield. Robospatial: Teaching spatial understanding to 2d and 3d vision-language models for robotics. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 15768–15780, 2025

    2025

  130. [138]

    Smith, Fei Xia, Dieter Fox, and Ranjay Krishna

    Long Cheng, Jiafei Duan, Yi Ru Wang, Haoquan Fang, Boyang Li, Yushan Huang, Elvis Wang, Ainaz Eftekhar, Jason Lee, Wentao Yuan, Rose Hendrix, Noah A. Smith, Fei Xia, Dieter Fox, and Ranjay Krishna. Pointarena: Probing multimodal grounding through language-guided pointing, 2025...

    2025

  131. [139]

    Roboafford: A dataset and benchmark for enhancing object and spatial affordance learning in robot manipulation

    Yingbo Tang, Lingfeng Zhang, Shuyi Zhang, Yinuo Zhao, and Xiaoshuai Hao. Roboafford: A dataset and benchmark for enhancing object and spatial affordance learning in robot manipulation. In Proceedings of the 33rd ACM International Conference on Multimedia, pages 12706–12713, 2025. 31

    2025

  132. [140]

    Evaluating real-world robot manipulation policies in simulation

    Xuanlin Li, Kyle Hsu, Jiayuan Gu, Karl Pertsch, Oier Mees, Homer Rich Walke, Chuyuan Fu, Ishikaa Lunawat, Isabel Sieh, Sean Kirmani, Sergey Levine, Jiajun Wu, Chelsea Finn, Hao Su, Quan Vuong, and Ted Xiao. Evaluating real-world robot manipulation policies in simulation. arXiv...

    2024 arXiv

  133. [141]

    Libero: Benchmarking knowledge transfer for lifelong robot learning

    Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. Libero: Benchmarking knowledge transfer for lifelong robot learning. In Advances in Neural Information Processing Systems (NeurIPS), 2023

    2023

  134. [142]

    Raw2drive: Reinforcement learning with aligned world models for end-to-end autonomous driving (in CARLA v2)

    Zhenjie Yang, Xiaosong Jia, Qifeng Li, Xue Yang, Maoqing Yao, and Junchi Yan. Raw2drive: Reinforcement learning with aligned world models for end-to-end autonomous driving (in CARLA v2). In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2026. URL ...

    2026

  135. [143]

    Llm4drive: A survey of large language models for autonomous driving, 2023

    Zhenjie Yang, Xiaosong Jia, Hongyang Li, and Junchi Yan. Llm4drive: A survey of large language models for autonomous driving, 2023

    2023

  136. [144]

    Drivemoe: Mixture-of-experts for vision-language-action model in end-to-end autonomous driving

    Zhenjie Yang, Yilin Chai, Xiaosong Jia, Qifeng Li, Yuqian Shao, Xuekai Zhu, Haisheng Su, and Junchi Yan. Drivemoe: Mixture-of-experts for vision-language-action model in end-to-end autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Rec...

    2026

  137. [145]

    Drivegpt4: Interpretable end-to-end autonomous driving via large language model

    Zhenhua Xu, Yujia Zhang, Enze Xie, Zhen Zhao, Yong Guo, Kwan-Yee K Wong, Zhenguo Li, and Hengshuang Zhao. Drivegpt4: Interpretable end-to-end autonomous driving via large language model. IEEE Robotics and Automation Letters, 2024

    2024

  138. [146]

    Drivelm: Driving with graph visual question answering

    Chonghao Sima, Katrin Renz, Kashyap Chitta, Li Chen, Hanxue Zhang, Chengen Xie, Jens Beißwenger, Ping Luo, Andreas Geiger, and Hongyang Li. Drivelm: Driving with graph visual question answering. In European conference on computer vision, pages 256–274. Springer, 2024

    2024

  139. [147]

    Bridgedata v2: A dataset for robot learning at scale

    Homer Rich Walke, Kevin Black, Tony Z Zhao, Quan Vuong, Chongyi Zheng, Philippe Hansen-Estruch, Andre Wang He, Vivek Myers, Moo Jin Kim, Max Du, et al. Bridgedata v2: A dataset for robot learning at scale. In Conference on Robot Learning (CoRL), 2023

    2023

  140. [148]

    Fine-tuning vision-language-action models: Optimizing speed and success

    Moo Jin Kim, Chelsea Finn, and Percy Liang. Fine-tuning vision-language-action models: Optimizing speed and success. arXiv preprint arXiv:2502.19645, 2025

    2025 arXiv

  141. [149]

    Starvla: A lego-like codebase for vision-language-action model developing

    StarVLA Community. Starvla: A lego-like codebase for vision-language-action model developing. arXiv preprint arXiv:2604.05014, 2026

    2026 arXiv

  142. [150]

    Mantis: A versatile vision-language-action model with disentangled visual foresight

    Yi Yang, Xueqi Li, Yiyang Chen, Jin Song, Yihan Wang, Zipeng Xiao, Jiadi Su, You Qiaoben, Pengfei Liu, and Zhijie Deng. Mantis: A versatile vision-language-action model with disentangled visual foresight. arXiv preprint arXiv:2511.16175, 2025

    2025

  143. [151]

    Memoryvla: Perceptual-cognitive memory in vision-language-action models for robotic manipulation, 2026

    Hao Shi, Bin Xie, Yingfei Liu, Lin Sun, Fengrong Liu, Tiancai Wang, Erjin Zhou, Haoqiang Fan, Xiangyu Zhang, and Gao Huang. Memoryvla: Perceptual-cognitive memory in vision-language-action models for robotic manipulation, 2026. URLhttps://arxiv.org/abs/2508.19236

    2026 arXiv

  144. [152]

    Vita-vla: Efficiently teaching vision-language models to act via action expert distillation, 2025

    Shaoqi Dong, Chaoyou Fu, Haihan Gao, Yi-Fan Zhang, Chi Yan, Chu Wu, Xiaoyu Liu, Yunhang Shen, Jing Huo, Deqiang Jiang, Haoyu Cao, Yang Gao, Xing Sun, Ran He, and Caifeng Shan. Vita-vla: Efficiently teaching vision-language models to act via action expert distillation, 2025. UR...

    2025

  145. [153]

    Guide, think, act: Interactive embodied reasoning in vision-language-action models

    Yiran Ling, Qing Lian, Jinghang Li, Qing Jiang, Tianming Zhang, Xiaoke Jiang, Chuanxiu Liu, Jie Liu, and Lei Zhang. Guide, think, act: Interactive embodied reasoning in vision-language-action models. arXiv preprint arXiv:2605.13632, 2026

    2026 arXiv

  146. [154]

    X-vla: Soft-prompted transformer as scalable cross-embodiment vision-language- action model

    Jinliang Zheng, Jianxiong Li, Zhihao Wang, Dongxiu Liu, Xirui Kang, Yuchun Feng, Yinan Zheng, Jiayin Zou, Yilun Chen, Jia Zeng, et al. X-vla: Soft-prompted transformer as scalable cross-embodiment vision-language- action model. arXiv preprint arXiv:2510.10274, 2025

    2025 arXiv

  147. [155]

    Thinkact: Vision- language-action reasoning via reinforced visual latent planning

    Chi-Pin Huang, Yueh-Hua Wu, Min-Hung Chen, Yu-Chiang Frank Wang, and Fu-En Yang. Thinkact: Vision- language-action reasoning via reinforced visual latent planning. arXiv preprint arXiv:2507.16815, 2025

    2025 arXiv

  148. [156]

    Unified vision-language-action model

    Yuqi Wang, Xinghang Li, Wenxuan Wang, Junbo Zhang, Yingyan Li, Yuntao Chen, Xinlong Wang, and Zhaoxiang Zhang. Unified vision-language-action model. arXiv preprint arXiv:2506.19850, 2025. 32

    2025 arXiv

  149. [157]

    Vlm-3r: Vision-language models augmented with instruction-aligned 3d reconstruction

    Zhiwen Fan, Jian Zhang, Renjie Li, Junge Zhang, Runjin Chen, Hezhen Hu, Kevin Wang, Huaizhi Qu, Dilin Wang, Zhicheng Yan, et al. Vlm-3r: Vision-language models augmented with instruction-aligned 3d reconstruction. arXiv preprint arXiv:2505.20279, 2025

    2025 arXiv

  150. [158]

    Gpt4scene: Understand 3d scenes from videos with vision-language models

    Zhangyang Qi, Zhixiong Zhang, Ye Fang, Jiaqi Wang, and Hengshuang Zhao. Gpt4scene: Understand 3d scenes from videos with vision-language models. arXiv preprint arXiv:2501.01428, 2025

    2025 arXiv

  151. [159]

    Smith, Hannaneh Hajishirzi, Ross Girshick, Ali Farhadi, and Aniruddha Kembhavi

    Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi, Yue Yang, Jae Sung Park, Mohammadreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, Jiasen Lu, Taira Anderson, Erin Bransom, Kiana Ehsani, Huong Ngo, YenSung Chen, Ajay Patel, Mark Yatskar, Chris Callison-Burch,...

    2024 arXiv

  152. [160]

    Room-across-room: Multilingual vision-and-language navigation with dense spatiotemporal grounding

    Alexander Ku, Peter Anderson, Roma Patel, Eugene Ie, and Jason Baldridge. Room-across-room: Multilingual vision-and-language navigation with dense spatiotemporal grounding. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2020

    2020

  153. [161]

    Learning to navigate unseen environments: Back translation with environmental dropout

    Hao Tan, Licheng Yu, and Mohit Bansal. Learning to navigate unseen environments: Back translation with environmental dropout. arXiv preprint arXiv:1904.04195, 2019

    1904 arXiv

  154. [162]

    Scaling data generation in vision-and-language navigation

    Zun Wang, Jialu Li, Yicong Hong, Yi Wang, Qi Wu, Mohit Bansal, Stephen Gould, Hao Tan, and Yu Qiao. Scaling data generation in vision-and-language navigation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023

    2023

  155. [163]

    Bootstrapping language-guided navigation learning with self-refining data flywheel

    Zun Wang, Jialu Li, Yicong Hong, Songze Li, Kunchang Li, Shoubin Yu, Yi Wang, Yu Qiao, Yali Wang, Mohit Bansal, and Limin Wang. Bootstrapping language-guided navigation learning with self-refining data flywheel. arXiv preprint arXiv:2412.08467, 2024

    2024 arXiv

  156. [164]

    Maplm: A real-world large-scale vision-language benchmark for map and traffic scene understanding

    Xu Cao, Tong Zhou, Yunsheng Ma, Wenqian Ye, Can Cui, Kun Tang, Zhipeng Cao, Kaizhao Liang, Ziran Wang, James M Rehg, et al. Maplm: A real-world large-scale vision-language benchmark for map and traffic scene understanding. In Proceedings of the IEEE/CVF conference on computer ...

    2024

  157. [165]

    Driveaction: A benchmark for exploring human-like driving decisions in vla models

    Yuhan Hao, Zhengning Li, Lei Sun, Weilong Wang, Naixin Yi, Sheng Song, Caihong Qin, Mofan Zhou, Yifei Zhan, and Xianpeng Lang. Driveaction: A benchmark for exploring human-like driving decisions in vla models. arXiv preprint arXiv:2506.05667, 2025

    2025

  158. [166]

    Lingoqa: Visual question answering for autonomous driving

    Ana-Maria Marcu, Long Chen, Jan H¨ unermann, Alice Karnsund, Benoit Hanotte, Prajwal Chidananda, Saurabh Nair, Vijay Badrinarayanan, Alex Kendall, Jamie Shotton, et al. Lingoqa: Visual question answering for autonomous driving. In European Conference on Computer Vision, pages ...

    2024

  159. [167]

    Nuscenes-qa: A multi-modal visual question answering benchmark for autonomous driving scenario

    Tianwen Qian, Jingjing Chen, Linhai Zhuo, Yang Jiao, and Yu-Gang Jiang. Nuscenes-qa: A multi-modal visual question answering benchmark for autonomous driving scenario. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 4542–4550, 2024

    2024

  160. [168]

    Nuplanqa: A large-scale dataset and benchmark for multi-view driving scene understanding in multi-modal large language models

    Sung-Yeon Park, Can Cui, Yunsheng Ma, Ahmadreza Moradipari, Rohit Gupta, Kyungtae Han, and Ziran Wang. Nuplanqa: A large-scale dataset and benchmark for multi-view driving scene understanding in multi-modal large language models. arXiv preprint arXiv:2503.12772, 2025

    2025 arXiv

  161. [169]

    A CE-Br ain Output: <OBJ040>. G T answ er: ['<OBJ040>. '] Question 2: Identify t he ID of t he object t hat closely r esembles t he description

    Yi-Fan Zhang, Huanyu Zhang, Haochen Tian, Chaoyou Fu, Shuangqing Zhang, Junfei Wu, Feng Li, Kun Wang, Qingsong Wen, Zhang Zhang, et al. Mme-realworld: Could your multimodal llm challenge high-resolution real-world scenarios that are difficult for humans? arXiv preprint arXiv:2...

    2024 arXiv

Pith tools

Reviewed July 11, 2026 · model on record in the stance chip above.