Pith. sign in

REVIEW 3 major objections 2 minor 2 cited by

Thermal cues can fix dark-scene 3D geometry without costing daylight performance if you inject only geometry-consistent, reliability-gated structure into a frozen RGB geometry model.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 18:05 UTC pith:OCMNDEMU

load-bearing objection We only have Embodied-R1.5’s abstract; the cached full text is a different paper (DarkVGGT), so the SOTA/VLA/real-robot claims are currently unauditable. the 3 major comments →

arxiv 2606.11324 v2 pith:OCMNDEMU submitted 2026-06-09 cs.RO cs.AIcs.LG

Embodied-R1.5: Evolving Physical Intelligence via Embodied Foundation Models

Yifu Yuan , Yaoting Huang , Xianze Yao , Yutong Li , Shuoheng Zhang , Linqi Han , Pengyi Li , Jiangeng Sun
show 15 more authors
Wenting Jia Zhao Zhang Yuhao Liu Ruihao Liao Yucheng Hu Qiyu Wu Yuxiao Li Zibin Dong Fei Ni Yan Zheng Shuyang Gu Yi Ma Hongyao Tang Han Hu Jianye Hao
This is my paper
classification cs.RO cs.AIcs.LG
keywords RGB-thermal fusionfeed-forward 3D reconstructionlow-light geometrythermal factorizationgeometry-shared routingdaylight taxvisual geometry transformer
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Feed-forward 3D models that map image streams straight to cameras, depth, and point maps work well in daylight but fail when RGB is dark and textureless. This paper argues that long-wave thermal images can supply the missing geometric evidence—if fusion is conservative and physics-aware rather than a naive second appearance stream. DarkVGGT freezes a pretrained RGB geometry backbone and treats thermal as selective corrective evidence: it factorizes thermal tokens into emissive (geometry-tied) and sparse reflective residuals, then routes only shared geometric structure into the RGB pathway through reliability gates. Thermal dropout and an RGB-prior preservation loss keep the model from over-relying on heat and from eroding its daylight skill. On low-visibility RGB-T benchmarks the method improves depth and pose over both RGB-only and prior RGB-T baselines, while RGB-only scores on well-lit benchmarks stay close to the original backbone. The practical claim is that dark-scene geometry need not force a daylight tax if thermal is used as gated structure, not unconstrained appearance.

Core claim

Dark-scene RGB-T geometry can be cast as conservative multisensor fusion: keep the pretrained RGB geometry pathway as the primary estimator, decompose thermal into emissive-dominant geometry cues versus sparse reflection residuals, and inject only modality-invariant thermal structure through a reliability gate. That design recovers accurate depth and camera pose under low visibility while largely preserving well-lit RGB performance.

What carries the argument

Physics-inspired thermal factorization plus geometry-shared thermal routing (GSTR): per-patch emissive/reflective gates (Kirchhoff-style complementarity, high-pass residual on reflection) and a late shared/private thermal split that stop-gradient-distills geometry-shared cues into RGB with a per-token reliability gate and thermal dropout.

Load-bearing premise

The method assumes moderately aligned paired RGB-thermal frames and that the learned emissive/reflective split plus reliability gate truly isolate geometry-consistent heat cues rather than dataset-specific correlations.

What would settle it

On held-out low-light RGB-T sequences with deliberate spatial misalignment or thermal reflection-dominated scenes, check whether depth AbsRel and pose AUC still beat an RGB-T LoRA baseline and whether RGB-only daylight metrics stay within the paper’s reported gap to the frozen backbone; large drops would falsify the selective-fusion claim.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Low-visibility robots and night driving can use thermal as a corrective channel without retraining or discarding a strong daylight RGB geometry model.
  • LoRA-scale multimodal adapters plus physics-structured gates can add a sensor modality with far fewer trainable parameters than full backbone fine-tuning.
  • Reliability-gated shared/private routing is a reusable pattern for other corrective modalities (events, sparse LiDAR) that should not rewrite the RGB prior.
  • Benchmarking low-light geometry must report paired daylight RGB-only scores; gains that destroy daylight performance are incomplete.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the emissive/reflective factorization is only weakly supervised by edge-disagreement pseudo-labels, stronger physical temperature or material labels might further cut residual daylight tax.
  • The same gated-injection idea could be stress-tested on unpaired or cross-sensor streams by adding an explicit cross-modal correspondence stage the paper leaves for future work.
  • Privacy and regulatory constraints on thermal sensing in public spaces may limit deployment even when geometric performance is strong.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 2 minor

Summary. The submission under review is identified as Embodied-R1.5 (arXiv:2606.11324), an 8B Embodied Foundation Model that unifies embodied cognition, planning, correction, and pointing; claims SOTA on 16/24 embodied VLM benchmarks, VLA transfer past π0.5 on four manipulation suites, and zero-shot real-robot generalization, supported by three automated data pipelines (~15B tokens), multi-task balanced RL, and a Planner-Grounder-Corrector (PGC) loop, with planned open-sourcing of weights, data, code, and EmbodiedEvalKit. The full manuscript text supplied for review is not that paper: it is DarkVGGT (arXiv:2606.11326), an RGB-T feed-forward geometry model that adds physics-inspired thermal factorization and geometry-shared thermal routing (GSTR) to VGGT for low-light depth and pose estimation while limiting daylight degradation. DarkVGGT’s methods, tables (ViViD++/STheReO/Dark3R, ETH3D/ScanNet++), and ablations address a different problem and do not contain Embodied-R1.5’s architecture, data pipelines, RL recipe, PGC design, benchmark tables, or robot protocols.

Significance. If Embodied-R1.5’s abstract claims held under audit—unified 8B EFM SOTA against Gemini-Robotics-ER-1.5 and GPT-5.4, light-data VLA transfer past π0.5, and zero-shot real-robot long-horizon behavior, plus open-sourced EmbodiedEvalKit—they would be a substantial contribution to embodied foundation models and physical intelligence. Those claims cannot be assessed from the provided package. Separately, the attached DarkVGGT manuscript is a competent, well-scoped RGB-T geometry paper with clear physics-motivated modules, LoRA-based adaptation, thermal dropout for prior preservation, and thorough low-light vs. well-lit evaluation; that work is not the paper under the Embodied-R1.5 identifier and is not the subject of this report’s recommendation.

major comments (3)
  1. Identity mismatch: the paper_id/title/abstract are Embodied-R1.5 (cs.RO, 2606.11324), but the full manuscript body is DarkVGGT (cs.CV, 2606.11326). No Embodied-R1.5 architecture, data-construction pipelines, multi-task RL recipe, PGC loop, VLM/VLA tables, contamination controls, or real-robot protocols appear in the supplied text. The central SOTA/VLA/robot claims are therefore unauditable; this is a load-bearing failure of the review package, not a presentation nit.
  2. Abstract of Embodied-R1.5 asserts three automated data pipelines, a 15B-token mixture, multi-task balanced RL, and PGC closed-loop execution as the basis for 16/24 SOTA and zero-shot robot success. Without the corresponding methods sections, dataset composition, leakage checks, ablations, error bars, or robot experimental protocols, those claims cannot be verified or falsified. A revised submission must supply the correct full manuscript for Embodied-R1.5.
  3. Even if only the abstract is treated as the Embodied-R1.5 claim set, free parameters (RL balance weights, data mixture, PGC thresholds, VLA fine-tune volume) and the risk that automated pipelines bake evaluation structure into supervision remain unaddressed in any available technical section. That is process risk for the headline physical-intelligence claim and cannot be resolved from DarkVGGT’s content.
minor comments (2)
  1. Editorial/process: arXiv IDs 2606.11324 vs 2606.11326 and titles Embodied-R1.5 vs DarkVGGT must be reconciled before any scientific review of Embodied-R1.5 can proceed.
  2. For the attached DarkVGGT text only (not the paper under review): Tables 1–5 and §§3.2–3.4 are internally coherent for RGB-T geometry; residual presentation issues (garbled tokens in some table headers/figures, e.g. Table 4 ‘RMSE ���’) would be minor if that paper were under review.

Circularity Check

0 steps flagged

No definitional circularity: Embodied-R1.5 has no derivation chain in the supplied source; the attached full text (DarkVGGT) is empirical fusion with GT-supervised auxiliaries, not self-definitional predictions.

full rationale

The query targets Embodied-R1.5 (2606.11324), but the only full manuscript provided is DarkVGGT (2606.11326)—an RGB-T feed-forward geometry model. Embodied-R1.5 appears solely as an abstract asserting SOTA via automated data pipelines, multi-task RL, and a PGC loop; that abstract contains no equations, fitted constants renamed as predictions, uniqueness theorems, or self-definitional steps. DarkVGGT’s chain (physics-inspired factorization Eqs. 1–4; GSTR Eqs. 5–9; thermal dropout / L_drop; main L_VGGT) uses Kirchhoff-style inductive bias and auxiliary losses (L_sparse, L_edge, L_ortho, L_distill, L_recon) supervised by available GT depth/edges or stop-gradient reconstruction—standard regularizers, not predictions forced by construction. Benchmarks (ViViD++, STheReO, Dark3R, ETH3D, ScanNet++) and ablations report empirical held-out metrics, not tautologies. Possible ML process risks (data/eval design for Embodied-R1.5; pseudo-thermal on Dark3R) are outside the definitional patterns this pass flags. No circular step meets the quote-and-reduce bar.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 4 invented entities

Abstract-only review of Embodied-R1.5. Free parameters and invented modules are those the abstract treats as load-bearing design choices without giving fitted values or independent external validation. Domain assumptions are standard robotics-VLM premises required for the SOTA and real-robot claims to mean what the abstract says.

free parameters (4)
  • multi-task RL balance weights / conflict-alleviation recipe
    Abstract credits a multi-task balanced RL recipe for heterogeneous task conflicts; the mixing weights, reward scales, and sampling ratios are free design parameters that determine reported multi-capability SOTA.
  • scale and composition of the 15B-token data mixture
    Coverage from three automated pipelines is central; token allocation across cognition/planning/correction/pointing is an unstated free mixture that drives benchmark outcomes.
  • PGC loop control thresholds and correction budget
    Closed-loop Planner-Grounder-Corrector behavior depends on when to replan/correct and how many steps are allowed; these knobs are not specified in the abstract but affect long-horizon success.
  • VLA fine-tune data volume and action-head hyperparameters
    Claim of beating π0.5 with only a small amount of data depends on unspecified data size, action representation, and fine-tuning schedule.
axioms (4)
  • domain assumption Automated synthetic/constructed embodied data can substitute for large-scale real interaction data without catastrophic sim-to-real or distribution gap on the reported benchmarks and robots.
    The abstract's data system is the main capability driver; this assumption is required for SOTA and zero-shot robot claims.
  • domain assumption A single 8B shared architecture can jointly host cognition, planning, correction, and pointing without irreducible task interference beyond what multi-task RL can fix.
    Unification claim depends on this capacity/interference assumption.
  • domain assumption Named comparison models (Gemini-Robotics-ER-1.5, GPT-5.4, π0.5) are evaluated under comparable protocols so 'SOTA on 16/24' is meaningful.
    Leaderboard claim requires matched eval; abstract does not document protocol parity.
  • ad hoc to paper Planner-Grounder-Corrector closed-loop execution is a valid operationalization of long-horizon embodied autonomy for the tasks tested.
    PGC is introduced as the paper's control framework; its adequacy is internal to the work.
invented entities (4)
  • Embodied-R1.5 (unified EFM) no independent evidence
    purpose: Single model integrating embodied cognition, planning, correction, and pointing toward general physical intelligence.
    Primary artifact; independent evidence would be public weights and third-party eval, not available in this review packet.
  • Planner-Grounder-Corrector (PGC) framework no independent evidence
    purpose: Enable one model to autonomously execute and self-correct over long-horizon tasks.
    Named closed-loop architecture introduced by the paper; no external falsifiable handle beyond their tasks.
  • Three automated data construction pipelines / 15B-token data system no independent evidence
    purpose: Expand coverage of critical embodied capabilities for training.
    Core resource claim; pipelines are paper-specific constructs without inspectable schemas here.
  • EmbodiedEvalKit no independent evidence
    purpose: Evaluation framework tailored for embodied tasks to facilitate EFM research.
    Promised tooling entity; not inspectable in provided text.

pith-pipeline@v1.1.0-grok45 · 24016 in / 3651 out tokens · 40503 ms · 2026-07-14T18:05:21.411712+00:00 · methodology

0 comments
read the original abstract

We introduce Embodied-R1.5, a unified Embodied Foundation Model (EFM) that integrates comprehensive embodied reasoning capabilities, spanning embodied cognition, task planning, correction, and pointing, within a single architecture toward general physical intelligence. Leveraging three automated data construction pipelines to significantly expand the data coverage of critical capabilities, we build a large-scale data system of over 15B tokens, and design a multi-task balanced RL recipe to alleviate heterogeneous task conflicts. We further introduce a Planner-Grounder-Corrector (PGC) closed-loop framework that enables a single model to autonomously execute and self-correct over long-horizon tasks. With only 8B parameters, Embodied-R1.5 achieves SOTA on 16 out of 24 embodied VLM benchmarks, surpassing leading models like Gemini-Robotics-ER-1.5 and GPT-5.4. Benefiting from the internalized embodied capabilities, Embodied-R1.5 can be fine-tuned into a VLA with only a small amount of data, outperforming leading VLA models like $\pi_{0.5}$ across 4 popular manipulation benchmark suites. We further conduct extensive zero-shot real-robot experiments, validating performance in instruction following, affordance grounding, articulated object manipulation, and long-horizon complex tasks, demonstrating strong generalization to the physical world. We open-source model weights, datasets, training code, and EmbodiedEvalKit, an evaluation framework tailored for embodied tasks, to facilitate future research in EFMs.

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loops

    cs.AI 2026-07 conditional novelty 6.0

    A survey of 1,250 papers organizes AI self-improvement along two axes—what is improved and loop closure—finding that demonstrated self-improvement strength tracks a verification hierarchy from formal verifiers down to...

  2. ACE-Brain-0.5: A Unified Embodied Foundational Model for Physical Agentic AI

    cs.RO 2026-07 conditional novelty 6.0

    A single 8B backbone unifies spatial perception, decision making, navigation/manipulation, and progress estimation with SSR+ merging, reporting gains on most spatial benchmarks and competitive action/progress results.

Reference graph

Works this paper leans on

116 extracted references · 62 linked inside Pith · cited by 2 Pith papers

  1. [1]

    Cosmos-reason1: From physical common sense to embodied reasoning

    Alisson Azzolini, Junjie Bai, Hannah Brandon, Jiaxin Cao, Prithvijit Chattopadhyay, Huayu Chen, Jinju Chu, Yin Cui, Jenna Diamond, Yifan Ding, et al. Cosmos-reason1: From physical common sense to embodied reasoning. arXiv preprint arXiv:2503.15558, 2025

    Pith/arXiv arXiv 2025

  2. [2]

    Qwen3-vl technical report

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Junyang Lin, Xuejing Liu, Jiawei Liu, Chenglong Liu, Yang Liu, Dayiheng Liu, Shixuan ...

    Pith/arXiv arXiv doi:10.48550/arxiv.2511.21631 2025

  3. [3]

    Gr00t n1: An open foundation model for generalist humanoid robots

    Johan Bjorck, Fernando Casta \ n eda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi Fan, Yu Fang, Dieter Fox, Fengyuan Hu, Spencer Huang, et al. Gr00t n1: An open foundation model for generalist humanoid robots. arXiv preprint arXiv:2503.14734, 2025

    Pith/arXiv arXiv 2025

  4. [4]

    _0 : A vision-language-action flow model for general robot control

    Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Sergey Levine, Adrian Li-Bell, Mohith Mothukuri, Suraj Nair, Karl Pertsch, Lucy Xiaoyang Shi, James Tanner, Quan Vuong, Anna Walling, Haohuan Wang, and Ury Zhilinsky. _0 : A vis...

    Pith/arXiv arXiv 2024

  5. [5]

    Gopalakrishnan, Karol Hausman, Alexander Herzog, Jasmine Hsu, Julian Ibarz, Brian Ichter, A

    Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, K. Gopalakrishnan, Karol Hausman, Alexander Herzog, Jasmine Hsu, Julian Ibarz, Brian Ichter, A. Irpan, Tomas Jackson, Sally Jesmonth, Nikhil J. Joshi, Ryan C. Julian, Dmitry Kalashnikov, Yuheng Kuang, Isabel Leal, Kuang-Huei Lee, S. Levine, Yao Lu, U. Malla, D. Manj...

    Pith/arXiv arXiv 2022

  6. [6]

    Rt-2: Vision-language-action models transfer web knowledge to robotic control

    Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choromanski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. arXiv preprint arXiv:2307.15818, 2023

    Pith/arXiv arXiv 2023

  7. [7]

    Agibot world colosseo: A large-scale manipulation platform for scalable and intelligent embodied systems

    Qingwen Bu, Jisong Cai, Li Chen, Xiuqi Cui, Yan Ding, Siyuan Feng, Shenyuan Gao, Xindong He, Xuan Hu, Xu Huang, et al. Agibot world colosseo: A large-scale manipulation platform for scalable and intelligent embodied systems. arXiv preprint arXiv:2503.06669, 2025

    Pith/arXiv arXiv 2025

  8. [8]

    Depthlm: Metric depth from vision language models

    Zhipeng Cai, Ching-Feng Yeh, Hu Xu, Zhuang Liu, Gregory Meyer, Xinjie Lei, Changsheng Zhao, Shang-Wen Li, Vikas Chandra, and Yangyang Shi. Depthlm: Metric depth from vision language models. arXiv preprint arXiv:2509.25413, 2025

    arXiv 2025

  9. [9]

    Chang, Angela Dai, T

    Angel X. Chang, Angela Dai, T. Funkhouser, Maciej Halber, M. Nießner, M. Savva, Shuran Song, Andy Zeng, and Yinda Zhang. Matterport3d: Learning from rgb-d data in indoor environments. 2017 International Conference on 3D Vision (3DV), pages 667--676, 2017

    2017

  10. [10]

    Spatialvlm: Endowing vision-language models with spatial reasoning capabilities

    Boyuan Chen, Zhuo Xu, Sean Kirmani, Brain Ichter, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. Spatialvlm: Endowing vision-language models with spatial reasoning capabilities. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14455--14465, 2024

    2024

  11. [11]

    Revisiting referring expression comprehension evaluation in the era of large multimodal models

    Jierun Chen, Fangyun Wei, Jinjing Zhao, Sizhe Song, Bohuai Wu, Zhuoxuan Peng, S-H Gary Chan, and Hongyang Zhang. Revisiting referring expression comprehension evaluation in the era of large multimodal models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 513--524, 2025 a

    2025

  12. [12]

    Robotwin 2.0: A scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipulation

    Tianxing Chen, Zanxin Chen, Baijun Chen, Zijian Cai, Yibin Liu, Zixuan Li, Qiwei Liang, Xianliang Lin, Yiheng Ge, Zhenyu Gu, et al. Robotwin 2.0: A scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipulation. arXiv preprint arXiv:2506.18088, 2025 b

    Pith/arXiv arXiv 2025

  13. [13]

    Egoplan-bench: Benchmarking multimodal large language models for human-level planning

    Yi Chen, Yuying Ge, Yixiao Ge, Mingyu Ding, Bohao Li, Rui Wang, Ruifeng Xu, Ying Shan, and Xihui Liu. Egoplan-bench: Benchmarking multimodal large language models for human-level planning. arXiv preprint arXiv:2312.06722, 2023

    Pith/arXiv arXiv 2023

  14. [14]

    Smith, Fei Xia, Dieter Fox, and Ranjay Krishna

    Long Cheng, Jiafei Duan, Yi Ru Wang, Haoquan Fang, Boyang Li, Yushan Huang, Elvis Wang, Ainaz Eftekhar, Jason Lee, Wentao Yuan, Rose Hendrix, Noah A. Smith, Fei Xia, Dieter Fox, and Ranjay Krishna. Pointarena: Probing multimodal grounding through language-guided pointing. arXiv preprint arXiv:2505.09990, 2025

    Pith/arXiv arXiv 2025

  15. [15]

    Diffusion policy: Visuomotor policy learning via action diffusion

    Cheng Chi, Siyuan Feng, Yilun Du, Zhenjia Xu, Eric Cousineau, Benjamin Burchfiel, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. Robotics: Science and Systems, 2023

    2023

  16. [16]

    Molmo2: Open weights and data for vision-language models with video understanding and grounding

    Christopher Clark, Jieyu Zhang, Zixian Ma, Jae Sung Park, Rohun Tripathi, Sangho Lee, Mohammadreza Salehi, Jason Ren, Chris Dongjoo Kim, Yinuo Yang, et al. Molmo2: Open weights and data for vision-language models with video understanding and grounding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 28652--28668, 2026

    2026

  17. [17]

    Open x-embodiment: Robotic learning datasets and RT-X models

    Open X-Embodiment Collaboration. Open x-embodiment: Robotic learning datasets and RT-X models. CoRR, abs/2310.08864, 2023. doi:10.48550/ARXIV.2310.08864. URL https://doi.org/10.48550/arXiv.2310.08864

    Pith/arXiv arXiv doi:10.48550/arxiv.2310.08864 2023

  18. [18]

    Starvla: A lego-like codebase for vision-language-action model developing

    StarVLA Community. Starvla: A lego-like codebase for vision-language-action model developing. arXiv preprint arXiv:2604.05014, 2026

    Pith/arXiv arXiv 2026

  19. [19]

    Chang, Manolis Savva, Maciej Halber, Thomas A

    Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas A. Funkhouser, and Matthias Nie ner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017 , pages 2432--2443. IEEE Computer Society, 2017. doi:10.1109/CVPR.2017.261. U...

    doi:10.1109/cvpr.2017.261 2017

  20. [20]

    Rynnbrain: Open embodied foundation models

    Ronghao Dang, Jiayan Guo, Bohan Hou, Sicong Leng, Kehan Li, Xin Li, Jiangpin Liu, Yunxuan Mao, Zhikai Wang, Yuqian Yuan, Minghao Zhu, Xiao Lin, Yang Bai, Qian Jiang, Yaxi Zhao, Minghua Zeng, Junlong Gao, Yuming Jiang, Jun Cen, Siteng Huang, Liuyi Wang, Wenqiao Zhang, Chengju Liu, Jianfei Yang, Shijian Lu, and Deli Zhao. Rynnbrain: Open embodied foundation...

    doi:10.48550/arxiv.2602.14979 2026

  21. [21]

    Arkitscenes: A diverse real-world dataset for 3d indoor scene understanding using mobile rgb-d data

    Afshin Dehghan, Gilad Baruch, Zhuoyuan Chen, Yuri Feigin, Peter Fu, Thomas Gebauer, Daniel Kurz, Tal Dimry, Brandon Joffe, Arik Schwartz, and Elad Shulman. Arkitscenes: A diverse real-world dataset for 3d indoor scene understanding using mobile rgb-d data. 2021

    2021

  22. [22]

    Molmo and pixmo: Open weights and open data for state-of-the-art vision-language models

    Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi, Yue Yang, Jae Sung Park, Mohammadreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, et al. Molmo and pixmo: Open weights and open data for state-of-the-art vision-language models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 91--104, 2025

    2025

  23. [23]

    Graspmolmo: Generalizable task-oriented grasping via large-scale synthetic data generation

    Abhay Deshpande, Yuquan Deng, Arijit Ray, Jordi Salvador, Winson Han, Jiafei Duan, Kuo-Hao Zeng, Yuke Zhu, Ranjay Krishna, and Rose Hendrix. Graspmolmo: Generalizable task-oriented grasping via large-scale synthetic data generation. arXiv preprint arXiv:2505.13441, 2025

    Pith/arXiv arXiv 2025

  24. [24]

    Mm-ifengine: Towards multimodal instruction following

    Shengyuan Ding, Shenxi Wu, Xiangyu Zhao, Yuhang Zang, Haodong Duan, Xiaoyi Dong, Pan Zhang, Yuhang Cao, Dahua Lin, and Jiaqi Wang. Mm-ifengine: Towards multimodal instruction following. arXiv preprint arXiv:2504.07957, 2025

    Pith/arXiv arXiv 2025

  25. [25]

    Embspatial-bench: Benchmarking spatial understanding for embodied tasks with large vision-language models

    Mengfei Du, Binhao Wu, Zejun Li, Xuanjing Huang, and Zhongyu Wei. Embspatial-bench: Benchmarking spatial understanding for embodied tasks with large vision-language models. arXiv preprint arXiv:2410.16147, 2024

    Pith/arXiv arXiv 2024

  26. [26]

    Vlmevalkit: An open-source toolkit for evaluating large multi-modality models

    Haodong Duan, Junming Yang, Yuxuan Qiao, Xinyu Fang, Lin Chen, Yuan Liu, Xiaoyi Dong, Yuhang Zang, Pan Zhang, Jiaqi Wang, et al. Vlmevalkit: An open-source toolkit for evaluating large multi-modality models. In Proceedings of the 32nd ACM international conference on multimedia, pages 11198--11201, 2024

    2024

  27. [27]

    VLM-3R: vision-language models augmented with instruction-aligned 3d reconstruction

    Zhiwen Fan, Jian Zhang, Renjie Li, Junge Zhang, Runjin Chen, Hezhen Hu, Kevin Wang, Huaizhi Qu, Dilin Wang, Zhicheng Yan, Hongyu Xu, Justin Theiss, Tianlong Chen, Jiachen Li, Zhengzhong Tu, Zhangyang Wang, and Rakesh Ranjan. VLM-3R: vision-language models augmented with instruction-aligned 3d reconstruction. CoRR, abs/2505.20279, 2025. doi:10.48550/ARXIV....

    Pith/arXiv arXiv doi:10.48550/arxiv.2505.20279 2025

  28. [28]

    Molmoact2: Action reasoning models for real-world deployment

    Haoquan Fang, Jiafei Duan, Donovan Clay, Sam Wang, Shuo Liu, Weikai Huang, Xiang Fan, Wei-Chuan Tsai, Shirui Chen, Yi Ru Wang, Shanli Xing, Jaemin Cho, Jae Sung Park, Ainaz Eftekhar, Peter Sushko, Karen Farley, Angad Wadhwa, Cole Harrison, Winson Han, Ying-Chun Lee, Eli VanderBilt, Rose Hendrix, Suveen Ellawela, Lucas Ngoo, Joyce Chai, Zhongzheng Ren, Ali...

    Pith/arXiv arXiv 2025

  29. [29]

    Libero-plus: In-depth robustness analysis of vision-language-action models

    Senyu Fei, Siyin Wang, Junhao Shi, Zihao Dai, Jikun Cai, Pengfang Qian, Li Ji, Xinzhe He, Shiduo Zhang, Zhaoye Fei, et al. Libero-plus: In-depth robustness analysis of vision-language-action models. arXiv preprint arXiv:2510.13626, 2025

    Pith/arXiv arXiv 2025

  30. [30]

    Onethinker: All-in-one reasoning model for image and video

    Kaituo Feng, Manyuan Zhang, Hongyu Li, Kaixuan Fan, Shuang Chen, Yilei Jiang, Dian Zheng, Peiwen Sun, Yiyuan Zhang, Haoze Sun, Yan Feng, Peng Pei, Xunliang Cai, and Xiangyu Yue. Onethinker: All-in-one reasoning model for image and video. CoRR, abs/2512.03043, 2025. doi:10.48550/ARXIV.2512.03043. URL https://doi.org/10.48550/arXiv.2512.03043

    Pith/arXiv arXiv doi:10.48550/arxiv.2512.03043 2025

  31. [31]

    Smith, Wei-Chiu Ma, and Ranjay Krishna

    Xingyu Fu, Yushi Hu, Bangzheng Li, Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth, Noah A. Smith, Wei-Chiu Ma, and Ranjay Krishna. Blink: Multimodal large language models can see but not perceive. arXiv preprint arXiv:2404.12390, 2024

    Pith/arXiv arXiv 2024

  32. [32]

    Towards generalizable vision-language robotic manipulation: A benchmark and llm-guided 3d policy

    Ricardo Garcia, Shizhe Chen, and Cordelia Schmid. Towards generalizable vision-language robotic manipulation: A benchmark and llm-guided 3d policy. In IROS, 2024

    2024

  33. [33]

    Ace-brain-0: Spatial intelligence as a shared scaffold for universal embodiments

    Ziyang Gong, Zehang Luo, Anke Tang, Zhe Liu, Shi Fu, Zhi Hou, Ganlin Yang, Weiyun Wang, Xiaofeng Wang, Jianbo Liu, et al. Ace-brain-0: Spatial intelligence as a shared scaffold for universal embodiments. arXiv preprint arXiv:2603.03198, 2026

    arXiv 2026

  34. [34]

    Handal: A dataset of real-world manipulable object categories with pose annotations, affordances, and reconstructions, 2023

    Andrew Guo, Bowen Wen, Jianhe Yuan, Jonathan Tremblay, Stephen Tyree, Jeffrey Smith, and Stan Birchfield. Handal: A dataset of real-world manipulable object categories with pose annotations, affordances, and reconstructions, 2023. URL https://arxiv.org/abs/2308.01477

    Pith/arXiv arXiv 2023

  35. [35]

    Lvis: A dataset for large vocabulary instance segmentation

    Agrim Gupta, Piotr Dollar, and Ross Girshick. Lvis: A dataset for large vocabulary instance segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5356--5364, 2019

    2019

  36. [36]

    Roboafford++: A generative ai-enhanced dataset for multimodal affordance learning in robotic manipulation and navigation

    Xiaoshuai Hao, Yingbo Tang, Lingfeng Zhang, Yanbiao Ma, Yunfeng Diao, Ziyu Jia, Wenbo Ding, Hangjun Ye, and Long Chen. Roboafford++: A generative ai-enhanced dataset for multimodal affordance learning in robotic manipulation and navigation. arXiv preprint arXiv:2511.12436, 2025 a

    arXiv 2025

  37. [37]

    Mimo-embodied: X-embodied foundation model technical report

    Xiaoshuai Hao, Lei Zhou, Zhijian Huang, Zhiwen Hou, Yingbo Tang, Lingfeng Zhang, Guang Li, Zheng Lu, Shuhuai Ren, Xianhui Meng, et al. Mimo-embodied: X-embodied foundation model technical report. arXiv preprint arXiv:2511.16518, 2025 b

    Pith/arXiv arXiv 2025

  38. [38]

    Open-Set Image Tagging with Multi-Grained Text Supervision

    Xinyu Huang, Yi-Jie Huang, Youcai Zhang, Weiwei Tian, Rui Feng, Yuejie Zhang, Yanchun Xie, Yaqian Li, and Lei Zhang. Open-Set Image Tagging with Multi-Grained Text Supervision. 2023

    2023

  39. [39]

    Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Manuel Y. Galliker, Dibya Ghosh, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Devin LeBlanc, Sergey Levine, Adrian Li - Bell, Mohith Mothukuri, Suraj Nair, Karl Perts...

    Pith/arXiv arXiv doi:10.48550/arxiv.2504.16054 2025

  40. [40]

    Robobrain: A unified brain model for robotic manipulation from abstract to concrete

    Yuheng Ji, Huajie Tan, Jiayu Shi, Xiaoshuai Hao, Yuan Zhang, Hengyuan Zhang, Pengwei Wang, Mengdi Zhao, Yao Mu, Pengju An, Xinda Xue, Qinghang Su, Huaihai Lyu, Xiaolong Zheng, Jiaming Liu, Zhongyuan Wang, and Shanghang Zhang. Robobrain: A unified brain model for robotic manipulation from abstract to concrete. In IEEE/CVF Conference on Computer Vision and ...

    doi:10.1109/cvpr52734.2025.00168 2025

  41. [41]

    Cotracker3: Simpler and better point tracking by pseudo-labelling real videos

    Nikita Karaev, Iurii Makarov, Jianyuan Wang, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. Cotracker3: Simpler and better point tracking by pseudo-labelling real videos. arXiv preprint arXiv:2410.11831, 2024

    Pith/arXiv arXiv 2024

  42. [42]

    Referitgame: Referring to objects in photographs of natural scenes

    Sahar Kazemzadeh, Vicente Ordonez, Mark Matten, and Tamara Berg. Referitgame: Referring to objects in photographs of natural scenes. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 787--798, 2014

    2014

  43. [43]

    Nair, Ashwin Balakrishna, S

    Alexander Khazatsky, Karl Pertsch, S. Nair, Ashwin Balakrishna, S. Dasari, Siddharth Karamcheti, Soroush Nasiriany, M. K. Srirama, L. Chen, Kirsty Ellis, P. Fagan, Joey Hejna, Masha Itkina, Marion Lepert, Ye Ma, Patrick Miller, Jimmy Wu, Suneel Belkhale, S. Dass, Huy Ha, Arhan Jain, Abraham Lee, Youngwoon Lee, Marius Memmel, Sungjae Park, Ilija Radosavovi...

    Pith/arXiv arXiv 2024

  44. [44]

    Openvla: An open-source vision-language-action model

    Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, Quan Vuong, Thomas Kollar, Benjamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, and Chelsea Finn. Openvla: An open-source vision-language-action model. arXiv preprint arXiv:2406.09246, 2024

    Pith/arXiv arXiv 2024

  45. [45]

    Fine-tuning vision-language-action models: Optimizing speed and success

    Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Quan Vuong, et al. Fine-tuning vision-language-action models: Optimizing speed and success. arXiv preprint arXiv:2502.19645, 2025

    Pith/arXiv arXiv 2025

  46. [46]

    Gonzalez, Haotong Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Haotong Zhang, and Ion Stoica. Efficient Memory Management for Large Language Model Serving with PagedAttention. 2023

    2023

  47. [47]

    Llava-onevision: Easy visual task transfer

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326, 2024 a

    Pith/arXiv arXiv 2024

  48. [48]

    Robointer: A holistic intermediate representation suite towards robotic manipulation

    Hao Li, Ziqin Wang, Zi han Ding, Shuai Yang, Yilun Chen, Yang Tian, Xiaolin Hu, Tai Wang, Dahua Lin, Feng Zhao, Si Liu, and Jiangmiao Pang. Robointer: A holistic intermediate representation suite towards robotic manipulation. arXiv preprint arXiv:2602.09973, 2026

    arXiv 2026

  49. [49]

    Manipllm: Embodied multimodal large language model for object-centric robotic manipulation

    Xiaoqi Li, Mingxu Zhang, Yiran Geng, Haoran Geng, Yuxing Long, Yan Shen, Renrui Zhang, Jiaming Liu, and Hao Dong. Manipllm: Embodied multimodal large language model for object-centric robotic manipulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18061--18070, 2024 b

    2024

  50. [50]

    Evaluating real-world robot manipulation policies in simulation

    Xuanlin Li, Kyle Hsu, Jiayuan Gu, Karl Pertsch, Oier Mees, Homer Rich Walke, Chuyuan Fu, Ishikaa Lunawat, Isabel Sieh, Sean Kirmani, et al. Evaluating real-world robot manipulation policies in simulation. arXiv preprint arXiv:2405.05941, 2024 c

    Pith/arXiv arXiv 2024

  51. [51]

    Euclid's gift: Enhancing spatial perception and reasoning in vision-language models via geometric surrogate tasks

    Shijie Lian, Changti Wu, Laurence Tianruo Yang, Hang Yuan, Bin Yu, Lei Zhang, and Kai Chen. Euclid's gift: Enhancing spatial perception and reasoning in vision-language models via geometric surrogate tasks. arXiv preprint arXiv:2509.24473, 2025

    arXiv 2025

  52. [52]

    Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. In International Conference on Learning Representations, ICLR 2023 , 2023. arXiv:2210.02747

    Pith/arXiv arXiv 2023

  53. [53]

    Libero: Benchmarking knowledge transfer for lifelong robot learning

    Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. Libero: Benchmarking knowledge transfer for lifelong robot learning. Advances in Neural Information Processing Systems, 36: 0 44776--44791, 2023 a

    2023

  54. [54]

    Skywork-reward-v2: Scaling preference data curation via human-ai synergy

    Chris Yuhao Liu, Liang Zeng, Yuzhen Xiao, Jujie He, Jiacai Liu, Chaojie Wang, Rui Yan, Wei Shen, Fuxiang Zhang, Jiacheng Xu, et al. Skywork-reward-v2: Scaling preference data curation via human-ai synergy. arXiv preprint arXiv:2507.01352, 2025

    Pith/arXiv arXiv 2025

  55. [55]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In CVPR, 2024

    2024

  56. [56]

    Reflect: Summarizing robot experiences for failure explanation and correction

    Zeyi Liu, Arpit Bahety, and Shuran Song. Reflect: Summarizing robot experiences for failure explanation and correction. arXiv preprint arXiv:2306.15724, 2023 b

    Pith/arXiv arXiv 2023

  57. [57]

    Vl-grasp: a 6-dof interactive grasp policy for language-oriented objects in cluttered indoor scenes

    Yuhao Lu, Yixuan Fan, Beixing Deng, Fangfu Liu, Yali Li, and Shengjin Wang. Vl-grasp: a 6-dof interactive grasp policy for language-oriented objects in cluttered indoor scenes. arXiv preprint arXiv:2308.00640, 2023

    Pith/arXiv arXiv 2023

  58. [58]

    Visual embodied brain: Let multimodal large language models see, think, and control in spaces

    Gen Luo, Ganlin Yang, Ziyang Gong, Guanzhou Chen, Haonan Duan, Erfei Cui, Ronglei Tong, Zhi Hou, Tianyi Zhang, Zhe Chen, et al. Visual embodied brain: Let multimodal large language models see, think, and control in spaces. arXiv preprint arXiv:2506.00123, 2025

    Pith/arXiv arXiv 2025

  59. [59]

    A survey on vision--language--action models for embodied ai

    Yueen Ma, Zixing Song, Yuzheng Zhuang, Jianye Hao, and Irwin King. A survey on vision--language--action models for embodied ai. arXiv preprint arXiv:2505.01244, 2025

    Pith/arXiv arXiv 2025

  60. [60]

    Openeqa: Embodied question answering in the era of foundation models

    Arjun Majumdar, Anurag Ajay, Xiaohan Zhang, Pranav Putta, Sriram Yenamandra, Mikael Henaff, Sneha Silwal, Paul Mcvay, Oleksandr Maksymets, Sergio Arnaud, et al. Openeqa: Embodied question answering in the era of foundation models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16488--16498, 2024

    2024

  61. [61]

    Guibas, Mustafa Mukadam, Abhinav Gupta, and Shubham Tulsiani

    Kaichun Mo, Leonidas J. Guibas, Mustafa Mukadam, Abhinav Gupta, and Shubham Tulsiani. Where2act: From pixels to actions for articulated 3d objects. 2021

    2021

  62. [62]

    Embodiedgpt: Vision-language pre-training via embodied chain of thought

    Yao Mu, Qinglong Zhang, Mengkang Hu, Wenhai Wang, Mingyu Ding, Jun Jin, Bin Wang, Jifeng Dai, Yu Qiao, and Ping Luo. Embodiedgpt: Vision-language pre-training via embodied chain of thought. Advances in Neural Information Processing Systems, 2024

    2024

  63. [63]

    Robocasa: Large-scale simulation of everyday tasks for generalist robots

    Soroush Nasiriany, Abhiram Maddukuri, Lance Zhang, Adeet Parikh, Aaron Lo, Abhishek Joshi, Ajay Mandlekar, and Yuke Zhu. Robocasa: Large-scale simulation of everyday tasks for generalist robots. arXiv preprint arXiv:2406.02523, 2024

    Pith/arXiv arXiv 2024

  64. [64]

    Fei Ni, Min Zhang, Pengyi Li, Yifu Yuan, Lingfeng Zhang, Yuecheng Liu, Peilong Han, Longxin Kou, Shaojin Ma, Jinbin Qiao, David Gamaliel Arcos Bravo, Yuening Wang, Xiao Hu, Zhanguang Zhang, X. Yao, Yutong Li, Zhao Zhang, Ying Wen, Ying-Cong Chen, Xiaodan Liang, Liang Lin, Bin He, Haitham Bou-Ammar, He Wang, Huazhe Xu, Jiankang Deng, Shan Luo, Shu Jiang, W...

    arXiv 2025

  65. [65]

    Gr00t n1.5: Advancing generalist robot foundation models

    NVIDIA . Gr00t n1.5: Advancing generalist robot foundation models. https://developer.nvidia.com/blog/advancing-generalist-robot-foundation-models-with-gr00t-n1-5/, 2025

    2025

  66. [66]

    Spacer: Reinforcing mllms in video spatial reasoning

    Kun Ouyang, Yuanxin Liu, Haoning Wu, Yi Liu, Hao Zhou, Jie Zhou, Fandong Meng, and Xu Sun. Spacer: Reinforcing mllms in video spatial reasoning. arXiv preprint arXiv:2504.01805, 2025

    Pith/arXiv arXiv 2025

  67. [67]

    Guardian: Detecting robotic planning and execution errors with vision-language models

    Paul Pacaud, Ricardo Garcia, Shizhe Chen, and Cordelia Schmid. Guardian: Detecting robotic planning and execution errors with vision-language models. CoRR, abs/2512.01946, 2025. doi:10.48550/ARXIV.2512.01946. URL https://doi.org/10.48550/arXiv.2512.01946

    doi:10.48550/arxiv.2512.01946 2025

  68. [68]

    Thinker: A vision-language foundation model for embodied intelligence

    Baiyu Pan, Daqin Luo, Junpeng Yang, Jiyuan Wang, Yixuan Zhang, Hailin Shi, and Jichao Jiao. Thinker: A vision-language foundation model for embodied intelligence. CoRR, abs/2601.21199, 2026. doi:10.48550/ARXIV.2601.21199. URL https://doi.org/10.48550/arXiv.2601.21199

    doi:10.48550/arxiv.2601.21199 2026

  69. [69]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, ICCV 2023 , pages 4195--4205, 2023. arXiv:2212.09748

    Pith/arXiv arXiv 2023

  70. [70]

    Egothinker: Unveiling egocentric reasoning with spatio-temporal cot

    Baoqi Pei, Yifei Huang, Jilan Xu, Yuping He, Guo Chen, Fei Wu, Jiangmiao Pang, and Yu Qiao. Egothinker: Unveiling egocentric reasoning with spatio-temporal cot. Advances in Neural Information Processing Systems, 38: 0 44140--44168, 2026

    2026

  71. [71]

    Fast: Efficient action tokenization for vision-language-action models

    Karl Pertsch, Kyle Luo, Gaurav Patel, Zhenjia Cui, Robin Strudel, Jie Lim, Brian Ichter, Karol Hausman, Chelsea Finn, Sergey Levine, et al. Fast: Efficient action tokenization for vision-language-action models. arXiv preprint arXiv:2501.09747, 2025

    Pith/arXiv arXiv 2025

  72. [72]

    Capture: Evaluating spatial reasoning in vision language models via occluded object counting

    Atin Pothiraj, Elias Stengel-Eskin, Jaemin Cho, and Mohit Bansal. Capture: Evaluating spatial reasoning in vision language models via occluded object counting. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8001--8010, 2025

    2025

  73. [73]

    Egoplan-bench2: A benchmark for multimodal large language model planning in real-world scenarios

    Lu Qiu, Yi Chen, Yuying Ge, Yixiao Ge, Ying Shan, and Xihui Liu. Egoplan-bench2: A benchmark for multimodal large language model planning in real-world scenarios. arXiv preprint arXiv:2412.04447, 2024

    Pith/arXiv arXiv 2024

  74. [74]

    Eo-1: Interleaved vision-text-action pretraining for general robot control

    Delin Qu, Haoming Song, Qizhi Chen, Zhaoqing Chen, Xianqiang Gao, Xinyi Ye, Qi Lv, Modi Shi, Guanghui Ren, Cheng Ruan, et al. Eo-1: Interleaved vision-text-action pretraining for general robot control. arXiv preprint arXiv:2508.21112, 2025 a

    arXiv 2025

  75. [75]

    Spatialvla: Exploring spatial representations for visual-language-action model

    Delin Qu, Haoming Song, Qizhi Chen, Yuanqi Yao, Xinyi Ye, Yan Ding, Zhigang Wang, JiaYuan Gu, Bin Zhao, Dong Wang, and Xuelong Li. Spatialvla: Exploring spatial representations for visual-language-action model. arXiv preprint arXiv:2501.15830, 2025 b

    Pith/arXiv arXiv 2025

  76. [76]

    Habitat-matterport 3d dataset (hm3d): 1000 large-scale 3d environments for embodied ai

    Santhosh K Ramakrishnan, Aaron Gokaslan, Erik Wijmans, Oleksandr Maksymets, Alex Clegg, John Turner, Eric Undersander, Wojciech Galuba, Andrew Westbury, Angel X Chang, et al. Habitat-matterport 3d dataset (hm3d): 1000 large-scale 3d environments for embodied ai. arXiv preprint arXiv:2109.08238, 2021

    Pith/arXiv arXiv 2021

  77. [77]

    PACO : Parts and attributes of common objects

    Vignesh Ramanathan, Anmol Kalia, Vladan Petrovic, Yi Wen, Baixue Zheng, Baishan Guo, Rui Wang, Aaron Marquez, Rama Kovvuri, Abhishek Kadian, Amir Mousavi, Yiwen Song, Abhimanyu Dubey, and Dhruv Mahajan. PACO : Parts and attributes of common objects. In arXiv preprint arXiv:2301.01795, 2023

    Pith/arXiv arXiv 2023

  78. [78]

    Sam 2: Segment anything in images and videos

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman R \"a dle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos. In International Conference on Learning Representations, volume 2025, pages 28085--28128, 2025

    2025

  79. [79]

    Sat: Spatial aptitude training for multimodal language models

    Arijit Ray, Jiafei Duan, Reuben Tan, Dina Bashkirova, Rose Hendrix, Kiana Ehsani, Aniruddha Kembhavi, Bryan A Plummer, Ranjay Krishna, Kuo-Hao Zeng, et al. Sat: Spatial aptitude training for multimodal language models. arXiv preprint arXiv:2412.07755, 3, 2024

    arXiv 2024

  80. [80]

    Grounded SAM: assembling open-world models for diverse visual tasks

    Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, Zhaoyang Zeng, Hao Zhang, Feng Li, Jie Yang, Hongyang Li, Qing Jiang, and Lei Zhang. Grounded SAM: assembling open-world models for diverse visual tasks. CoRR, abs/2401.14159, 2024. doi:10.48550/ARXIV.2401.14159. URL https://doi.org/10.4855...

    Pith/arXiv arXiv doi:10.48550/arxiv.2401.14159 2024

Showing first 80 references.