Pith. sign in

REVIEW 3 major objections 3 minor 9 cited by

Robix unifies robot reasoning, planning, and natural language interaction in a single vision-language model.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A three-stage-trained VLM unifies robot planning and dialogue, and beats commercial VLMs on the authors' interactive-task benchmarks.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection Credible public-benchmark gains and a plausible three-stage recipe, but the headline interactive-task advantage over Gemini is not statistically supported as reported. the 3 major comments →

arxiv 2509.01106 v2 pith:AX5VGLO5 submitted 2025-09-01 cs.AI cs.CVcs.RO

Robix: A Unified Model for Robot Interaction, Reasoning and Planning

Huang Fang , Mengxi Zhang , Heng Dong , Wei Li , Zixuan Wang , Qifeng Zhang , Xueyun Tian , Yucheng Hu
show 1 more author
Hang Li
This is my paper
classification cs.AI cs.CVcs.RO
keywords vision-language modelsrobot task planninghuman-robot interactionchain-of-thought reasoningembodied reasoningreinforcement learninghierarchical robot systemsdata synthesis
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Robix is built as the high-level cognitive layer of a hierarchical robot: one vision-language model that takes in camera images and user speech, thinks about what is happening, emits one atomic command for a low-level controller, and answers in natural language. The paper argues that this single-model design beats both modular pipelines and large commercial vision-language models on interactive long-horizon tasks such as clearing a table, packing groceries, and filtering food by dietary constraints. The model is trained in three stages: continued pretraining on embodied-reasoning data, supervised fine-tuning on synthetically generated human-robot interaction traces, and reinforcement learning that rewards consistency between the model's private thinking and its emitted action. If the claim holds, a general-purpose robot companion can be built around one open-weight model rather than a hand-wired stack of separate planner, reasoner, and dialogue components.

Core claim

The paper's central claim is that interactive long-horizon robot behavior does not have to be assembled from separate modules. Robix is a single vision-language model that at each step receives the latest observations and user utterance and outputs a thought, an atomic action command, and sometimes a verbal response, formalized as a conditional distribution over (thought, action, response) given the interaction history. The authors build it by continuing pretraining a general VLM on about 200B tokens weighted toward 3D spatial understanding, visual grounding, and task-centric reasoning; supervised fine-tuning on synthesized interaction trajectories that cover multi-stage, constrained, open-e

What carries the argument

The carrying mechanism is the 'reasoning-action sequence': every interaction step is one language-model decoding that emits a thought, an optional atomic plan command, and an optional verbal response. This single output format absorbs instruction understanding, task-status monitoring, replanning, interruption handling, proactive clarification, and dialogue into one next-token prediction problem. The three-stage training pipeline installs the sequence: continued pretraining supplies embodied perception and grounding; supervised fine-tuning supplies interaction diversity through data synthesis; and reinforcement learning, using an external judge model to score whether the emitted action matche

Load-bearing premise

The measured advantage over commercial baselines rests on a small, mostly in-house evaluation: the offline in-distribution set is sampled from the same synthesized data used for training, the out-of-distribution scripts are manually designed and annotated, and the online tasks are only a few curated scenarios repeated four times; if those evaluations do not represent real interactive generality, the headline lead collapses.

What would settle it

Run Robix-32B-RL and Gemini-2.5-Pro on a fresh set of interactive long-horizon tasks built by an independent team, with scenes, objects, and interruption types not drawn from the paper's synthesized or teleoperated distribution, using the same VLA low-level controller and dozens of episodes per task. If the 1.6-4.3 percentage-point task-progress advantage does not reproduce outside those in-house sets, the paper's central performance claim fails.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • A single open-weight VLM can replace hand-designed modular planner-and-chat pipelines as the high-level controller in a hierarchical robot system.
  • Chain-of-thought reasoning is load-bearing: removing it drops accuracy by more than 7 points on the internal OOD set and 26.7 points on open-ended in-distribution tasks.
  • Reinforcement learning with a thought-action consistency reward improves both plan accuracy and interactive behavior beyond supervised fine-tuning alone.
  • The high-level planner and the low-level VLA must share an action vocabulary; baselines lost large fractions of task progress when the VLA could not parse semantically correct commands.
  • Real-time interruption handling and proactive dialogue can be trained from synthesized data instead of being hand-coded as separate workflows.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The three-stage recipe (broad embodied pretraining, interaction-focused SFT, thought-action RL) is a template that could likely be lifted onto other open vision-language backbones, making the paper's contribution a training strategy as much as a single model.
  • Because Robix keeps only the latest N visual observations in a short-term context window, very long interactive sessions would require external long-term memory; the paper's own limitation section points in that direction, so memory-augmented variants are the natural next test.
  • An independent evaluation with scenes, objects, and interruption types built outside the authors' synthesized and teleoperated distribution would be the cleanest check of whether the 1.6-4.3 point lead over Gemini-2.5-Pro is a property of the model or of the evaluation sets.
  • The thought-action consistency reward is a general idea: any agent that emits natural-language plans could be trained to keep its stated reasoning aligned with its chosen action, not just robot manipulators.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper introduces Robix, a vision-language model built on Qwen2.5-VL that acts as a high-level cognitive layer in a hierarchical robot, unifying reasoning, task planning, and natural-language interaction. Training is a three-stage recipe: (1) continued pretraining on roughly 200B tokens of embodied reasoning data (3D spatial understanding, visual grounding, task-centric reasoning, general VQA); (2) supervised fine-tuning on synthesized human-robot interaction trajectories with chain-of-thought traces; and (3) GRPO reinforcement learning with a thought-action consistency reward. The model is evaluated on 31 public vision-language benchmarks (Table 1), on an internal offline benchmark (Table 3), and on five real-world online tasks in two modes: with a human UMI teleoperator (Fig. 5) and with the GR-3 VLA controller on the ByteMini robot (Fig. 6). The authors claim Robix-32B surpasses Gemini-2.5-Pro by 1.6 and 4.3 percentage points in task progress.

Significance. The continued-pretraining results on public benchmarks are a credible and valuable contribution: Robix-7B improves over Qwen2.5-VL-7B by 39.6 F1 points on LVIS-MG, and Robix-32B outperforms commercial baselines on most spatial and grounding tasks while retaining general VQA performance (Table 1). The problem formulation—training a single VLM for interactive long-horizon robotic tasks, with explicit thought-action consistency RL—is timely and practically relevant. However, the paper's headline claim of superior interactive task execution rests on small, internal, and partly circular evaluations. If those gaps are closed, the work would be significant; as presented, the interactive-execution claim is not yet established at the level the abstract and introduction assert.

major comments (3)
  1. [§4.3.1, Fig. 5] The claimed 1.6 pp advantage over Gemini-2.5-Pro is not statistically established. Each task–model pair is evaluated only four times; no standard deviations, confidence intervals, per-episode scores, or inter-annotator agreement are reported, and the text does not state that the task-progress annotators were blinded. The paper itself says both models 'rank first in 3 of the 5 tasks' and that Robix's average is only 'slightly higher.' With five tasks, one miscored episode can move a task mean by about 25 points, so the headline margin is plausibly within sampling noise. This is load-bearing because the abstract and Section 1 use this result to claim Robix surpasses Gemini-2.5-Pro. Please report per-episode data, increase repetitions or use pooled statistical tests, add confidence intervals, and use blinded scoring.
  2. [§4.2, Table 3, §A.5] The offline evaluation that carries much of the interactive-task evidence has two confounds. First, the Internal ID set is 'randomly sampled from our synthesized data' (Section 4.2), i.e., from the same distribution as the SFT training data, so it measures distribution fit rather than generalization. Second, the Internal OOD scripts are manually designed in-house and judged by a Seed-1.5-VL similarity model (A.5), while the training traces in §3.2.2 were also generated using Seed-1.5-VL. The judge is therefore from the same model family as the teacher, which can systematically favor Robix-style output phrasings. Please report judge-versus-human agreement, provide an independent human evaluation on a held-out set, or release the benchmark so it can be audited.
  3. [§4.3.2, Fig. 6] The integrated-system 4.3 pp advantage is explicitly attributed by the authors to VLM-VLA misalignment: 'baseline methods—particularly GPT-4o—sometimes generate actions that are semantically correct but unrecognizable to the VLA' and 'This VLM–VLA misalignment mainly accounts for the online performance drop observed in Gemini-2.5-Pro and GPT-4o.' Under this explanation, the comparison measures action-format compatibility with GR-3, not the quality of high-level reasoning, planning, or interaction. Therefore Fig. 6 cannot support the claim that Robix-32B surpasses Gemini-2.5-Pro in interactive task execution. To make that claim, the action space should be controlled (e.g., normalizing all baselines to Robix's action vocabulary), or the VLM-only results with adequate statistics should be presented as the primary evidence.
minor comments (3)
  1. [Eq. (1)] The notation $[(o_{n-i}, u_{n-i}, t_{n-i}, a_{n-i}, r_{n-i})]_{i=1}^{N}$ is ambiguous: the text says $N$ is the context-window length, but the subscript/superscript imply $N$ is the number of retained interaction steps. Please clarify the indexing and the relationship between $N$ and the 32k token budget.
  2. [Section 3.1] The stated token counts for the pretraining components (40B + 70B + 10B + 10B + 80B = 210B) are slightly inconsistent with the 'approximately 200 billion tokens' summary. A brief note on rounding or double counting would avoid confusion.
  3. [§3.3, Eq. (2)] In Eq. (2), $y^*_n$ is used as the ground-truth output but is not explicitly defined in the text preceding the equation. Please define it, along with the relationship between the reward $R(y^{(i)}_n, y^*_n)$ and the thought-action consistency reward described later.

Circularity Check

0 steps flagged

No significant circularity: the paper's results are empirical and the in-house evaluation components do not reduce outputs to inputs by construction.

full rationale

Robix is an empirical systems paper: its chain is data synthesis -> three-stage training -> evaluation, not a formal derivation whose predictions are equivalent to its inputs. The closest candidates to circularity are not circular in the required sense. (1) The Internal ID offline set is 'randomly sampled from our synthesized data' (Section 4.2), but the paper explicitly labels it in-distribution and does not use it as evidence of OOD generalization; the AGIBot and Internal OOD sets are manually selected/designed and excluded from training. (2) Offline action accuracy is judged by Seed-1.5-VL ('similarity judged by Seed-1.5-VL', Section 4.2), the same model family used to generate CoT training traces; this is a potential evaluation bias, but the judged quantity is not defined by the training objective and the same judge is applied uniformly to all baselines, so it is not a fitted input renamed as a prediction. (3) The RL thought-action consistency reward uses Qwen-2.5-32B as an external judge, and Robix initializes from Qwen2.5-VL; this is LLM-as-judge, not a self-definitional loop. (4) Citations to GR-3 and Seed-1.5-VL supply internal data, a VLA controller, and pretraining corpora; they are resource citations rather than load-bearing theorems or uniqueness arguments. The online evaluation uses human teleoperation and human task-progress annotation, independent of the training labels. The paper's Limitations section concedes hallucinations, flawed reasoning, and short-term-memory gaps, but no passage asserts or exhibits a step where a claimed prediction is equivalent by construction to a fitted parameter or to a self-cited result. No Eq. X = Eq. Y, no fitted parameter called prediction, and no imported uniqueness theorem were found.

Axiom & Free-Parameter Ledger

4 free parameters · 6 axioms · 0 invented entities

The central scientific contribution is an engineering recipe; the main 'free parameters' are training hyperparameters. The key assumptions are about the quality of the teacher models and the representativeness of the evaluation sets, which are load-bearing because the headline gains over commercial baselines come exclusively from these sets.

free parameters (4)
  • RL variance threshold tau = 0
    Set to zero in Eq. 2, which makes the data-filtering step a no-op for almost all samples; the paper claims a filtering procedure but the stated hyperparameter disables it.
  • RL sample count M = 8
    Number of candidate answers sampled per question in GRPO; an engineering choice without clear sensitivity analysis.
  • CoT trace length cap = 200 tokens
    Robot reasoning traces are capped at 200 tokens to support real-time interaction; the cap shapes the learned reasoning format.
  • Context window N = 3 views (latest N observations retained)
    Only the latest N visual observations are kept as explicit input under a 32k token budget; this is a hand-set tradeoff.
axioms (6)
  • domain assumption Qwen2.5-VL provides a strong base for embodied reasoning and grounding.
    Continued pretraining and SFT are built on Qwen2.5-VL checkpoints; if the base model were weak, the results would not transfer, but no comparison against other bases is given.
  • domain assumption Seed-1.5-VL teacher traces are high-quality and logically consistent.
    Used to generate reasoning traces for task-centric reasoning data and reasoning synthesis (Sections 3.1, 3.2.2); the paper filters them with a model but does not quantify teacher error.
  • domain assumption An external LLM (Qwen-2.5-32B) can reliably judge thought-action consistency.
    Used as the RL reward model; if the judge is biased or noisy, the consistency reward is unreliable (Section 3.3).
  • ad hoc to paper The offline evaluation candidate-action list and the Seed-1.5-VL judge capture all acceptable actions.
    Accuracy is measured by matching the predicted action to a manually annotated candidate list via a VLM judge (Section 4.2, Appendix A.5); mistakes here directly alter all reported accuracy numbers.
  • ad hoc to paper The internal OOD/ID benchmarks are representative of real interactive tasks.
    The in-distribution set is sampled from the synthetic training data, and the OOD scripts are authored by the same team; representativeness is assumed, not externally validated.
  • domain assumption Teacher-forcing evaluation (providing the ground-truth history) yields valid estimates of planning ability.
    Models are evaluated by feeding them the correct prior actions, which removes error recovery behavior from the metric and can inflate plan accuracy (Section 4.2).

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Robix: A Unified Model for Robot Interaction, Reasoning and Planning." pith.science (2026). https://pith.science/paper/AX5VGLO5

@misc{pith2026250901106,
  author       = {Pith},
  title        = {Pith review of: Robix: A Unified Model for Robot Interaction, Reasoning and Planning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AX5VGLO5}},
  note         = {Machine review of arXiv:2509.01106}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

We introduce Robix, a unified model that integrates robot reasoning, task planning, and natural language interaction within a single vision-language architecture. Acting as the high-level cognitive layer in a hierarchical robot system, Robix dynamically generates atomic commands for the low-level controller and verbal responses for human interaction, enabling robots to follow complex instructions, plan long-horizon tasks, and interact naturally with human within an end-to-end framework. Robix further introduces novel capabilities such as proactive dialogue, real-time interruption handling, and context-aware commonsense reasoning during task execution. At its core, Robix leverages chain-of-thought reasoning and adopts a three-stage training strategy: (1) continued pretraining to enhance foundational embodied reasoning abilities including 3D spatial understanding, visual grounding, and task-centric reasoning; (2) supervised finetuning to model human-robot interaction and task planning as a unified reasoning-action sequence; and (3) reinforcement learning to improve reasoning-action consistency and long-horizon task coherence. Extensive experiments demonstrate that Robix outperforms both open-source and commercial baselines (e.g., GPT-4o and Gemini 2.5 Pro) in interactive task execution, demonstrating strong generalization across diverse instruction types (e.g., open-ended, multi-stage, constrained, invalid, and interrupted) and various user-involved tasks such as table bussing, grocery shopping, and dietary filtering.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. MindVLA-U1: VLA Beats VA with Unified Streaming Architecture for Autonomous Driving

    cs.RO 2026-05 unverdicted novelty 7.0

    MindVLA-U1 introduces a unified streaming VLA with shared backbone, framewise memory, and language-guided action diffusion that surpasses human drivers on WOD-E2E planning metrics.

  2. RoboInter1.5: A Holistic Intermediate Representation Suite for Embodied World Modeling and Robotic Manipulation

    cs.RO 2026-07 conditional novelty 6.0

    Dense per-frame intermediate representations (traces, masks, grasp poses, subtasks) improve embodied VQA, VLA action generation, and world-model video prediction in the new 230k-episode RoboInter-Data suite.

  3. Task-Focused Memorization for Multimodal Agents

    cs.CV 2026-05 unverdicted novelty 6.0

    TaskMem uses RL in two phases to learn a task-focused memorization policy for multimodal agents, yielding 5.3-7.0% VQA accuracy gains on reformulated streaming benchmarks from VideoMME, EgoLife, and EgoTempo.

  4. RoboEvolve: Co-Evolving Planner-Simulator for Robotic Manipulation with Limited Data

    cs.RO 2026-05 unverdicted novelty 6.0

    A co-evolutionary VLM-VGM loop on 500 unlabeled images raises planner success by 30 points and simulator success by 48 percent while beating fully supervised baselines.

  5. MindVLA-U1: VLA Beats VA with Unified Streaming Architecture for Autonomous Driving

    cs.RO 2026-05 unverdicted novelty 6.0

    MindVLA-U1 is the first unified streaming VLA architecture that surpasses human drivers on WOD-E2E planning metrics while matching VA latency and preserving language interfaces.

  6. Long-Horizon Manipulation via Trace-Conditioned VLA Planning

    cs.RO 2026-04 unverdicted novelty 6.0

    LoHo-Manip enables robust long-horizon robot manipulation by using a receding-horizon VLM manager to output progress-aware subtask sequences and 2D visual traces that condition a VLA executor for automatic replanning.

  7. ReMoT: Reinforcement Learning with Motion Contrast Triplets

    cs.CV 2026-02 conditional novelty 6.0

    Training a 4B vision-language model on rule-generated motion-contrast triplets with GRPO lifts spatio-temporal QA accuracy by about 17 points on the authors' own benchmark and by smaller margins on standard benchmarks.

  8. Steerable Vision-Language-Action Policies for Embodied Reasoning and Hierarchical Control

    cs.RO 2026-02 unverdicted novelty 6.0

    Steerable VLAs trained on rich synthetic commands at subtask, motion, and pixel levels enable VLMs to steer robot behavior more effectively, outperforming prior hierarchical baselines on real-world manipulation and ge...

  9. RoboAgent: Chaining Basic Capabilities for Embodied Task Planning

    cs.RO 2026-04 unverdicted novelty 5.0

    RoboAgent chains basic vision-language capabilities inside a single VLM via a scheduler and trains it in three stages (behavior cloning, DAgger, RL) to improve embodied task planning.

Reference graph

Works this paper leans on

121 extracted references · 36 canonical work pages · cited by 8 Pith papers · 2 internal anchors

  1. [1]

    Do as i can, not as i say: Grounding language in robotic affordances

    Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Chuyuan Fu, Keerthana Gopalakrishnan, Karol Hausman, et al. Do as i can, not as i say: Grounding language in robotic affordances. arXiv preprint arXiv:2204.01691, 2022

    Pith/arXiv arXiv 2022

  2. [2]

    Cosmos-reason1: From physical common sense to embodied reasoning

    Alisson Azzolini, Junjie Bai, Hannah Brandon, Jiaxin Cao, Prithvijit Chattopadhyay, Huayu Chen, Jinju Chu, Yin Cui, Jenna Diamond, Yifan Ding, et al. Cosmos-reason1: From physical common sense to embodied reasoning. arXiv preprint arXiv:2503.15558, 2025

    Pith/arXiv arXiv 2025

  3. [3]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report.arXiv preprint arXiv:2502.13923, 2025

    Pith/arXiv arXiv 2025

  4. [4]

    ARKitscenes: A diverse real-world dataset for 3d indoor scene understanding using mobile RGB-d data

    Gilad Baruch, Zhuoyuan Chen, Afshin Dehghan, Yuri Feigin, Peter Fu, Thomas Gebauer, Daniel Kurz, Tal Dimry, Brandon Joffe, Arik Schwartz, and Elad Shulman. ARKitscenes: A diverse real-world dataset for 3d indoor scene understanding using mobile RGB-d data. InConference on Neural Information Processing Systems Datasets and Benchmarks Track(Round 1), 2021

    2021

  5. [5]

    Rt-h: Action hierarchies using language.arXiv preprint arXiv:2403.01823, 2024

    Suneel Belkhale, Tianli Ding, Ted Xiao, Pierre Sermanet, Quon Vuong, Jonathan Tompson, et al. Rt-h: Action hierarchies using language.arXiv preprint arXiv:2403.01823, 2024

    Pith/arXiv arXiv 2024

  6. [6]

    Agibot world colosseo: A large-scale manipulation platform for scalable and intelligent embodied systems

    Qingwen Bu, Jisong Cai, Li Chen, Xiuqi Cui, Yan Ding, Siyuan Feng, Shenyuan Gao, Xindong He, Xuan Hu, Xu Huang, et al. Agibot world colosseo: A large-scale manipulation platform for scalable and intelligent embodied systems. arXiv preprint arXiv:2503.06669, 2025

    Pith/arXiv arXiv 2025

  7. [7]

    Gr-3 technical report.arXiv preprint arXiv:2507.15493, 2025

    Chilam Cheang, Sijin Chen, Zhongren Cui, Yingdong Hu, Liqun Huang, Tao Kong, Hang Li, Yifeng Li, Yuxiao Liu, Xiao Ma, et al. Gr-3 technical report.arXiv preprint arXiv:2507.15493, 2025. 17

    Pith/arXiv arXiv 2025

  8. [8]

    Navila: Legged robot vision-language-action model for navigation.arXiv preprint arXiv:2412.04453, 2024

    An-Chieh Cheng, Yandong Ji, Zhaojing Yang, Zaitian Gongye, Xueyan Zou, Jan Kautz, Erdem Bıyık, Hongxu Yin, Sifei Liu, and Xiaolong Wang. Navila: Legged robot vision-language-action model for navigation.arXiv preprint arXiv:2412.04453, 2024

    Pith/arXiv arXiv 2024

  9. [9]

    Simplevqa: Multimodal factuality evaluation for multimodal large language models.arXiv preprint arXiv:2502.13059, 2025

    Xianfu Cheng, Wei Zhang, Shiwei Zhang, Jian Yang, Xiangyuan Guan, Xianjie Wu, Xiang Li, Ge Zhang, Jiaheng Liu, Yuying Mai, et al. Simplevqa: Multimodal factuality evaluation for multimodal large language models.arXiv preprint arXiv:2502.13059, 2025

    Pith/arXiv arXiv 2025

  10. [10]

    Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots

    Cheng Chi, Zhenjia Xu, Chuer Pan, Eric Cousineau, Benjamin Burchfiel, Siyuan Feng, Russ Tedrake, and Shuran Song. Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots. InProceedings of Robotics: Science and Systems (RSS), 2024

    2024

  11. [11]

    One to rule them all: natural language to bind communication, perception and action

    Simone Colombani, Dimitri Ognibene, and Giuseppe Boccignone. One to rule them all: natural language to bind communication, perception and action.arXiv preprint arXiv:2411.15033, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  12. [12]

    Scannet: Richly-annotated 3d reconstructions of indoor scenes

    Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 5828–5839, 2017

    2017

  13. [13]

    Racer: Rich language-guided failure recovery policies for imitation learning

    Yinpei Dai, Jayjun Lee, Nima Fazeli, and Joyce Chai. Racer: Rich language-guided failure recovery policies for imitation learning. arXiv preprint arXiv:2409.14674, 2024

    Pith/arXiv arXiv 2024

  14. [14]

    Molmo and pixmo: Open weights and open data for state-of-the-art multimodal models.arXiv e-prints, pages arXiv–2409, 2024

    Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi, Yue Yang, Jae Sung Park, Mohammadreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, et al. Molmo and pixmo: Open weights and open data for state-of-the-art multimodal models.arXiv e-prints, pages arXiv–2409, 2024

    2024

  15. [15]

    Palm-e: An embodied multimodal language model

    Danny Driess, Fei Xia, Mehdi SM Sajjadi, Corey Lynch, Aakanksha Chowdhery, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, et al. Palm-e: An embodied multimodal language model. 2023

    2023

  16. [16]

    Mengfei Du, Binhao Wu, Zejun Li, Xuanjing Huang, and Zhongyu Wei. Embspatial-bench: Benchmarking spatial understanding for embodied tasks with large vision-language models.Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 2024

    2024

  17. [17]

    Manipulate-anything: Automating real-world robots using vision-language models

    Jiafei Duan, Wentao Yuan, Wilbert Pumacay, Yi Ru Wang, Kiana Ehsani, Dieter Fox, and Ranjay Kr- ishna. Manipulate-anything: Automating real-world robots using vision-language models. arXiv preprint arXiv:2406.18915, 2024

    Pith/arXiv arXiv 2024

  18. [18]

    Mme-survey: A comprehensive survey on evaluation of multimodal llms.arXiv preprint arXiv:2411.15296, 2024

    Chaoyou Fu, Yi-Fan Zhang, Shukang Yin, Bo Li, Xinyu Fang, Sirui Zhao, Haodong Duan, Xing Sun, Ziwei Liu, Liang Wang, et al. Mme-survey: A comprehensive survey on evaluation of multimodal llms.arXiv preprint arXiv:2411.15296, 2024

    Pith/arXiv arXiv 2024

  19. [19]

    Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis

    Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis. InProceedings of the Computer Vision and PatternRecognition Conference, pages 24108–24118, 2025

    2025

  20. [20]

    Blink: Multimodal large language models can see but not perceive

    Xingyu Fu, Yushi Hu, Bangzheng Li, Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth, Noah A Smith, Wei-Chiu Ma, and Ranjay Krishna. Blink: Multimodal large language models can see but not perceive. InEuropean Conference on Computer Vision, pages 148–166. Springer, 2024

    2024

  21. [21]

    Seedream 2.0: A native chinese-english bilingual image generation foundation model, 2025

    Lixue Gong, Xiaoxia Hou, Fanshi Li, Liang Li, Xiaochen Lian, Fei Liu, Liyang Liu, Wei Liu, Wei Lu, Yichun Shi, Shiqi Sun, Yu Tian, Zhi Tian, Peng Wang, Xun Wang, Ye Wang, Guofeng Wu, Jie Wu, Xin Xia, Xuefeng Xiao, Linjie Yang, Zhonghua Zhai, Xinyu Zhang, Qi Zhang, Yuwei Zhang, Shijia Zhao, Jianchao Yang, and Weilin Huang. Seedream 2.0: A native chinese-en...

    2025

  22. [22]

    Ego4d: Around the world in 3,000 hours of egocentric video

    Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, et al. Ego4d: Around the world in 3,000 hours of egocentric video. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 18995–19012, 2022

    2022

  23. [23]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    Pith/arXiv arXiv 2025

  24. [24]

    Dong Guo, Faming Wu, Feida Zhu, Fuxing Leng, Guang Shi, Haobin Chen, Haoqi Fan, Jian Wang, Jianyu Jiang, Jiawei Wang, et al. Seed1. 5-vl technical report.arXiv preprint arXiv:2505.07062, 2025. 18

    Pith/arXiv arXiv 2025

  25. [25]

    Lvis: A dataset for large vocabulary instance segmentation

    Agrim Gupta, Piotr Dollar, and Ross Girshick. Lvis: A dataset for large vocabulary instance segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5356–5364, 2019

    2019

  26. [26]

    Glm-4.1 v-thinking: Towards versatile multimodal reasoning with scalable reinforcement learning

    Wenyi Hong, Wenmeng Yu, Xiaotao Gu, Guo Wang, Guobing Gan, Haomiao Tang, Jiale Cheng, Ji Qi, Junhui Ji, Lihang Pan, et al. Glm-4.1 v-thinking: Towards versatile multimodal reasoning with scalable reinforcement learning. arXiv preprint arXiv:2507.01006, 2025

    Pith/arXiv arXiv 2025

  27. [27]

    Egodex: Learning dexterous manipulation from large-scale egocentric video.arXiv preprint arXiv:2505.11709, 2025

    Ryan Hoque, Peide Huang, David J Yoon, Mouli Sivapurapu, and Jian Zhang. Egodex: Learning dexterous manipulation from large-scale egocentric video.arXiv preprint arXiv:2505.11709, 2025

    Pith/arXiv arXiv 2025

  28. [28]

    Look before you leap: Unveiling the power of gpt-4v in robotic vision-language planning.arXiv preprint arXiv:2311.17842, 2023

    Yingdong Hu, Fanqi Lin, Tong Zhang, Li Yi, and Yang Gao. Look before you leap: Unveiling the power of gpt-4v in robotic vision-language planning.arXiv preprint arXiv:2311.17842, 2023

    Pith/arXiv arXiv 2023

  29. [29]

    Inner monologue: Embodied reasoning through planning with language models

    Wenlong Huang, Fei Xia, Ted Xiao, Harris Chan, Jacky Liang, Pete Florence, Andy Zeng, Jonathan Tompson, Igor Mordatch, Yevgen Chebotar, et al. Inner monologue: Embodied reasoning through planning with language models. arXiv preprint arXiv:2207.05608, 2022

    Pith/arXiv arXiv 2022

  30. [30]

    Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    Pith/arXiv arXiv 2024

  31. [31]

    Egotaskqa: Understanding human tasks in egocentric videos

    Baoxiong Jia, Ting Lei, Song-Chun Zhu, and Siyuan Huang. Egotaskqa: Understanding human tasks in egocentric videos. Advances in Neural Information Processing Systems, 35:3343–3360, 2022

    2022

  32. [32]

    Droid: A large-scale in-the-wild robot manipulation dataset.arXiv preprint arXiv:2403.12945, 2024

    Alexander Khazatsky, Karl Pertsch, Suraj Nair, Ashwin Balakrishna, Sudeep Dasari, Siddharth Karamcheti, Soroush Nasiriany, Mohan Kumar Srirama, Lawrence Yunliang Chen, Kirsty Ellis, et al. Droid: A large-scale in-the-wild robot manipulation dataset.arXiv preprint arXiv:2403.12945, 2024

    Pith/arXiv arXiv 2024

  33. [33]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. InInternational Conference on Learning Representations, 2015

    2015

  34. [34]

    ERQA: Edge-Restoration Quality Assessment for Video Super-Resolution

    Anastasia Kirillova, Eugene Lyapustin, Anastasia Antsiferova, and Dmitry Vatolin. Erqa: Edge-restoration quality assessment for video super-resolution.arXiv preprint arXiv:2110.09992, 2021

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  35. [35]

    Cubify anything: Scaling indoor 3d object detection

    Justin Lazarow, David Griffiths, Gefen Kohavi, Francisco Crespo, and Afshin Dehghan. Cubify anything: Scaling indoor 3d object detection. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 22225–22233, 2025

    2025

  36. [36]

    Evolvenav: Self-improving embodied reasoning for llm-based vision-language navigation

    Bingqian Lin, Yunshuang Nie, Khun Loun Zai, Ziming Wei, Mingfei Han, Rongtao Xu, Minzhe Niu, Jianhua Han, Liang Lin, Cewu Lu, et al. Evolvenav: Self-improving embodied reasoning for llm-based vision-language navigation. arXiv preprint arXiv:2506.01551, 2025

    arXiv 2025

  37. [37]

    Visual spatial reasoning.Transactions of the Association for Computational Linguistics, 11:635–651, 2023

    Fangyu Liu, Guy Emerson, and Nigel Collier. Visual spatial reasoning.Transactions of the Association for Computational Linguistics, 11:635–651, 2023

    2023

  38. [38]

    Interactive robot learning from verbal correction.arXiv preprint arXiv:2310.17555, 2023

    Huihan Liu, Alice Chen, Yuke Zhu, Adith Swaminathan, Andrey Kolobov, and Ching-An Cheng. Interactive robot learning from verbal correction.arXiv preprint arXiv:2310.17555, 2023

    Pith/arXiv arXiv 2023

  39. [39]

    Visualwebbench: How far have multimodal llms evolved in web page understanding and grounding?arXivpreprintarXiv:2404.05955, 2024

    Junpeng Liu, Yifan Song, Bill Yuchen Lin, Wai Lam, Graham Neubig, Yuanzhi Li, and Xiang Yue. Visualwebbench: How far have multimodal llms evolved in web page understanding and grounding?arXivpreprintarXiv:2404.05955, 2024

    Pith/arXiv arXiv 2024

  40. [40]

    Mmbench: Is your multi-modal model an all-around player? InEuropean conference on computer vision, pages 216–233

    Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. Mmbench: Is your multi-modal model an all-around player? InEuropean conference on computer vision, pages 216–233. Springer, 2024

    2024

  41. [41]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. InInternationalConference on Learning Representations, 2019

    2019

  42. [42]

    Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts

    Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. arXiv preprint arXiv:2310.02255, 2023

    Pith/arXiv arXiv 2023

  43. [43]

    Openeqa: Embodied question answering in the era of foundation models

    Arjun Majumdar, Anurag Ajay, Xiaohan Zhang, Pranav Putta, Sriram Yenamandra, Mikael Henaff, Sneha Silwal, Paul Mcvay, Oleksandr Maksymets, Sergio Arnaud, et al. Openeqa: Embodied question answering in the era of foundation models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16488–16498, 2024. 19

    2024

  44. [44]

    Egoschema: A diagnostic benchmark for very long-form video language understanding.Advances in Neural Information Processing Systems, 36:46212–46244, 2023

    Karttikeya Mangalam, Raiymbek Akshulakov, and Jitendra Malik. Egoschema: A diagnostic benchmark for very long-form video language understanding.Advances in Neural Information Processing Systems, 36:46212–46244, 2023

    2023

  45. [45]

    Learning to parse natural language commands to a robot control system

    Cynthia Matuszek, Evan Herbst, Luke Zettlemoyer, and Dieter Fox. Learning to parse natural language commands to a robot control system. InExperimental robotics: the 13th international symposium on experimental robotics, pages 403–415. Springer, 2013

    2013

  46. [46]

    Is feedback all you need? leveraging natural language feedback in goal-conditioned rl

    Sabrina McCallum, Max Taylor-Davies, Stefano Albrecht, and Alessandro Suglia. Is feedback all you need? leveraging natural language feedback in goal-conditioned rl. InNeurIPS 2023 Workshop on Goal-Conditioned Reinforcement Learning, 2023

    2023

  47. [47]

    Replanvlm: Replanning robotic tasks with visual language models

    Aoran Mei, Guo-Niu Zhu, Huaxiang Zhang, and Zhongxue Gan. Replanvlm: Replanning robotic tasks with visual language models. IEEE Robotics and Automation Letters, 2024

    2024

  48. [48]

    Learning neuro-symbolic programs for language guided robot manipulation

    K Namasivayam, Himanshu Singh, Vishal Bindal, Arnav Tuli, Vishwajeet Agrawal, Rahul Jain, Parag Singla, and Rohan Paul. Learning neuro-symbolic programs for language guided robot manipulation. In2023 IEEE International Conference on Robotics and Automation (ICRA), pages 7973–7980. IEEE, 2023

    2023

  49. [49]

    Inferring compact representations for efficient natural language understanding of robot instructions

    Siddharth Patki, Andrea F Daniele, Matthew R Walter, and Thomas M Howard. Inferring compact representations for efficient natural language understanding of robot instructions. In2019 International Conference on Robotics and Automation (ICRA), pages 6926–6933. IEEE, 2019

    2019

  50. [50]

    Ui-tars: Pioneering automated gui interaction with native agents

    Yujia Qin, Yining Ye, Junjie Fang, Haoming Wang, Shihao Liang, Shizuo Tian, Junda Zhang, Jiahao Li, Yunxin Li, Shijue Huang, et al. Ui-tars: Pioneering automated gui interaction with native agents. arXiv preprint arXiv:2501.12326, 2025

    Pith/arXiv arXiv 2025

  51. [51]

    Sat: Dynamic spatial aptitude training for multimodal language models

    Arijit Ray, Jiafei Duan, Ellis Brown, Reuben Tan, Dina Bashkirova, Rose Hendrix, Kiana Ehsani, Aniruddha Kembhavi, Bryan A Plummer, Ranjay Krishna, et al. Sat: Dynamic spatial aptitude training for multimodal language models. arXiv preprint arXiv:2412.07755, 2024

    arXiv 2024

  52. [52]

    Robovqa: Multimodal long-horizon reasoning for robotics

    Pierre Sermanet, Tianli Ding, Jeffrey Zhao, Fei Xia, Debidatta Dwibedi, Keerthana Gopalakrishnan, Christine Chan, Gabriel Dulac-Arnold, Sharath Maddineni, Nikhil J Joshi, et al. Robovqa: Multimodal long-horizon reasoning for robotics. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 645–652. IEEE, 2024

    2024

  53. [53]

    Bumble: Unifying reasoning and acting with vision-language models for building-wide mobile manipulation.arXiv preprint arXiv:2410.06237, 2024

    Rutav Shah, Albert Yu, Yifeng Zhu, Yuke Zhu, and Roberto Martín-Martín. Bumble: Unifying reasoning and acting with vision-language models for building-wide mobile manipulation.arXiv preprint arXiv:2410.06237, 2024

    Pith/arXiv arXiv 2024

  54. [54]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024

    Pith/arXiv arXiv 2024

  55. [55]

    Hybridflow: A flexible and efficient rlhf framework.arXiv preprint arXiv: 2409.19256, 2024

    Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework.arXiv preprint arXiv: 2409.19256, 2024

    Pith/arXiv arXiv 2024

  56. [56]

    Yell at your robot: Improving on-the-fly from language corrections.arXiv preprint arXiv:2403.12910, 2024

    Lucy Xiaoyang Shi, Zheyuan Hu, Tony Z Zhao, Archit Sharma, Karl Pertsch, et al. Yell at your robot: Improving on-the-fly from language corrections.arXiv preprint arXiv:2403.12910, 2024

    Pith/arXiv arXiv 2024

  57. [57]

    Hi robot: Open-ended instruction following with hierarchical vision-language-action models

    Lucy Xiaoyang Shi, brian ichter, Michael Robert Equi, Liyiming Ke, Karl Pertsch, Quan Vuong, James Tanner, Anna Walling, Haohuan Wang, Niccolo Fusai, Adrian Li-Bell, Danny Driess, Lachy Groom, Sergey Levine, and Chelsea Finn. Hi robot: Open-ended instruction following with hierarchical vision-language-action models. In International Conference on Machine ...

    2025

  58. [58]

    Lgr2: Language guided reward relabeling for accelerating hierarchical reinforcement learning.arXiv preprint arXiv:2406.05881, 2024

    Utsav Singh, Pramit Bhattacharyya, and Vinay P Namboodiri. Lgr2: Language guided reward relabeling for accelerating hierarchical reinforcement learning.arXiv preprint arXiv:2406.05881, 2024

    Pith/arXiv arXiv 2024

  59. [59]

    Replan: Robotic replanning with perception and language models.arXiv preprint arXiv:2401.04157, 2024

    Marta Skreta, Zihan Zhou, Jia Lin Yuan, Kourosh Darvish, Alán Aspuru-Guzik, and Animesh Garg. Replan: Robotic replanning with perception and language models.arXiv preprint arXiv:2401.04157, 2024

    Pith/arXiv arXiv 2024

  60. [60]

    Llm-planner: Few-shot grounded planning for embodied agents with large language models

    Chan Hee Song, Jiaman Wu, Clayton Washington, Brian M Sadler, Wei-Lun Chao, and Yu Su. Llm-planner: Few-shot grounded planning for embodied agents with large language models. InProceedings of the IEEE/CVF international conference on computer vision, pages 2998–3009, 2023

    2023

  61. [61]

    Sun rgb-d: A rgb-d scene understanding benchmark suite

    Shuran Song, Samuel P Lichtenberg, and Jianxiong Xiao. Sun rgb-d: A rgb-d scene understanding benchmark suite. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 567–576, 2015. 20

    2015

  62. [62]

    A computational model for the alignment of hierarchical scene representations in human-robot interaction

    Agnes Swadzba, Sven Wachsmuth, Constanze Vorwerg, and Gert Rickheit. A computational model for the alignment of hierarchical scene representations in human-robot interaction. InIJCAI, pages 1857–1863, 2009

    2009

  63. [63]

    Robobrain 2.0 technical report.arXiv preprint arXiv:2507.02029, 2025

    BAAI RoboBrain Team, Mingyu Cao, Huajie Tan, Yuheng Ji, Minglan Lin, Zhiyu Li, Zhou Cao, Pengwei Wang, Enshen Zhou, Yi Han, et al. Robobrain 2.0 technical report.arXiv preprint arXiv:2507.02029, 2025

    Pith/arXiv arXiv 2025

  64. [64]

    Gemini robotics: Bringing ai into the physical world.arXiv preprint arXiv:2503.20020, 2025

    Gemini Robotics Team, Saminda Abeyruwan, Joshua Ainslie, Jean-Baptiste Alayrac, Montserrat Gonzalez Arenas, Travis Armstrong, Ashwin Balakrishna, Robert Baruch, Maria Bauza, Michiel Blokzijl, et al. Gemini robotics: Bringing ai into the physical world.arXiv preprint arXiv:2503.20020, 2025

    Pith/arXiv arXiv 2025

  65. [65]

    Cambrian-1: A fully open, vision-centric exploration of multimodal llms.Advances in Neural Information Processing Systems, 37:87310–87356, 2024

    Peter Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Adithya Jairam Vedagiri IYER, Sai Charitha Akula, Shusheng Yang, Jihan Yang, Manoj Middepogu, Ziteng Wang, et al. Cambrian-1: A fully open, vision-centric exploration of multimodal llms.Advances in Neural Information Processing Systems, 37:87310–87356, 2024

    2024

  66. [66]

    Rio: 3d object instance re-localization in changing indoor environments

    Johanna Wald, Armen Avetisyan, Nassir Navab, Federico Tombari, and Matthias Nießner. Rio: 3d object instance re-localization in changing indoor environments. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 7658–7667, 2019

    2019

  67. [67]

    Bridgedata v2: A dataset for robot learning at scale

    Homer Rich Walke, Kevin Black, Tony Z Zhao, Quan Vuong, Chongyi Zheng, Philippe Hansen-Estruch, An- dre Wang He, Vivek Myers, Moo Jin Kim, Max Du, et al. Bridgedata v2: A dataset for robot learning at scale. In Conference on Robot Learning, pages 1723–1736. PMLR, 2023

    2023

  68. [68]

    Vlm see, robot do: Human demo video to robot action plan via vision language model.arXiv preprint arXiv:2410.08792, 2024

    Beichen Wang, Juexiao Zhang, Shuwen Dong, Irving Fang, and Chen Feng. Vlm see, robot do: Human demo video to robot action plan via vision language model.arXiv preprint arXiv:2410.08792, 2024

    arXiv 2024

  69. [69]

    Measuring multimodal mathematical reasoning with math-vision dataset.Advances in Neural Information Processing Systems, 37:95095–95169, 2024

    Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Houxing Ren, Aojun Zhou, Mingjie Zhan, and Hongsheng Li. Measuring multimodal mathematical reasoning with math-vision dataset.Advances in Neural Information Processing Systems, 37:95095–95169, 2024

    2024

  70. [70]

    Holoassist: an egocentric human interaction dataset for interactive ai assistants in the real world

    Xin Wang, Taein Kwon, Mahdi Rad, Bowen Pan, Ishani Chakraborty, Sean Andrist, Dan Bohus, Ashley Feniello, Bugra Tekin, Felipe Vieira Frujeri, et al. Holoassist: an egocentric human interaction dataset for interactive ai assistants in the real world. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 20270–20281, 2023

    2023

  71. [71]

    Realworldqa: A benchmark for real-world spatial understanding

    xAI. Realworldqa: A benchmark for real-world spatial understanding. 2024. URLhttps://huggingface.co/ datasets/xai-org/RealworldQA

    2024

  72. [72]

    Robi butler: Remote multimodal interactions with household robot assistant.arXiv e-prints, pages arXiv–2409, 2024

    Anxing Xiao, Nuwan Janaka, Tianrun Hu, Anshul Gupta, Kaixin Li, Cunjun Yu, and David Hsu. Robi butler: Remote multimodal interactions with household robot assistant.arXiv e-prints, pages arXiv–2409, 2024

    2024

  73. [73]

    Next-qa: Next phase of question-answering to explaining temporal actions

    Junbin Xiao, Xindi Shang, Angela Yao, and Tat-Seng Chua. Next-qa: Next phase of question-answering to explaining temporal actions. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9777–9786, 2021

    2021

  74. [74]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report.arXiv preprint arXiv:2412.15115, 2024

    Pith/arXiv arXiv 2024

  75. [75]

    Thinking in space: How multimodal large language models see, remember, and recall spaces

    Jihan Yang, Shusheng Yang, Anjali W Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. Thinking in space: How multimodal large language models see, remember, and recall spaces. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 10632–10643, 2025

    2025

  76. [76]

    Depth anything v2

    Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything v2. Advances in Neural Information Processing Systems, 37:21875–21911, 2024

    2024

  77. [77]

    Guiding long-horizon task and motion planning with vision language models.arXiv preprint arXiv:2410.02193, 2024

    Zhutian Yang, Caelan Garrett, Dieter Fox, Tomás Lozano-Pérez, and Leslie Pack Kaelbling. Guiding long-horizon task and motion planning with vision language models.arXiv preprint arXiv:2410.02193, 2024

    Pith/arXiv arXiv 2024

  78. [78]

    React meets actre: Autonomous annotation of agent trajectories for contrastive self-training

    Zonghan Yang, Peng Li, Ming Yan, Ji Zhang, Fei Huang, and Yang Liu. React meets actre: Autonomous annotation of agent trajectories for contrastive self-training. InFirst Conference on Language Modeling, 2024

    2024

  79. [79]

    Scannet++: A high-fidelity dataset of 3d indoor scenes

    Chandan Yeshwanth, Yueh-Cheng Liu, Matthias Nießner, and Angela Dai. Scannet++: A high-fidelity dataset of 3d indoor scenes. InProceedings of the IEEE/CVF International Conference on Computer Vision, 2023

    2023

  80. [80]

    Modeling context in referring expressions

    Licheng Yu, Patrick Poirson, Shan Yang, Alexander C Berg, and Tamara L Berg. Modeling context in referring expressions. In European conference on computer vision, pages 69–85. Springer, 2016. 21

    work page 2016

Showing first 80 references.

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.