Pith. sign in

REVIEW 2 major objections 6 minor 1 cited by

MobileManiBench is a simulation-generated benchmark with 300K trajectories meant to verify mobile-manipulation VLA models before real-world deployment.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 04:17 UTC pith:VBD7YZVY

load-bearing objection A substantial new mobile-manipulation benchmark with real utility; the sim-to-real verification claim is the one part that outruns the evidence. the 2 major comments →

arxiv 2602.05233 v2 pith:VBD7YZVY submitted 2026-02-05 cs.RO

MobileManiBench: Simplifying Model Verification for Mobile Manipulation

Wenbo Wang , Fangyun Wei , QiXiu Li , Xi Chen , Yaobo Liang , Chang Xu , Jiaolong Yang , Baining Guo This is my paper
classification cs.RO
keywords mobile manipulationvision-language-action modelssimulation benchmarkRL trajectory generationdexterous manipulationsim-to-real transfermulti-view RGB-Drobot learning dataset
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that a large, purely simulated benchmark can substitute for expensive real-robot data collection when developing vision-language-action (VLA) models for mobile manipulation. It introduces MobileManiBench, generated by reinforcement-learning policies rather than teleoperation, covering two mobile robots — one with a parallel gripper and one with a 12-DOF dexterous hand — 630 objects across 20 categories, five manipulation skills, and 100 scenes, yielding 300K annotated trajectories with language, multi-view RGB-depth-segmentation, states, and actions. The authors then train a universal VLA model on this data and show it generalizes to unseen objects and scenes, while also benchmarking existing VLA models to make the testbed useful for architecture verification. The empirical takeaways — multi-view RGB-D matters, unseen objects are harder than unseen scenes, base mobility is essential — are the controlled results the benchmark is designed to deliver. The advertised purpose is to let researchers decide which architectures, sensors, and embodiments are worth transferring to the real world.

Core claim

The paper's central claim is that a keypoint-parameterized RL teacher can generate a large, diverse, richly annotated dataset for mobile manipulation without teleoperation, and that this dataset is sufficient to train and evaluate universal vision-language-action (VLA) policies. For each of 1,182 robot-object-skill combinations, a single state-based policy learns to move hand points to an object grasp point and then to a goal point under one shared reward with a 5 cm success threshold; deployed in 100 scenes, these policies produce 300K successful trajectories with language, multi-view RGB-D-segmentation images, states, and actions. A VLA model trained on the data reaches 56.7% (gripper robo

What carries the argument

The key mechanism is the keypoint-based universal reward: every task is reduced to three point sets — robot gripper/hand points, the object grasp point, and the goal point — so that one reward formula (distance, approach, grasp, move, success) trains every robot-object-skill combination. This abstraction makes 1,182 combinations trainable without per-task rewards, which is what allows the benchmark's data generation to scale. The second mechanism is the action-chunking diffusion-transformer policy that consumes multi-view RGB-D and wrist-pose state and predicts 16-step action chunks; ablations show that this multi-modal conditioning is the main driver of generalization from the state-based t

Load-bearing premise

The load-bearing premise is that simulation physics, rendered sensors, and the 5 cm success threshold predict real-world mobile-manipulation performance; the paper's only current evidence is one task — opening a laptop — with 40% success over 10 trials.

What would settle it

Deploy the same policies on a physical mobile manipulator across all five skills and compare real success rates with simulated rates. If a model with high simulated success consistently underperforms one with lower simulated success — or if the 5 cm threshold is met in simulation yet the object is not functionally opened, closed, pulled, pushed, or picked in reality — the verification claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Researchers can use MobileManiBench to compare VLA architectures, sensor suites, and robot embodiments under standardized conditions without collecting teleoperated data.
  • The 300K-trajectory dataset with synchronized language, multi-view RGB-D-segmentation, states, and actions supports training universal mobile-manipulation policies that generalize to unseen objects and scenes.
  • Multi-view, multi-modal perception is required for competitive mobile manipulation: head-view RGB alone gives 7.9% success on the challenge subset, while head+wrist RGB-D plus wrist pose gives 28.2%.
  • Base mobility is a necessary part of the task: fixing the base drops the state-based teacher from 82.8% to 25.4% success.
  • Existing vision-language-action models can be fine-tuned and ranked within a single evaluation protocol, giving the field a common yardstick for mobile-manipulation progress.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the sim-to-real fidelity holds beyond the single laptop task, the pipeline's real payoff is cheap embodiment studies: changing the hand or adding a camera means retraining an RL teacher, not re-collecting human teleoperation data.
  • The 5 cm keypoint-distance success criterion measures pose error, not function; a door brought within 5 cm of closed may still be unlatched, so a functional success metric could change model rankings.
  • The appendix's 40% success on one real-world task (10 trials) suggests a substantial sim-to-real gap; a public leaderboard of real-robot transfer results across the five skills would turn the benchmark from a simulation testbed into a true verification platform.
  • Because unseen objects cost more performance than unseen scenes, the data-generation strategy should probably weight object and articulation diversity over scene variety — a testable prediction given the benchmark's held-out splits.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. MobileManiBench is a simulation benchmark and dataset for mobile manipulation built in NVIDIA Isaac Sim. It provides two robot embodiments (G1 with parallel gripper, XHand with 12-DOF dexterous hand), 630 objects in 20 categories, five skills, and 100 scenes. The authors train per-robot-object-skill RL policies and use them to generate 300K annotated trajectories with language instructions, multi-view RGB-D and segmentation, states, and actions. They introduce MobileManiVLA, a PaliGemma-2/Gemma-2 + DiT model, train it per robot on the generated data, and evaluate on unseen objects/scenes, reporting sim success rates, ablations, and comparisons with OpenVLA, CogACT, π0, and π0.5. One real-world deployment pilot (opening a laptop on the G1 robot) is reported in Appendix C with 10 trials and 40% success.

Significance. I see the contribution as potentially significant: the field lacks large, publicly available simulation benchmarks for mobile platforms with articulated objects and dexterous hands, and a scalable RL-based data generation pipeline with released code/data/models would be a reusable asset. The paper's internal evaluations are largely consistent in direction (multi-view helps, state helps, unseen-object generalization harder than unseen-scene), and the train/test splits are object- and scene-based, which is a genuine generalization test. The main weakness is that the advertised 'verify before real-world deployment' claim is not supported by a single 10-trial pilot. With the claims scoped to a simulation benchmark plus a pilot sanity check, this could be a strong contribution.

major comments (2)
  1. [Abstract; §1; Appendix C] The central framing of the paper is that MobileManiBench enables verification of VLA architectures 'before real-world deployment.' The only real-world evidence is Appendix C: one task (open laptop), 10 trials, 40% success. For 10 trials the 95% binomial CI is roughly 12-74%, so the simulated 40.8% success rate (Table 10) and the real-world 40% success rate are statistically indistinguishable, and no rank-order predictive claim can be validated. Moreover, the real-world success criterion is not defined and is not aligned with the 5 cm keypoint success threshold used in simulation (Eq. 10, A.2). This overclaim is load-bearing because the abstract and introduction motivate the benchmark as a verification framework. Please either provide multi-task, higher-n real-world validation with a defined success metric, or revise the central claim to describe a sim-only benchmark with a pilot sanity c
  2. [Table 3] The 'Mean' row of Table 3 is not the arithmetic mean of the five skill rows. For the G1 robot, (86.6 + 96.2 + 80.8 + 93.1 + 66.4) / 5 = 84.6%, not 89.6%; for XHand, (91.9 + 96.5 + 97.3 + 97.2 + 72.6) / 5 = 91.1%, not 92.9%. If the reported mean is weighted by episodes or by robot-object-skill combinations, the weighting must be stated. As printed, the table supports Takeaway 3 and the headline RL numbers poorly and misleads readers.
minor comments (6)
  1. [A.3, Eq. (4)] The grasp flag is defined with threshold λ_c = 0.1 in the text, but Eq. (4) uses λ_g. Use consistent notation.
  2. [§3; Appendix C.1] The AgiBot robot reference appears as '[?]' in two places. A proper citation is needed.
  3. [§4.2; Abstract] The abstract states 300K trajectories, while §4.2 says each robot has 15,232 train/test combinations with 10 trajectories each, resulting in 150K training trajectories. Clarify that 150K is per robot and 300K is the total across both robots.
  4. [§5.2; Tables 4-6; Table 10] VLA success rates are based on 10 episodes per robot-object-skill-scene combination. Category-level numbers in Table 10 are therefore coarse (e.g., 40.8% corresponds to roughly 4 successes per 10 episodes). Reporting 95% confidence intervals, or at least the number of episodes underlying each aggregate, would make the model comparisons in Table 6 more interpretable.
  5. [Appendix C] The real-world model is described as the 'RGB only' variant, citing the 40.8% open-laptop result in Table 10, but the real-world deployment also feeds wrist-pose state. Table 10 appears to vary only image inputs; specify whether the sim reference number includes the wrist-pose state, and define how real-world success was judged.
  6. [Throughout] Typos: 'Uneen' in Table 5 header, 'utlizes' in A.3, 'vallina' in B.3, 'usded' in C.2, and 'dependents' in §1.

Circularity Check

0 steps flagged

No significant circularity; benchmark construction is self-consistent and only a minor non-load-bearing self-citation is present.

full rationale

MobileManiBench is a dataset/benchmark construction paper rather than a derivation. The RL reward (Eqs. 1 and 10) uses the 5 cm keypoint threshold to define task success and to filter which trajectories are recorded (Secs. 4.1–4.2), and the VLA evaluation (Sec. 5.1, Table 3) uses the same threshold. This alignment is task conditioning, not circularity: at inference MobileManiVLA receives only language, multi-view RGB-D, and wrist pose, not the privileged object-grasp/goal keypoint state used by the reward, so its success measures generalization under the same task definition. No fitted parameter is later renamed as a prediction. The single real-world probe (Appendix C: one task, 10 trials, 40% success) is an external datapoint, not a parameter fitted back into the simulator; the paper explicitly states its real-world inferences 'focus exclusively on the open laptop task' (App. C.1). The self-citation to CogACT [23] is used for the MobileManiVLA architecture ('Following the design of CogACT [23]', Sec. 4.3), but this is a design choice rather than a load-bearing uniqueness theorem or evidence chain, and it does not force the benchmark results. Thus the central construction is self-contained; the main risk is sim-to-real validity, which is an evidence limitation, not a circular derivation.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 0 invented entities

All quantities needed to reproduce the central numbers are listed above. The benchmark relies on a fixed success criterion (5 cm keypoint threshold) and on simulation fidelity; the only real-world evidence of that fidelity is a single 10-trial task. No new physical entities are introduced.

free parameters (5)
  • Reward weights ω_d, ω_a, ω_g, ω_m, ω_s = 1.0, 0.2, 1.0, 0.2, 2.0
    Equations (3)-(10): hand-set weights balancing approach, move, and success rewards; they determine which trajectories the RL policy discovers, and hence the content of the dataset.
  • Grasp/success thresholds λ_c=0.1, λ_g=0.05 (m) = 0.1 chamfer; 0.05 m
    Eq. (4), Eq. (10), A.2: these thresholds define what counts as a grasp and a successful manipulation, so they set both the positive examples in the dataset and the evaluation metric.
  • Goal-state definitions (60% open for open skill; 40%-80% open initialization for close) = 60%; 40-80%
    Appendix A.2: task construction choices that determine the distribution of goal points and therefore what the VLA must learn to achieve.
  • Trajectory horizon T=300 steps and 10 trajectories per training combination = T=300; 10
    A.3/4.2: these limits cap the dataset size and define the RL success window; also affect episode length distribution in the data.
  • VLA inference window K=4 and action chunk N=16 = K=4, N=16
    Section 4.3: architectural parameters of MobileManiVLA; influence action smoothness and control frequency in all reported VLA numbers.
axioms (4)
  • domain assumption The 5 cm keypoint-distance success metric is a valid proxy for 'task correctly performed' across all five skills and 630 objects.
    Defined in A.2 and encoded in the reward (Eq. 10); the benchmark's evaluation, dataset positivity labels, and real-world transfer claims all inherit this geometric proxy.
  • domain assumption Isaac Sim with the converted URDF/USD assets and Genie/Isaac scene assets is dynamically and visually faithful enough that policies trained on it transfer to the physical G1/XHand robots.
    Stated as the motivation in Abstract/Section 1; only tested on one task ('open laptop', 40% success over 10 trials, Appendix C).
  • standard math PPO with the given input features, reward shaping, and 4K iterations obtains adequate policies for each robot-object-skill combination.
    Section 4.1/5: RL is standard, but the 89.6%/92.9% success rates are empirical outcomes of a specific training setup, not a proof of optimality.
  • domain assumption The data-generation procedure of running the RL policies in scenes and recording 10 successful trajectories per training combination yields a diverse-enough dataset for VLA training.
    Section 4.2: the policy-conditioned data distribution determines what MobileManiVLA learns; no diversity statistics (e.g., pose/object variance) are reported.

pith-pipeline@v1.3.0-alltime-deepseek · 20626 in / 15261 out tokens · 146368 ms · 2026-08-03T04:17:54.482166+00:00 · methodology

0 comments
read the original abstract

Vision-language-action models have advanced robotic manipulation but remain constrained by reliance on the large, teleoperation-collected datasets dominated by the static, tabletop scenes. We propose a simulation-first framework to verify VLA architectures before real-world deployment and introduce MobileManiBench, a large-scale benchmark for mobile-based robotic manipulation. Built on NVIDIA Isaac Sim and powered by reinforcement learning, our pipeline autonomously generates diverse manipulation trajectories with rich annotations (language instructions, multi-view RGB-depth-segmentation images, synchronized object/robot states and actions). MobileManiBench features 2 mobile platforms (parallel-gripper and dexterous-hand robots), 2 synchronized cameras (head and right wrist), 630 objects in 20 categories, 5 skills (open, close, pull, push, pick) with over 100 tasks performed in 100 realistic scenes, yielding 300K trajectories. This design enables controlled, scalable studies of robot embodiments, sensing modalities, and policy architectures, accelerating research on data efficiency and generalization. We benchmark representative VLA models and report insights into perception, reasoning, and control in complex simulated environments, with all code, datasets, and models publicly released.

Figures

Figures reproduced from arXiv: 2602.05233 by Baining Guo, Chang Xu, Fangyun Wei, Jiaolong Yang, Qixiu Li, Wenbo Wang, Xi Chen, Yaobo Liang.

Figure 1
Figure 1. Figure 1: Overview of MobileManiBench. It features 2 mobile-based robots: the G1 robot with a parallel gripper and the XHand robot with a dexterous hand. The benchmark includes 630 articulated and holistic objects across 20 categories and supports 5 mobile manipulation skills—open, close, pull, push, and pick—enabling over 100 tasks. To efficiently scale data generation while ensuring task success, we train a univer… view at source ↗
Figure 2
Figure 2. Figure 2: Definitions of the robot gripper/hand points (blue), ob [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Illustrations of simplified scenes for MobileManiRL [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Success rates of MobileManiRL and MobileManiVLA on the G1 robot and XHand robot across 20 object categories and 5 [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Initialization of the robot, object, ground, and table. [PITH_FULL_IMAGE:figures/full_fig_p012_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: MobileManiBench object distribution. A.4. MobileManiDataset Generation MobileManiBench generates the MobileManiDataset by rendering MobileManiRL in diverse realistic scenes. Realistic Scenes. We place the 20 categories of objects into 5 realistic scene settings for trajectory rendering: space, wall, door, tabletop, and outdoor. For each setting, we man￾ually annotate 20 scene placements using digital asset… view at source ↗
Figure 7
Figure 7. Figure 7: Model architecture of MobileManiVLA. A.5. MobileManiVLA [PITH_FULL_IMAGE:figures/full_fig_p015_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Real-world inference of MobileManiVLA on the G1 robot for the open laptop task. [PITH_FULL_IMAGE:figures/full_fig_p016_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Real-world setup of the G1 robot. 1280 × 720, and two wrist-mounted cameras capturing im￾ages at 640 × 480, as shown in [PITH_FULL_IMAGE:figures/full_fig_p016_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Realistic space scenes for cart [PITH_FULL_IMAGE:figures/full_fig_p017_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Realistic wall scenes for toilet, trashcan, refrigerator, washing machine, fridge, cabinet, table [PITH_FULL_IMAGE:figures/full_fig_p017_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Realistic door scenes for window, lever door, round door [PITH_FULL_IMAGE:figures/full_fig_p018_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Realistic tabletop scenes for box, laptop, dishwasher, oven, microwave, safe, faucet, holistic objects [PITH_FULL_IMAGE:figures/full_fig_p018_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Realistic outdoor scenes for car [PITH_FULL_IMAGE:figures/full_fig_p019_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Open trajectories from the G1 robot and the XHand robot [PITH_FULL_IMAGE:figures/full_fig_p019_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: Close trajectories from the G1 robot and the XHand robot [PITH_FULL_IMAGE:figures/full_fig_p020_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: Pull trajectories from the G1 robot and the XHand robot [PITH_FULL_IMAGE:figures/full_fig_p020_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Push trajectories from the G1 robot and the XHand robot [PITH_FULL_IMAGE:figures/full_fig_p021_18.png] view at source ↗
Figure 19
Figure 19. Figure 19: Pick trajectories from the G1 robot and the XHand robot [PITH_FULL_IMAGE:figures/full_fig_p021_19.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. AnnotateAnything: Automatic Annotation of 3D Assets for Robot Manipulation

    cs.RO 2026-06 unverdicted novelty 6.0

    AnnotateAnything converts passive 3D assets into manipulation-ready assets by combining vision-language reasoning for semantics with parallel physics pipelines for executable action annotations such as grasps and arti...

Reference graph

Works this paper leans on

59 extracted references · 22 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Agi- bot world colosseo: A large-scale manipulation platform for scalable and intelligent embodied systems, 2025

    AgiBot-World-Contributors, Qingwen Bu, Jisong Cai, Li Chen, Xiuqi Cui, Yan Ding, Siyuan Feng, Shenyuan Gao, Xindong He, Xuan Hu, Xu Huang, Shu Jiang, Yuxin Jiang, Cheng Jing, Hongyang Li, Jialu Li, Chiming Liu, Yi Liu, Yuxiang Lu, Jianlan Luo, Ping Luo, Yao Mu, Yuehan Niu, Yixuan Pan, Jiangmiao Pang, Yu Qiao, Guanghui Ren, Cheng Ruan, Jiaqi Shan, Yongjian...

    2025

  2. [2]

    Dexart: Benchmarking generalizable dexterous manipula- tion with articulated objects, 2023

    Chen Bao, Helin Xu, Yuzhe Qin, and Xiaolong Wang. Dexart: Benchmarking generalizable dexterous manipula- tion with articulated objects, 2023. 2, 3

    2023

  3. [3]

    Paligemma: A versatile 3b vlm for trans- fer.arXiv preprint arXiv:2407.07726, 2024

    Lucas Beyer, Andreas Steiner, Andr ´e Susano Pinto, Alexan- der Kolesnikov, Xiao Wang, Daniel Salz, Maxim Neumann, Ibrahim Alabdulmohsin, Michael Tschannen, Emanuele Bugliarello, et al. Paligemma: A versatile 3b vlm for trans- fer.arXiv preprint arXiv:2407.07726, 2024. 3

    Pith/arXiv arXiv 2024

  4. [4]

    Gr00t n1: An open foundation model for generalist humanoid robots.arXiv preprint arXiv:2503.14734, 2025

    Johan Bjorck, Fernando Casta ˜neda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi Fan, Yu Fang, Dieter Fox, Fengyuan Hu, Spencer Huang, et al. Gr00t n1: An open foundation model for generalist humanoid robots.arXiv preprint arXiv:2503.14734, 2025. 2, 3

    Pith/arXiv arXiv 2025

  5. [5]

    arXiv:2410.24164, 2024

    Kevin Black, Noah Brown, Danny Driess, Adnan Es- mail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al.π 0: A vision-language-action flow model for general robot control. arXiv:2410.24164, 2024. 2, 3, 8

    Pith/arXiv arXiv 2024

  6. [6]

    RT-1: Robotics transformer for real-world control at scale

    Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakr- ishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, et al. RT-1: Robotics transformer for real-world control at scale. arXiv:2212.06817, 2022. 2, 3

    Pith/arXiv arXiv 2022

  7. [7]

    Srinivasa, Pieter Abbeel, and Aaron M

    Berk Calli, Arjun Singh, Aaron Walsman, Siddhartha S. Srinivasa, Pieter Abbeel, and Aaron M. Dollar. The ycb ob- ject and model set: Towards common benchmarks for manip- ulation research. InProceedings of the IEEE International Conference on Advanced Robotics (ICAR), pages 510–517,

  8. [8]

    Gr-3 technical report.arXiv preprint arXiv:2507.15493, 2025

    Chilam Cheang, Sijin Chen, Zhongren Cui, Yingdong Hu, Liqun Huang, Tao Kong, Hang Li, Yifeng Li, Yuxiao Liu, Xiao Ma, et al. Gr-3 technical report.arXiv preprint arXiv:2507.15493, 2025. 2, 3

    Pith/arXiv arXiv 2025

  9. [9]

    Owmm-agent: Open world mobile manipulation with multi-modal agentic data synthe- sis, 2025

    Junting Chen, Haotian Liang, Lingxiao Du, Weiyun Wang, Mengkang Hu, Yao Mu, Wenhai Wang, Jifeng Dai, Ping Luo, Wenqi Shao, and Lin Shao. Owmm-agent: Open world mobile manipulation with multi-modal agentic data synthe- sis, 2025. 2, 3

    2025

  10. [10]

    Robotwin 2.0: A scalable data generator and bench- mark with strong domain randomization for robust bimanual robotic manipulation, 2025

    Tianxing Chen, Zanxin Chen, Baijun Chen, Zijian Cai, Yibin Liu, Zixuan Li, Qiwei Liang, Xianliang Lin, Yiheng Ge, Zhenyu Gu, Weiliang Deng, Yubin Guo, Tian Nian, Xuan- bing Xie, Qiangyu Chen, Kailun Su, Tianling Xu, Guodong Liu, Mengkang Hu, Huan ang Gao, Kaixuan Wang, Zhix- uan Liang, Yusen Qin, Xiaokang Yang, Ping Luo, and Yao Mu. Robotwin 2.0: A scalab...

    2025

  11. [11]

    Pali-x: On scaling up a multilingual vision and language model.arXiv preprint arXiv:2305.18565, 2023

    Xi Chen, Josip Djolonga, Piotr Padlewski, Basil Mustafa, Soravit Changpinyo, Jialin Wu, Carlos Riquelme Ruiz, Se- bastian Goodman, Xiao Wang, Yi Tay, et al. Pali-x: On scaling up a multilingual vision and language model.arXiv preprint arXiv:2305.18565, 2023. 3

    Pith/arXiv arXiv 2023

  12. [12]

    Isaac lab, 2023

    NVIDIA Corporation. Isaac lab, 2023. Accessed: 2025-11-

    2023

  13. [13]

    Palm-e: An embodied multimodal language model

    Danny Driess, Fei Xia, Mehdi SM Sajjadi, Corey Lynch, Aakanksha Chowdhery, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, Wenlong Huang, et al. Palm-e: An embodied multimodal language model. 2023. 3

    2023

  14. [14]

    Zhao, and Chelsea Finn

    Zipeng Fu, Tony Z. Zhao, and Chelsea Finn. Mobile aloha: Learning bimanual mobile manipulation with low- cost whole-body teleoperation, 2024. 2, 3

    2024

  15. [15]

    Maniskill2: A unified benchmark for generalizable manipulation skills, 2023

    Jiayuan Gu, Fanbo Xiang, Xuanlin Li, Zhan Ling, Xiqiang Liu, Tongzhou Mu, Yihe Tang, Stone Tao, Xinyue Wei, Yun- chao Yao, Xiaodi Yuan, Pengwei Xie, Zhiao Huang, Rui Chen, and Hao Su. Maniskill2: A unified benchmark for generalizable manipulation skills, 2023. 2, 3

    2023

  16. [16]

    Rekep: Spatio-temporal reasoning of rela- tional keypoint constraints for robotic manipulation, 2024

    Wenlong Huang, Chen Wang, Yunzhu Li, Ruohan Zhang, and Li Fei-Fei. Rekep: Spatio-temporal reasoning of rela- tional keypoint constraints for robotic manipulation, 2024. 4

    2024

  17. [17]

    Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, et al.π 0.5: a vision-language-action model with open-world generaliza- tion.arXiv:2504.16054, 2025. 2, 3, 8

    Pith/arXiv arXiv 2025

  18. [18]

    Stephen James, Zicong Ma, David Rovick Arrojo, and An- drew J. Davison. Rlbench: The robot learning benchmark & learning environment, 2019. 2, 3

    2019

  19. [19]

    Prismatic vlms: Investigating the design space of visually-conditioned language models

    Siddharth Karamcheti, Suraj Nair, Ashwin Balakrishna, Percy Liang, Thomas Kollar, and Dorsa Sadigh. Prismatic vlms: Investigating the design space of visually-conditioned language models. InForty-first International Conference on Machine Learning, 2024. 3

    2024

  20. [20]

    Alexander Khazatsky, Karl Pertsch, Suraj Nair, Ashwin Bal- akrishna, Sudeep Dasari, Siddharth Karamcheti, Soroush Nasiriany, Mohan Kumar Srirama, Lawrence Yunliang Chen, Kirsty Ellis, Peter David Fagan, Joey Hejna, Masha Itkina, Marion Lepert, Yecheng Jason Ma, Patrick Tree Miller, Jimmy Wu, Suneel Belkhale, Shivin Dass, Huy Ha, Arhan Jain, Abraham Lee, Y...

  21. [21]

    OpenVLA: An open- source vision-language-action model.arXiv:2406.09246,

    Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Fos- ter, Grace Lam, Pannag Sanketi, et al. OpenVLA: An open- source vision-language-action model.arXiv:2406.09246,

    Pith/arXiv arXiv

  22. [22]

    Fine-tuning vision-language-action models: Optimizing speed and suc- cess.arXiv:2502.19645, 2025

    Moo Jin Kim, Chelsea Finn, and Percy Liang. Fine-tuning vision-language-action models: Optimizing speed and suc- cess.arXiv:2502.19645, 2025. 2, 3

    Pith/arXiv arXiv 2025

  23. [23]

    CogACT: A foundational vision- language-action model for synergizing cognition and action in robotic manipulation.arXiv:2411.19650, 2024

    Qixiu Li, Yaobo Liang, Zeyu Wang, Lin Luo, Xi Chen, Mozheng Liao, Fangyun Wei, Yu Deng, Sicheng Xu, Yizhong Zhang, et al. CogACT: A foundational vision- language-action model for synergizing cognition and action in robotic manipulation.arXiv:2411.19650, 2024. 2, 3, 5, 6, 8

    Pith/arXiv arXiv 2024

  24. [24]

    Scalable vision-language-action model pretraining for robotic manipulation with real-life human ac- tivity videos.arXiv preprint arXiv:2510.21571, 2025

    Qixiu Li, Yu Deng, Yaobo Liang, Lin Luo, Lei Zhou, Chengtang Yao, Lingqi Zeng, Zhiyuan Feng, Huizhi Liang, Sicheng Xu, et al. Scalable vision-language-action model pretraining for robotic manipulation with real-life human ac- tivity videos.arXiv preprint arXiv:2510.21571, 2025

    arXiv 2025

  25. [25]

    Vision-language foundation models as effective robot imitators

    Xinghang Li, Minghuan Liu, Hanbo Zhang, Cunjun Yu, Jie Xu, Hongtao Wu, Chilam Cheang, Ya Jing, Weinan Zhang, Huaping Liu, et al. Vision-language foundation models as effective robot imitators. InICLR, 2022. 2, 3

    2022

  26. [26]

    Evaluating real-world robot manipulation policies in simulation.arXiv preprint arXiv:2405.05941, 2024

    Xuanlin Li, Kyle Hsu, Jiayuan Gu, Karl Pertsch, Oier Mees, Homer Rich Walke, Chuyuan Fu, Ishikaa Lunawat, Isabel Sieh, Sean Kirmani, Sergey Levine, Jiajun Wu, Chelsea Finn, Hao Su, Quan Vuong, and Ted Xiao. Evaluating real-world robot manipulation policies in simulation.arXiv preprint arXiv:2405.05941, 2024. 3

    Pith/arXiv arXiv 2024

  27. [27]

    Unidoormanip: Learn- ing universal door manipulation policy over large-scale and diverse door manipulation environments, 2024

    Yu Li, Xiaojie Zhang, Ruihai Wu, Zilong Zhang, Yiran Geng, Hao Dong, and Zhaofeng He. Unidoormanip: Learn- ing universal door manipulation policy over large-scale and diverse door manipulation environments, 2024. 2, 3, 4, 12

    2024

  28. [28]

    Libero: Benchmarking knowl- edge transfer for lifelong robot learning, 2023

    Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. Libero: Benchmarking knowl- edge transfer for lifelong robot learning, 2023. 3

    2023

  29. [29]

    Visual instruction tuning.NeurIPS, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.NeurIPS, 2023. 3

    2023

  30. [30]

    RDT-1B: A diffusion foundation model for bimanual manip- ulation.ICLR, 2025

    Songming Liu, Lingxuan Wu, Bangguo Li, Hengkai Tan, Huayu Chen, Zhengyi Wang, Ke Xu, Hang Su, and Jun Zhu. RDT-1B: A diffusion foundation model for bimanual manip- ulation.ICLR, 2025. 2, 3

    2025

  31. [31]

    Calvin: A benchmark for language- conditioned policy learning for long-horizon robot manip- ulation tasks, 2022

    Oier Mees, Lukas Hermann, Erick Rosete-Beas, and Wol- fram Burgard. Calvin: A benchmark for language- conditioned policy learning for long-horizon robot manip- ulation tasks, 2022. 3

    2022

  32. [32]

    Robocasa: Large-scale simulation of every- day tasks for generalist robots, 2024

    Soroush Nasiriany, Abhiram Maddukuri, Lance Zhang, Adeet Parikh, Aaron Lo, Abhishek Joshi, Ajay Mandlekar, and Yuke Zhu. Robocasa: Large-scale simulation of every- day tasks for generalist robots, 2024. 2, 3

    2024

  33. [33]

    Isaac Sim

    NVIDIA. Isaac Sim. 2, 3, 5, 12, 14

  34. [34]

    Open X- Embodiment: Robotic learning datasets and RT-X models

    Abby O’Neill, Abdul Rehman, Abhiram Maddukuri, Ab- hishek Gupta, Abhishek Padalkar, Abraham Lee, Acorn Pooley, Agrim Gupta, Ajay Mandlekar, et al. Open X- Embodiment: Robotic learning datasets and RT-X models. InICRA, 2024. 2, 3

    2024

  35. [35]

    SpatialVLA: Exploring spatial representations for visual- language-action model.arXiv:2501.15830, 2025

    Delin Qu, Haoming Song, Qizhi Chen, Yuanqi Yao, Xinyi Ye, Yan Ding, Zhigang Wang, JiaYuan Gu, Bin Zhao, et al. SpatialVLA: Exploring spatial representations for visual- language-action model.arXiv:2501.15830, 2025. 2, 3

    Pith/arXiv arXiv 2025

  36. [36]

    Xhand, 2025.https://www.robotera

    Robotera. Xhand, 2025.https://www.robotera. com/en/. 3, 12

    2025

  37. [37]

    Proximal policy optimization algo- rithms, 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Rad- ford, and Oleg Klimov. Proximal policy optimization algo- rithms, 2017. 2, 12

    2017

  38. [38]

    Memoryvla: Perceptual-cognitive memory in vision-language-action models for robotic manipulation

    Hao Shi, Bin Xie, Yingfei Liu, Lin Sun, Fengrong Liu, Tiancai Wang, Erjin Zhou, Haoqiang Fan, Xiangyu Zhang, and Gao Huang. Memoryvla: Perceptual-cognitive memory in vision-language-action models for robotic manipulation. arXiv preprint arXiv:2508.19236, 2025. 2, 3

    Pith/arXiv arXiv 2025

  39. [39]

    Paligemma 2: A family of versatile vlms for transfer.arXiv preprint arXiv:2412.03555, 2024

    Andreas Steiner, Andr ´e Susano Pinto, Michael Tschannen, Daniel Keysers, Xiao Wang, Yonatan Bitton, Alexey Grit- senko, Matthias Minderer, Anthony Sherbondy, Shangbang Long, et al. Paligemma 2: A family of versatile vlms for transfer.arXiv preprint arXiv:2412.03555, 2024. 5

    Pith/arXiv arXiv 2024

  40. [40]

    Gemini: a family of highly capable multimodal models

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean- Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalk- wyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv:2312.11805, 2023. 3

    Pith/arXiv arXiv 2023

  41. [41]

    Gemma 2: Improving open language models at a practical size.arXiv preprint arXiv:2408.00118,

    Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupati- raju, L´eonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ram´e, et al. Gemma 2: Improving open language models at a practical size.arXiv preprint arXiv:2408.00118,

    Pith/arXiv arXiv

  42. [42]

    Gemini robotics: Bringing ai into the physical world.arXiv preprint arXiv:2503.20020, 2025

    Gemini Robotics Team, Saminda Abeyruwan, Joshua Ainslie, Jean-Baptiste Alayrac, Montserrat Gonzalez Are- nas, Travis Armstrong, Ashwin Balakrishna, Robert Baruch, Maria Bauza, Michiel Blokzijl, et al. Gemini robotics: Bringing ai into the physical world.arXiv preprint arXiv:2503.20020, 2025. 2, 3

    Pith/arXiv arXiv 2025

  43. [43]

    Genie sim assets.https://github

    Genie Sim Team. Genie sim assets.https://github. com/AgibotTech/genie_sim, 2025. 5, 12, 14

    2025

  44. [44]

    Octo: An open- source generalist robot policy.arXiv:2405.12213, 2024

    Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Tobias Kreiman, Charles Xu, et al. Octo: An open- source generalist robot policy.arXiv:2405.12213, 2024. 2, 3

    Pith/arXiv arXiv 2024

  45. [45]

    Grutopia: Dream general robots in a city at scale, 2024

    Hanqing Wang, Jiahe Chen, Wensi Huang, Qingwei Ben, Tai Wang, Boyu Mi, Tao Huang, Siheng Zhao, Yilun Chen, Sizhe Yang, Peizhou Cao, Wenye Yu, Zichao Ye, Jialun Li, Junfeng Long, Zirui Wang, Huiling Wang, Ying Zhao, Zhongying Tu, Yu Qiao, Dahua Lin, and Jiangmiao Pang. Grutopia: Dream general robots in a city at scale, 2024. 3

    2024

  46. [46]

    Trtm: Template-based reconstruction and target-oriented manipulation of crumpled cloths, 2024

    Wenbo Wang, Gen Li, Miguel Zamora, and Stelian Coros. Trtm: Template-based reconstruction and target-oriented manipulation of crumpled cloths, 2024. 2

    2024

  47. [47]

    Unigrasptransformer: Simplified policy distillation for scalable dexterous robotic grasping, 2025

    Wenbo Wang, Fangyun Wei, Lei Zhou, Xi Chen, Lin Luo, Xiaohan Yi, Yizhong Zhang, Yaobo Liang, Chang Xu, Yan Lu, Jiaolong Yang, and Baining Guo. Unigrasptransformer: Simplified policy distillation for scalable dexterous robotic grasping, 2025. 3

    2025

  48. [48]

    Unified vision-language-action model

    Yuqi Wang, Xinghang Li, Wenxuan Wang, Junbo Zhang, Yingyan Li, Yuntao Chen, Xinlong Wang, and Zhaox- iang Zhang. Unified vision-language-action model. arXiv:2506.19850, 2025. 2, 3

    Pith/arXiv arXiv 2025

  49. [49]

    DexVLA: Vision-language model with plug-in diffusion expert for general robot control

    Junjie Wen, Yichen Zhu, Jinming Li, Zhibin Tang, Chaomin Shen, and Feifei Feng. DexVLA: Vision-language model with plug-in diffusion expert for general robot control. In CoRL, 2025. 2, 3

    2025

  50. [50]

    Guibas, Jingwei Xiao, Hao Dong, Ye Yuan, et al

    Fan Xiang, Kaichun Mo, Ye Xia, Hao Liu, Fangyin Zhang, Liang Han, Leonidas J. Guibas, Jingwei Xiao, Hao Dong, Ye Yuan, et al. Partnet-mobility: A large-scale database for ar- ticulated objects.Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11359–11368, 2020. 4, 12

    2020

  51. [51]

    Magma: A foundation model for multi- modal ai agents

    Jianwei Yang, Reuben Tan, Qianhui Wu, Ruijie Zheng, Baolin Peng, Yongyuan Liang, Yu Gu, Mu Cai, Seonghyeon Ye, Joel Jang, et al. Magma: A foundation model for multi- modal ai agents. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 14203–14214, 2025. 2, 3

    2025

  52. [52]

    GraspXL: Generating grasping motions for di- verse objects at scale

    Hui Zhang, Sammy Christen, Zicong Fan, Otmar Hilliges, and Jie Song. GraspXL: Generating grasping motions for di- verse objects at scale. InEuropean Conference on Computer Vision (ECCV), 2024. 4

    2024

  53. [53]

    Vlabench: A large-scale benchmark for language-conditioned robotics manipulation with long-horizon reasoning tasks, 2024

    Shiduo Zhang, Zhe Xu, Peiju Liu, Xiaopeng Yu, Yuan Li, Qinghui Gao, Zhaoye Fei, Zhangyue Yin, Zuxuan Wu, Yu- Gang Jiang, and Xipeng Qiu. Vlabench: A large-scale benchmark for language-conditioned robotics manipulation with long-horizon reasoning tasks, 2024. 2, 3

    2024

  54. [54]

    Dreamvla: a vision-language-action model dreamed with comprehensive world knowledge.arXiv preprint arXiv:2507.04447, 2025

    Wenyao Zhang, Hongsi Liu, Zekun Qi, Yunnan Wang, Xin- qiang Yu, Jiazhao Zhang, Runpei Dong, Jiawei He, Fan Lu, He Wang, et al. Dreamvla: a vision-language-action model dreamed with comprehensive world knowledge.arXiv preprint arXiv:2507.04447, 2025. 2, 3

    Pith/arXiv arXiv 2025

  55. [55]

    Humanoid everyday: A compre- hensive robotic dataset for open-world humanoid manipula- tion, 2025

    Zhenyu Zhao, Hongyi Jing, Xiawei Liu, Jiageng Mao, Abha Jha, Hanwen Yang, Rong Xue, Sergey Zakharor, Vitor Guizilini, and Yue Wang. Humanoid everyday: A compre- hensive robotic dataset for open-world humanoid manipula- tion, 2025. 2, 3

    2025

  56. [56]

    X-vla: Soft-prompted transformer as scalable cross-embodiment vision-language-action model

    Jinliang Zheng, Jianxiong Li, Zhihao Wang, Dongxiu Liu, Xirui Kang, Yuchun Feng, Yinan Zheng, Jiayin Zou, Yilun Chen, Jia Zeng, et al. X-vla: Soft-prompted transformer as scalable cross-embodiment vision-language-action model. arXiv preprint arXiv:2510.10274, 2025. 3

    Pith/arXiv arXiv 2025

  57. [57]

    DexGraspVLA: A vision-language-action framework towards general dexterous grasping.arXiv:2502.20900,

    Yifan Zhong, Xuchuan Huang, Ruochong Li, Ceyao Zhang, Zhang Chen, Tianrui Guan, Fanlian Zeng, Ka Num Lui, et al. DexGraspVLA: A vision-language-action framework towards general dexterous grasping.arXiv:2502.20900,

    arXiv

  58. [58]

    robosuite: A modular simulation framework and benchmark for robot learning

    Yuke Zhu, Josiah Wong, Ajay Mandlekar, Roberto Mart ´ın- Mart´ın, Abhishek Joshi, Kevin Lin, Abhiram Maddukuri, Soroush Nasiriany, and Yifeng Zhu. robosuite: A modular simulation framework and benchmark for robot learning. In arXiv preprint arXiv:2009.12293, 2020. 3

    Pith/arXiv arXiv 2009

  59. [59]

    <skill> <object>

    Brianna Zitkovich, Tianhe Yu, Sichun Xu, Peng Xu, Ted Xiao, Fei Xia, Jialin Wu, Paul Wohlhart, Stefan Welker, Ayzaan Wahid, et al. RT-2: Vision-language-action models transfer web knowledge to robotic control. InCoRL, 2023. 2, 3 MobileManiBench: Simplifying Model Verification for Mobile ManipulationSupplementary Material Figure 5. Initialization of the ro...

    2023