pith. sign in

arxiv: 2604.21017 · v3 · pith:AIJCP2BPnew · submitted 2026-04-22 · 💻 cs.RO · cs.AI

Open-H-Embodiment: A Large-Scale Dataset for Enabling Foundation Models in Medical Robotics

Open-H-Embodiment Consortium: Nigel Nelson , Juo-Tung Chen , Jesse Haworth , Xinhao Chen , Lukas Zbinden , Dianye Huang , Alaa Eldin Abdelaal , Alberto Arezzo
show 206 more authors
Ayberk Acar Farshid Alambeigi Carlo Alberto Ammirati Yunke Ao Pablo David Aranda Rodriguez Soofiyan Atar Mattia Ballo Noah Barnes Federica Barontini Filip Binkiewicz Peter Black Sebastian Bodenstedt Leonardo Borgioli Nikola Budjak Benjamin Calm\'e Fabio Carrillo Nicola Cavalcanti Changwei Chen Haoxin Chen Sihang Chen Qihan Chen Zhongyu Chen Ziyang Chen Shing Shin Cheng Meiqing Cheng Min Cheng Zih-Yun Sarah Chiu Xiangyu Chu Camilo Correa-Gallego Giulio Dagnino Anton Deguet Jacob Delgado Jonathan C. DeLong Kaizhong Deng Alexander Dimitrakakis Qingpeng Ding Hao Ding Giovanni Distefano Daniel Donoho Anqing Duan Marco Esposito Shane Farritor Jad Fayad Zahi Fayad Mario Ferradosa Filippo Filicori Chelsea Finn Philipp F\"urnstahl Jiawei Ge Stamatia Giannarou Xavier Giralt Ludevid Frederic Giraud Aditya Amit Godbole Ken Goldberg Antony Goldenberg Diego Granero Marana Xiaoqing Guo Tam\'as Haidegger Evan Hailey Pascal Hansen Ziyi Hao Kush Hari Kengo Hayashi Jonathon Hawkins Shelby Haworth Ortrun Hellig S. Duke Herrell Zhouyang Hong Andrew Howe Junlei Hu Zhaoyang Jacopo Hu Ria Jain Mohammad Rafiee Javazm Howard Ji Rui Ji Jianmin Ji Zhongliang Jiang Dominic Jones Jeffrey Jopling Britton Jordan Ran Ju Michael Kam Luoyao Kang Fausto Kang Siddhartha Kapuria Peter Kazanzides Sonika Kiehler Ethan Kilmer Ji Woong Kim Przemys{\l}aw Korzeniowski Chandra Kuchi Nithesh Kumar Alan Kuntz Federico Lavagno Yu Chung Lee Hao-Chih Lee Hang Li Zhen Li Xiao Liang Xinxin Lin Jinsong Lin Chang Liu Fei Liu Pei Liu Yun-hui Liu Wanli Liuchen Eszter Luk\'acs Sareena Mann Miles Mannas Brett Marinelli Sabina Martyniak Francesco Marzola Lorenzo Mazza Xueyan Mei Maria Clara Morais Luigi Muratore Chetan Reddy Narayanaswamy Micha{\l} Naskr\k{e}t David Navarro-Alarcon Cyrus Neary Chi Kit Ng Christopher Nguan David Noonan Ki Hwan Oh Tom Christian Olesch Allison M. Okamura Justin Opfermann Matteo Pescio Doan Xuan Viet Pham Tito Porras Hongliang Ren Ariel Rodriguez Jimenez Ferdinando Rodriguez y Baena Septimiu E. Salcudean Asmitha Sathya Preethi Satish Lalithkumar Seenivasan Jiaqi Shao Yiqing Shen Yu Sheng Lucy XiaoYang Shi Zoe Soul\'e Stefanie Speidel Mingwu Su Jianhao Su Idris Sunmola Krist\'of Tak\'acs Yunxi Tang Patrick Thornycroft Yu Tian Jordan Thompson Mehmet K. Turkcan Mathias Unberath Pietro Valdastri Carlos Vives Quan Vuong Martin Wagner Farong Wang Wei Wang Lidian Wang Chung-Pang Wang Guankun Wang Junyi Wang Erqi Wang Ziyi Wang Tanner Watts Wolfgang Wein Yimeng Wu Zijian Wu Hongjun Wu Luohong Wu Jie Ying Wu Junlin Wu Victoria Wu Kaixuan Wu Mateusz W\'ojcikowski Yunye Xiao Nan Xiao Wenxuan Xie Hao Yang Tianqi Yang Yinuo Yang Menglong Ye Ryan S. Yeung Nural Yilmaz Chim Ho Yin Michael Yip Rayan Younis Chenhao Yu Sayem Nazmuz Zaman Milos Zefran Han Zhang Yuelin Zhang Yidong Zhang Yanyong Zhang Xuyang Zhang Yameng Zhang Joyce Zhang Ning Zhong Peng Zhou Haoying Zhou Xiuli Zuo Nassir Navab Mahdi Azizian Sean D. Huver Axel Krieger
This is my paper

Pith reviewed 2026-07-05 01:54 UTC · model glm-5.2

classification 💻 cs.RO cs.AI
keywords medicalroboticsurgicalfoundationdatadatasetmodelmodels
0
0 comments X

The pith

First open medical robotics dataset trains model to suture end-to-end

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces Open-H-Embodiment, the largest openly shared dataset of medical robotic video paired with synchronized kinematic data, collected across more than 50 institutions and seven or more distinct robotic platforms spanning surgical manipulation, robotic ultrasound, and endoscopy. The authors argue that the field of autonomous medical robotics has been bottlenecked by small, single-embodiment, closed datasets, and that open data at this scale can serve as foundational infrastructure for training generalist robot models. To demonstrate the dataset's research value, the authors train two foundation models. The first, GR00T-H, is a vision-language-action model for medical robotics that becomes the only evaluated model to complete full end-to-end suturing tasks on a structured benchmark, achieving 25 percent full-trial success versus zero for all other models tested, and 64 percent average success across a 29-step ex vivo suturing sequence. The second, Cosmos-H-Surgical-Simulator, is an action-conditioned world model that simulates surgical scenes across nine robotic platforms from a single checkpoint, enabling in-silico policy evaluation and synthetic data generation. Together, these results are presented as evidence that large-scale, multi-embodiment, openly shared medical robot data can unlock foundation-model approaches that were previously infeasible in this domain.

Core claim

The central discovery is that when medical robotic data is aggregated at sufficient scale and diversity across heterogeneous platforms, a single vision-language-action model can learn to perform multi-step surgical suturing end-to-end, and a single world-model checkpoint can simulate surgical environments across nine different robot platforms. The dataset itself, Open-H-Embodiment, is the central object: it spans over 50 institutions, multiple robot platforms including the da Vinci, Versius, and others, and covers surgical manipulation, ultrasound, and endoscopy. The mechanism carrying the argument is that cross-embodiment training at this scale yields emergent task competence that no single

What carries the argument

The paper's argument rests on three named objects: (1) Open-H-Embodiment, the dataset of synchronized video plus kinematics from 50+ institutions across 7+ robot platforms; (2) GR00T-H, a vision-language-action foundation model trained on this data that maps visual observations and language instructions to robot actions; and (3) Cosmos-H-Surgical-Simulator, an action-conditioned world model that predicts future surgical video frames conditioned on robot actions, trained to generalize across nine robotic platforms from a single checkpoint. The vision-language-action paradigm means the model receives what the robot sees and a language description of the goal, then outputs the kinematic command

If this is right

  • If open multi-embodiment medical robot data enables a single model to suture end-to-end, then scaling the dataset further could push autonomous surgical sub-tasks toward clinical viability for routine steps like closure and trocar placement.
  • A single world-model checkpoint spanning nine platforms suggests that surgical simulation and policy evaluation could become decoupled from expensive physical simulators, lowering the cost of training and validating new surgical robot policies.
  • The dataset format, synchronized video plus kinematics across heterogeneous platforms, could become a community standard that future medical robot data collection efforts adopt, accelerating aggregation.
  • If cross-embodiment transfer works in surgical robotics, institutions with smaller or less common robot platforms could benefit from models pretrained on data collected on entirely different hardware.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper does not detail how kinematic data is normalized across platforms with different joint structures, control modes, and coordinate frames. If such normalization is ad hoc or incomplete, the multi-embodiment claims may rest on an unstated premise that the platforms are more kinematically compatible than they actually are.
  • The 25 percent full-task completion rate, while a relative improvement over zero, leaves three-quarters of trials incomplete. The paper frames this as a proof of concept, but the gap between proof of concept and clinical utility may be larger than the framing suggests.
  • A world model that simulates surgical video from a single checkpoint across nine platforms could, if sufficiently accurate, serve as a universal surgical data augmentor, potentially reducing the need for physical data collection in underrepresented procedures or platforms.

Load-bearing premise

The paper assumes that kinematic data collected across seven or more different robot platforms, each with different joint structures, control modes, and coordinate systems, can be made sufficiently interoperable to train a single foundation model without the abstract describing how that normalization is achieved or validated.

What would settle it

If the kinematic data across platforms cannot be meaningfully normalized, or if the dataset's scale is insufficient to overcome platform heterogeneity, then GR00T-H's suturing performance and Cosmos-H's multi-platform simulation would not generalize beyond the specific platforms seen during training, undermining the cross-embodiment claim.

read the original abstract

Autonomous medical robots hold promise to improve patient outcomes, reduce provider workload, democratize access to care, and enable superhuman precision. However, autonomous medical robotics has been limited by a fundamental data problem: existing medical robotic datasets are small, single-embodiment, and rarely shared openly, restricting the development of foundation models that the field needs to advance. We introduce Open-H-Embodiment, the largest open dataset of medical robotic video with synchronized kinematics to date, spanning more than 50 institutions and multiple robotic platforms including the CMR Versius, Intuitive Surgical's da Vinci, da Vinci Research Kit (dVRK), Rob Surgical BiTrack, Virtual Incision's MIRA, Moon Surgical Maestro, and a variety of custom systems, spanning surgical manipulation, robotic ultrasound, and endoscopy procedures. We demonstrate the research enabled by this dataset through two foundation models. GR00T-H is the first open foundation vision-language-action model for medical robotics, which is the only evaluated model to achieve full end-to-end task completion on a structured suturing benchmark (25% of trials vs. 0% for all others) and achieves 64% average success across a 29-step ex vivo suturing sequence. We also train Cosmos-H-Surgical-Simulator, the first action-conditioned world model to enable multi-embodiment surgical simulation from a single checkpoint, spanning nine robotic platforms and supporting in silico policy evaluation and synthetic data generation for the medical domain. These results suggest that open, large-scale medical robot data collection can serve as critical infrastructure for the research community, enabling advances in robot learning, world modeling, and beyond.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 1 minor

Summary. The manuscript under review, arXiv:2604.21017 (Open-H-Embodiment), proposes a large-scale open dataset for medical robotics spanning 50+ institutions and multiple robotic platforms, along with two foundation models (GR00T-H and Cosmos-H) trained on this data. However, I am unable to provide a substantive review because the full text provided for review is from a completely different paper (arXiv:2604.21018, 'Adaptive Test-Time Compute Allocation with Evolving In-Context Demonstrations' by Zuo and Zhou, in the area of LLM test-time compute scaling). Only the abstract of the Open-H-Embodiment paper was available. Without access to the actual manuscript, no assessment of the methods, data documentation, experimental design, or results is possible.

Significance. Based solely on the abstract, the paper's claims are potentially significant for the medical robotics community: an open, large-scale, multi-embodiment medical robotics dataset and associated foundation models would address a well-recognized infrastructure gap. However, these claims cannot be verified without the full text.

major comments (1)
  1. The full text provided for review is from an unrelated paper (arXiv:2604.21018 on adaptive test-time compute allocation for LLMs). The actual manuscript for arXiv:2604.21017 (Open-H-Embodiment) was not available. No substantive review of methods, data, or results is possible until the correct full text is provided. This is a procedural issue that must be resolved before any assessment of the paper's central claims can be made.
minor comments (1)
  1. This is a procedural note rather than a comment on the manuscript itself: the review system should be checked to ensure the correct PDF is associated with arXiv:2604.21017 so that a proper peer review can be conducted.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the careful and honest report. The referee's sole concern is procedural: the full text provided for review was from an unrelated paper (arXiv:2604.21018, 'Adaptive Test-Time Compute Allocation with Evolving In-Context Demonstrations' by Zuo and Zhou), not our manuscript (arXiv:2604.21017, 'Open-H-Embodiment'). We confirm that the referee is correct — the full text attached to the review package was indeed the wrong paper. This is a submission system error on the editorial side, not an issue with our manuscript itself. We have verified that our manuscript is correctly posted at arXiv:2604.21017 and have contacted the editorial office to ensure the correct full text is delivered to the referee. We respectfully request that the referee be provided with the correct manuscript so that a substantive review can proceed. We are happy to address any technical questions once the referee has access to the actual paper.

read point-by-point responses

  1. Referee: The full text provided for review is from an unrelated paper (arXiv:2604.21018 on adaptive test-time compute allocation for LLMs). The actual manuscript for arXiv:2604.21017 (Open-H-Embodiment) was not available. No substantive review of methods, data, or results is possible until the correct full text is provided.

    Authors: The referee is entirely correct. The full text in the review package was from arXiv:2604.21018 (Zuo and Zhou), a paper on LLM test-time compute scaling, which is completely unrelated to our work on medical robotics datasets and foundation models. This was a submission system error. Our manuscript is available at arXiv:2604.21017 and we have notified the editorial office to rectify the delivery. We respectfully request that the referee receive the correct manuscript and look forward to addressing any substantive feedback. revision: no

Circularity Check

0 steps flagged

No circularity detectable from available material; full text is from an unrelated paper

full rationale

The full text provided is from an unrelated paper (arXiv:2604.21018 on test-time compute allocation), not from the Open-H-Embodiment paper. Only the abstract of the target paper is available. From the abstract, this is a dataset-and-empirical-evaluation paper: it describes collecting a large-scale medical robotics dataset from 50+ institutions, training two foundation models (GR00T-H and Cosmos-H), and evaluating them on benchmarks. There is no mathematical derivation chain to audit for circularity. The claims are empirical (25% vs 0% task completion, 64% average success) and the dataset is described as externally sourced from multiple institutions, not constructed to fit a result. No circularity can be identified from the abstract alone, and the full text needed to verify any derivation is absent. This is an honest non-finding: score 0.

Axiom & Free-Parameter Ledger

0 free parameters · 3 axioms · 0 invented entities

No free parameters or invented entities are identifiable from the abstract alone. The axioms listed are domain assumptions that the paper's claims structurally depend on but that the abstract does not explicitly justify. The full text was unavailable for a more exhaustive audit.

axioms (3)
  • domain assumption Synchronized kinematic data across heterogeneous robotic platforms can be normalized into a common representation suitable for cross-embodiment model training.
    The abstract claims multi-embodiment training across 7+ platforms (CMR Versius, da Vinci, dVRK, etc.) with different kinematic structures. This normalization is a prerequisite for the central claims but is not described in the abstract.
  • domain assumption Video and kinematics data collected across 50+ institutions with different protocols, surgeons, and setups are sufficiently consistent to support foundation model training.
    The dataset spans 50+ institutions. The assumption that this multi-site data is consistent enough for training is load-bearing for the dataset's value proposition.
  • domain assumption The structured suturing benchmark and 29-step ex vivo suturing sequence are valid evaluations of surgical task capability.
    The 25% completion and 64% success metrics depend on these benchmarks being meaningful. Without the full text, the validity of these evaluation protocols cannot be assessed.

pith-pipeline@v1.1.0-glm · 12812 in / 2715 out tokens · 259899 ms · 2026-07-05T01:54:36.484148+00:00 · methodology

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Adversarial Attacks on Learned Policies for Surgical Robotic Tasks

    cs.RO 2026-06 unverdicted novelty 8.0

    Adversarial visual perturbations reduce success rates of end-to-end policies for debridement and suturing by an average of 61% in physical experiments on three policy architectures.

  2. BiliVLA: Scene-Aware Vision-Language-Action Model with Reinforcement Learning for Autonomous Biliary Endoscopic Navigation

    cs.RO 2026-06 unverdicted novelty 6.0

    BiliVLA reports 91.96% average action precision and 84.85% success rate on three ERCP subtasks in phantom experiments by combining scene-aware supervision, safety-aware recovery, and two-stage SFT+GRPO training.

  3. BiliVLA: Scene-Aware Vision-Language-Action Model with Reinforcement Learning for Autonomous Biliary Endoscopic Navigation

    cs.RO 2026-06 unverdicted novelty 5.0

    BiliVLA applies scene-aware VLA with grounding-enhanced SFT and GRPO to achieve 91.96% action precision and 84.85% success rate across three ERCP subtasks in phantom experiments.

  4. World Models for Robotic Manipulation: A Survey

    cs.RO 2026-05 accept novelty 5.0

    Survey organizing world models for robotic manipulation into representation families, a functional taxonomy, and infrastructure roles across pretraining, post-training, and inference, while reviewing 34 datasets and e...

Reference graph

Works this paper leans on

2 extracted references · 2 canonical work pages · cited by 3 Pith papers · 1 internal anchor

  1. [1]

    Why Language Models Hallucinate

    URLhttps://arxiv.org/abs/2509.04664. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models, 2020. URL https://arxiv.org/abs/2001.08361. 12 Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large ...

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.1007/s10489-025-07044-6 2020

  2. [2]

    Fanzeng Xia, Yidong Luo, Tinko Sebastian Bartels, Yaqi Xu, and Tongxin Li

    URLhttps://arxiv.org/abs/2510.02611. Fanzeng Xia, Yidong Luo, Tinko Sebastian Bartels, Yaqi Xu, and Tongxin Li. Rethinking the unsolvable: When in-context search meets test-time scaling, 2025. URLhttps://arxiv.org/abs/2505.22290. Benfeng Xu, Quan Wang, Zhendong Mao, Yajuan Lyu, Qiaoqiao She, and Yongdong Zhang.�k�NN prompting: Beyond-context learning with...

    work page doi:10.18653/v1/2025.findings-acl.547 2025