Pith. sign in

REVIEW 4 major objections 5 minor 95 references

Surgical WAM: A World-Action Model for Data-Efficient Surgical Robot Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Under a fixed action-labeled budget, action-free video pretraining improves closed-loop surgical manipulation, lifting average success from 63.5% to 77.8% on four SurRoL tasks.

desk verdict Well-designed controlled study showing that surgical video pretraining helps, but the central causal claim rests on an unspecified pretraining corpus and a fragile-looking 80k checkpoint. read the letter →

arxiv 2608.11204 v1 pith:4BAHIHLB submitted 2026-08-11 cs.RO cs.AIcs.CV

Wenrui Bao , Tianyun Jiang , Zhiben Chen , Ser-Nam Lim , Peter D. Peng , Yuzhang Shang This is my paper · ORCID
classification cs.ROcs.AIcs.CV
keywords surgicalroboticsworld-actionmodelaction-freevideopretrainingdata-efficientrobotlearningclosed-loopmanipulationdVRKSurRoLimitation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Surgical robot policies are starved for action-labeled demonstrations because teleoperating a dVRK with synchronized kinematics is slow and expensive, while endoscopic video is comparatively cheap and abundant. This paper asks whether that abundant video alone can carry the visual-dynamics learning that policies need. It answers yes: a world-action model pretrained on action-free surgical video and then fine-tuned on a fixed budget of demonstrations improves closed-loop task success from an average of 63.5% to 77.8% across four simulated dVRK manipulation tasks, with the biggest gains on contact-rich and bimanual tasks. The result matters because it suggests the data bottleneck for surgical robot learning can shift from costly teleoperation to video that surgery already produces at scale.

What carries the argument

The central object is the world-action model (WAM), instantiated with Cosmos Policy: a single diffusion transformer whose latent sequence contains slots for past endoscopic frames, proprioceptive state, future-video predictions, and action chunks, all denoised jointly. Stage 1 pretrains only the visual and future-prediction slots on action-free video using the video denoising loss, so weights learned without kinematic labels become the initialization for action prediction. Stage 2 fine-tunes the whole sequence on action-labeled demonstrations with the joint denoising objective, coupling action tokens to the video-derived dynamics. At test time the model is a receding-horizon controller: it predicts a chunk of Hc=16 actions and future frames, executes the first He=4 actions, then replans from the new observation.

What would settle it

Collect a Stage 1 corpus that provably excludes all SurRoL task frames or rollouts (e.g., only unrelated surgical recordings or only non-task tissue motion), run the identical two-stage protocol, and compare success rates; if the 14.3-point average gain disappears, the attribution to action-free video pretraining fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that under a fixed budget of action-labeled demonstrations, action-free video pretraining improves closed-loop surgical manipulation. The authors establish this by building Surgical WAM, a world-action model that jointly predicts future endoscopic observations and executable dVRK action chunks, pretraining it on action-free surgical video, and then fine-tuning it on the fixed action-labeled set. With pretraining, average closed-loop success across four SurRoL tasks rises from 63.5% to 77.8%, including a 20-point absolute gain on PegTransfer; ablations show the pretrained model peaks with half the fine-tuning steps and stays robust across execution horizons, and the trend reproduces on real dVRK video from JIGSAWS.

Load-bearing premise

The action-free video corpus used in Stage 1 is not described, so the causal claim that gains come from action-free video pretraining assumes this corpus does not contain frames or rollouts from the SurRoL evaluation tasks.

Editorial extensions

If this is right

  • With the same 10k demonstrations, video-pretrained Surgical WAM reaches 77.8% average success versus 63.5% without pretraining, and peaks in 80k fine-tuning steps instead of 160k.
  • The gain is concentrated where surgical dynamics matter most: contact-rich and bimanual tasks (PegTransfer +20 points, BiPegTransfer 64% vs 42% without pretraining).
  • The benefit generalizes beyond simulation: the same two-stage protocol on real JIGSAWS dVRK video shows the same qualitative advantage, arguing the prior captures genuine surgical dynamics rather than simulator artifacts.
  • Generalist vision-language-action pretraining transfers poorly to surgery (one baseline averages 22.3%), whereas the surgical video prior transfers well, suggesting domain-matched dynamics matter more than broad language grounding for precision tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the Stage 1 corpus is truly action-free and disjoint from evaluation tasks, a similar recipe could extend to other surgical platforms and to partially automated data curation, since the bottleneck shifts from teleoperation to video collection and filtering.
  • The finding predicts a scaling law: success should continue to rise with the volume and diversity of action-free surgical video, which the paper does not directly measure but which is the natural next experiment.
  • The receding-horizon design suggests that joint sampling of video and action tokens at inference may be replaceable by cheaper open-loop action sampling once the dynamics prior is fixed; the paper discards predicted frames, so ablating joint sampling could cut inference cost without much loss.
  • A testable extension is to pretrain on out-of-domain video from a different surgical site or simulator to distinguish generic video priors from task-specific leakage, since the current corpus is unspecified.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces Surgical WAM, a two-stage world-action model built on the Cosmos Policy backbone, in which a model is first pretrained on action-free surgical video and then fine-tuned on a fixed set of action-labeled demonstrations. Closed-loop evaluation on four SurRoL tasks (Needle Pick, Peg Transfer, Needle Regrasp, BiPeg Transfer) reports average success of 63.5% without pretraining and 77.8% with pretraining, with the largest gain (20 absolute points) on PegTransfer. Additional ablations vary the fine-tuning step count (Table 2) and execution horizon (Table 3), and a qualitative experiment on the JIGSAWS real dVRK video is described in Section 4.4. The central claim is that action-free video pretraining, rather than the action-labeled budget, drives the improvement.

Significance. If the central claim is correct, the paper offers a practical route to reducing the demand for teleoperated kinematics in surgical robot learning: abundant unlabeled endoscopic video can be used to learn visual dynamics, while a small action-labeled set grounds the policy. The main strengths are the controlled w/ PT versus w/o PT comparison, which shares architecture, action representation, optimization, and evaluation protocol, and the consistent gains across four tasks that include contact-rich and bimanual behaviors. The two-stage recipe is clearly described, and the dependence of the benefit on fine-tuning budget and execution horizon is explicitly probed. However, the causal interpretation currently rests on an unspecified pretraining corpus and on single-run results, as detailed below.

major comments (4)
  1. [Section 3.2 (Stage 1; Eq. (2))] The causal claim that action-free video pretraining improves closed-loop control is only as strong as the Stage 1 corpus. The paper states that the Stage 1 weights are obtained from Cosmos-H-Surgical (He et al. 2026), fine-tuned on 'large-scale surgical video', but it never describes the composition of that corpus, nor reports any overlap check against the SurRoL evaluation tasks. If the corpus contains SurRoL-style rollouts or task-specific frames, then the w/ PT variant has already seen the evaluation distribution during pretraining, and the 63.5% to 77.8% gain would not demonstrate transfer from generic action-free surgical video. Please specify the corpus, rule out overlap with the evaluation tasks, or add a pretraining variant on a clearly disjoint corpus (e.g., JIGSAWS-only video) with quantitative closed-loop results.
  2. [Table 2; Section 4.3] The fine-tuning-step ablation is the main evidence for 'faster convergence' and 'higher peak with half the budget', yet the w/ PT curve is severely non-monotonic: 52% at 40k, 86% at 80k, 34% at 120k, and 56.5% at 160k. At 120k and 160k the pretrained model is far worse than the non-pretrained model at the same steps (71.5% and 79%). This pattern is inconsistent with a simple convergence-speed advantage and suggests overfitting, instability, or evaluation noise. The 'Best' row selects checkpoints post hoc. Please report multiple seeds with error bars, explain the collapse, and justify the choice of 80k steps as the default. If the 86% at 80k is not reproducible, the central claim is not supported by this table.
  3. [Table 3; Section 4.3] The execution-horizon ablation reports single runs for each He. The claim that pretraining makes the controller 'robust across horizons' rests on w/ PT values of 86%, 70%, 86%, and 80%, while w/o PT ranges from 50% to 66%. Without variance estimates or additional seeds, it is hard to separate a real robustness effect from checkpoint or seed noise. Add confidence intervals or repeat each configuration.
  4. [Section 4.4] The real-data experiment is presented only qualitatively: no quantitative closed-loop success, no description of how JIGSAWS demonstrations are converted to the absolute Cartesian action space used in SurRoL, and no evaluation protocol. This section is used to assert that the benefit 'persists under the visual complexity and manipulation difficulty of real surgical scenes.' Either provide quantitative evidence (e.g., success metrics, or at least prediction or action metrics with standard errors) or weaken the claim to a qualitative observation.
minor comments (5)
  1. [Section 4.1, Table 1 caption] The caption states that 'All methods are trained on the same action-labeled dataset with an identical fine-tuning budget,' but DEX (Huang et al. 2023) is a demonstration-guided RL method that likely uses environment interaction, not only fine-tuning on a fixed dataset. Clarify what 'fine-tuning budget' means for each baseline.
  2. [Figure 2(ii); Section 3.2] Figure 2(ii) labels 'Large-scale Footage' and 'Internet Videos' as Stage 1 inputs, while Section 3.2 says Stage 1 uses the He et al. (2026) model fine-tuned on unspecified surgical video. Align the figure with the text or describe the actual pretraining corpus.
  3. [Section 3.2] The sentence 'the gain thus comes entirely from the video pretraining, at no extra cost in action supervision' should be rephrased as 'at no extra action-label cost', since Stage 1 incurs compute and data-curation costs; it should also be conditioned on the corpus-overlap caveat raised above.
  4. [Section 2.3] The paper claims 'the first world-action model applied to surgical manipulation'; given the related work on SAW and Cosmos-Surg-dVRK, specify the sense in which Surgical WAM is first (e.g., first to integrate an action head into the same generative model and evaluate closed-loop task execution).
  5. [Section 5] The paper does not include a limitations section; the non-monotonicity in Table 2 and the unspecified pretraining corpus are limitations that should be discussed explicitly in the text.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central comparison is a controlled empirical experiment, not a derivation that reduces to its inputs.

full rationale

The paper's central claim is that, under a fixed action-labeled budget, action-free video pretraining improves closed-loop surgical manipulation. This is established by comparing two variants, Surgical WAM w/ PT and w/o PT, that share the same architecture, optimizer, action representation, sampler, and evaluation protocol, with only the Stage 1 surgical-video pretraining differing. Neither the video-only objective L_pre (Eq. 2) nor the fine-tuning objective L_ft (Eq. 3) embeds the evaluation success metric, and no fitted parameter is later relabeled as a prediction. The w/PT variant uses an external surgical world model from He et al. (2026), so the pretraining signal is not produced by the paper's own equations and then recovered from them. There is no load-bearing self-citation, no imported uniqueness theorem, and no ansatz disguised as a result. The reader's concern about the unspecified composition of the He et al. (2026) video corpus and possible overlap with SurRoL evaluation tasks is a legitimate external-validity or data-leakage risk, not a circularity: the paper's derivation chain does not reduce Eq. (2), Eq. (3), or Table 1 to its own inputs by construction. Accordingly, the honest finding is no significant circularity, with score 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

Surgical WAM introduces no new model components or physical entities; it combines Cosmos Policy with a pretrained surgical video model. The central claim depends mainly on data assumptions (pretraining corpus composition, simulator fidelity) and a compatibility claim, rather than on fitted parameters.

free parameters (3)
  • Execution horizon H_e = 4
    Default in all main experiments; Table 3 shows the pretrained model varies between 70% and 86% across H_e values, so the headline gain depends on this manual choice.
  • Action chunk length H_c = 16
    Fixed chunk length used throughout; not ablated in the paper, so the reported results all use this value.
  • Diffusion denoising steps = 5
    Sampling hyperparameter used for all results; no ablation is shown.
assumptions (4)
  • domain assumption SurRoL simulator dynamics faithfully represent dVRK surgical manipulation, including contact and bimanual coordination.
    All quantitative results are closed-loop success rates in SurRoL (Section 4.1), while the central claim is framed for surgical manipulation generally.
  • domain assumption The action-free video corpus from He et al. (2026) is representative of the surgical domain and does not contain task-overlapping SurRoL content.
    Section 3.2 invokes large-scale surgical video but does not disclose composition or overlap; the causal attribution of the pretraining gain assumes no leakage.
  • ad hoc to paper Cosmos Policy's visual and future-prediction slots are architecturally compatible with the video-pretrained weights, allowing direct transfer into the action head.
    Section 3.1 asserts compatibility; no quantitative transfer check is provided.
  • domain assumption Closed-loop success on 100 episodes per task is a sufficient evaluation without reporting seed variance or confidence intervals.
    Section 4.1 defines the metric; the paper does not report multiple seeds or statistical significance.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Surgical WAM: A World-Action Model for Data-Efficient Surgical Robot Learning." pith.science (2026). https://pith.science/paper/4BAHIHLB

@misc{pith2026260811204,
  author       = {Pith},
  title        = {Pith review of: Surgical WAM: A World-Action Model for Data-Efficient Surgical Robot Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4BAHIHLB}},
  note         = {Machine review of arXiv:2608.11204}
}
read the original abstract

Learning reliable surgical manipulation policies is bottlenecked by the scarcity of action-labeled demonstrations: teleoperated surgical robot (e.g., dVRK) trajectories with synchronized kinematics are costly to collect, while surgical tasks demand precise contact handling, long-horizon reasoning, and bimanual coordination. Endoscopic video is comparatively inexpensive and abundant relative to synchronized video--kinematics trajectories, and a natural way to exploit it is to learn world models of surgical scenes. However, existing surgical world models use video primarily for simulation or policy evaluation, and rarely translate the learned dynamics into closed-loop control. This gap raises our central question: under a fixed budget of action-labeled demonstrations, does action-free video pretraining improve closed-loop surgical manipulation? To answer it, we introduce the Surgical World-Action Model (Surgical WAM), a unified generative model built on Cosmos Policy that jointly predicts future endoscopic observations and executable surgical robot action chunks. Surgical WAM first learns surgical visual dynamics from action-free video and is then fine-tuned on the fixed action-labeled budget; at deployment, it acts as a closed-loop, receding-horizon controller that executes a short prefix of each predicted action chunk and replans from the resulting observation. On a suite of four simulated surgical manipulation tasks, video pretraining improves the average success rate from 63.5% to 77.8%, including an absolute gain of 20 percentage points on PegTransfer, with the largest improvements on contact-rich and bimanual tasks. These results demonstrate that action-free video provides transferable visual dynamics priors for learning surgical robot control with limited action supervision, positioning data-efficient video pretraining as a practical path toward scaling up surgical robot learning.

Figures

Figures reproduced from arXiv: 2608.11204 by the authors.

Figure 1
Figure 1. Surgical WAM drives closed-loop dVRK manipulation and makes policy learning markedly more efficient. (i) A dVRK robot driven by Surgical WAM performs the PegTransfer task in closed loop. At each control step, the model receives the current endoscopic observation and dVRK proprioception, jointly predicts future observations and an action chunk, executes a short prefix of the predicted actions, and replans from the re… view at source ↗
Figure 2
Figure 2. Surgical WAM architecture and two-stage training recipe. (Top) Architecture. Closed-loop inference with the Surgical WAM on SurRoL peg transfer. At each control step, the model conditions on the current endoscopic frame, robot state and goal, and the task text. A single video-action diffusion transformer samples an action chunk in the same latent sequence used for visual/state prediction. (Bottom) Two-stage training… view at source ↗
Figure 3
Figure 3. Real-robot surgical manipulation sequences from JIGSAWS. Representative endoscopic frames from the JIGSAWS dVRK teleoperation recordings used in our real-data experiments, with task progress shown from left to right. (Top) Knot tying. (Bottom) Suturing [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

95 extracted references · 61 canonical work pages

  1. [1]

    2601.16163 , archivePrefix=

    Kim, Moo Jin and Gao, Yihuai and Lin, Tsung-Yi and Lin, Yen-Chen and Ge, Yunhao and Lam, Grace and Liang, Percy and Song, Shuran and Liu, Ming-Yu and Finn, Chelsea and Gu, Jinwei , year=. 2601.16163 , archivePrefix=

    arXiv

  2. [2]

    2501.03575 , archivePrefix=

    Agarwal, Niket and others , year=. 2501.03575 , archivePrefix=

    arXiv

  3. [3]

    Jang, Joel and others , booktitle=

  4. [4]

    Shen, Yichao and Wei, Fangyun and Du, Zhiying and Liang, Yaobo and Lu, Yan and Yang, Jiaolong and Zheng, Nanning and Guo, Baining , booktitle=

  5. [5]

    Mimic-Video: Video-Action Models for Generalizable Robot Control Beyond

    Pai, Jonas and Achenbach, Liam and Montesinos, Victoriano and Forrai, Benedek and Mees, Oier and Nava, Elvis , year=. Mimic-Video: Video-Action Models for Generalizable Robot Control Beyond. 2512.15692 , archivePrefix=

    arXiv

  6. [6]

    2603.16666 , archivePrefix=

    Yuan, Tianyuan and Dong, Zibin and Liu, Yicheng and Zhao, Hang , year=. 2603.16666 , archivePrefix=

    arXiv

  7. [7]

    2026 , eprint=

    World Action Models Are Zero-Shot Policies , author=. 2026 , eprint=

    2026

  8. [8]

    Advances in Neural Information Processing Systems (

    Learning Universal Policies via Text-Guided Video Generation , author=. Advances in Neural Information Processing Systems (

Show all 95 references
  1. [9]

    Xu, Jiaqi and Li, Bin and Lu, Bo and Liu, Yun-Hui and Dou, Qi and Heng, Pheng-Ann , booktitle=

  2. [10]

    Yu, Qinxi and Moghani, Masoud and Dharmarajan, Karthik and Schorp, Vincent and Panitch, William Chung-Ho and Liu, Jingzhou and Goldberg, Ken and Garg, Animesh and others , booktitle=

  3. [11]

    2510.20965 , archivePrefix=

    Haworth, Jesse and Chen, Juo-Tung and Nelson, Nigel and Kim, Ji Woong and Moghani, Masoud and Finn, Chelsea and Krieger, Axel , year=. 2510.20965 , archivePrefix=

  4. [12]

    and Schmidgall, Samuel and Scheikl, Paul Maria and Finn, Chelsea and Krieger, Axel , year=

    Kim, Ji Woong and Chen, Juo-Tung and Hansen, Pascal and Shi, Lucy X. and Schmidgall, Samuel and Scheikl, Paul Maria and Finn, Chelsea and Krieger, Axel , year=. 2505.10251 , archivePrefix=

    arXiv

  5. [13]

    2024 , eprint=

    Robotic Constrained Imitation Learning for the Peg Transfer Task in Fundamentals of Laparoscopic Surgery , author=. 2024 , eprint=

    2024

  6. [14]

    2025 , eprint=

    You Only Teach Once: Learn One-Shot Bimanual Robotic Manipulation from Video Demonstrations , author=. 2025 , eprint=

    2025

  7. [15]

    2512.23162 , archivePrefix=

    He, Yufan and Guo, Pengfei and Xu, Mengya and Li, Zhaoshuo and Myronenko, Andriy and Xu, Daguang and others , year=. 2512.23162 , archivePrefix=

  8. [16]

    2510.16240 , archivePrefix=

    Zbinden, Lukas and Nelson, Nigel and Chen, Juo-Tung and Chen, Xinhao and Kim, Ji Woong and Azizian, Mahdi and Krieger, Axel and Huver, Sean , year=. 2510.16240 , archivePrefix=

  9. [17]

    2022 , eprint=

    Data Splits and Metrics for Method Benchmarking on Surgical Action Triplet Datasets , author=. 2022 , eprint=

    2022

  10. [18]

    2506.02555 , archivePrefix=

    Zeng, Zhitao and Zhuo, Zhu and Jia, Xiaojun and Jin, Yueming and others , year=. 2506.02555 , archivePrefix=

    arXiv

  11. [19]

    2025 , eprint=

    Evaluating Gemini Robotics Policies in a Veo World Simulator , author=. 2025 , eprint=

    2025

  12. [20]

    2507.12440 , archivePrefix=

    Yang, Ruihan and Yu, Qinxi and Wang, Xiaolong and others , year=. 2507.12440 , archivePrefix=

    arXiv

  13. [21]

    2025 , eprint=

    OpenHelix: A Short Survey, Empirical Analysis, and Open-Source Dual-System VLA Model for Robotic Manipulation , author=. 2025 , eprint=

    2025

  14. [22]

    2509.09372 , archivePrefix=

    Wang, Yihao and Ding, Pengxiang and Wang, Donglin and others , year=. 2509.09372 , archivePrefix=

  15. [23]

    2024 , eprint=

    _0 : A Vision-Language-Action Flow Model for General Robot Control , author=. 2024 , eprint=

    2024

  16. [24]

    International Conference on Learning Representations (

    Latent Action Pretraining from Videos , author=. International Conference on Learning Representations (

  17. [25]

    2511.07732 , archivePrefix=

    Routray, Sandeep and Pan, Hengkai and Jain, Unnat and Bahl, Shikhar and Pathak, Deepak , year=. 2511.07732 , archivePrefix=

  18. [26]

    Science Translational Medicine , volume=

    Supervised Autonomous Robotic Soft Tissue Surgery , author=. Science Translational Medicine , volume=

  19. [27]

    Science Robotics , volume=

    Autonomous Robotic Laparoscopic Surgery for Intestinal Anastomosis , author=. Science Robotics , volume=

  20. [28]

    and Taylor, Russell H

    Kazanzides, Peter and Chen, Zihan and Deguet, Anton and Fischer, Gregory S. and Taylor, Russell H. and DiMaio, Simon P. , booktitle=. An Open-Source Research Kit for the da

  21. [29]

    Swaroop and Reiley, Carol E

    Gao, Yixin and Vedula, S. Swaroop and Reiley, Carol E. and Ahmidi, Narges and Varadarajan, Balakrishnan and Lin, Henry C. and Tao, Lingling and Zappella, Luca and B

  22. [30]

    Journal of Machine Learning Research , volume=

    Scheikl, Paul Maria and Gyenes, Bal. Journal of Machine Learning Research , volume=

  23. [31]

    Surgical Gym: A High-Performance

    Schmidgall, Samuel and Krieger, Axel and Eshraghian, Jason , booktitle=. Surgical Gym: A High-Performance

  24. [32]

    Advances in Neural Information Processing Systems (

    Hindsight Experience Replay , author=. Advances in Neural Information Processing Systems (

  25. [33]

    Overcoming Exploration in Reinforcement Learning with Demonstrations , author=

  26. [34]

    2006.09359 , archivePrefix=

    Nair, Ashvin and Gupta, Abhishek and Dalal, Murtaza and Levine, Sergey , year=. 2006.09359 , archivePrefix=

    2006 arXiv

  27. [35]

    and Levine, Sergey , booktitle=

    Reddy, Siddharth and Dragan, Anca D. and Levine, Sergey , booktitle=

  28. [36]

    Peng, Xue Bin and Ma, Ze and Abbeel, Pieter and Levine, Sergey and Kanazawa, Angjoo , journal=

  29. [37]

    International Conference on Autonomous Agents and Multiagent Systems (

    Integrating Behavior Cloning and Reinforcement Learning for Improved Performance in Dense and Sparse Reward Environments , author=. International Conference on Autonomous Agents and Multiagent Systems (

  30. [38]

    Autonomous Blood Suction for Robot-Assisted Surgery: A Sim-to-Real Reinforcement Learning Approach , author=

  31. [39]

    Learning Autonomous Surgical Irrigation and Suction With the da Vinci Research Kit Using Reinforcement Learning , year=

    Ou, Yafei and Tavakoli, Mahdi , journal=. Learning Autonomous Surgical Irrigation and Suction With the da Vinci Research Kit Using Reinforcement Learning , year=

  32. [40]

    International Conference on Learning Representations (

    Zero-Shot Robotic Manipulation with Pretrained Image-Editing Diffusion Models , author=. International Conference on Learning Representations (

  33. [41]

    2024 , eprint=

    This&That: Language-Gesture Controlled Video Generation for Robot Planning , author=. 2024 , eprint=

    2024

  34. [42]

    International Conference on Learning Representations (

    Learning to Act from Actionless Videos through Dense Correspondences , author=. International Conference on Learning Representations (

  35. [43]

    International Conference on Learning Representations (

    Unleashing Large-Scale Video Generative Pre-Training for Visual Robot Manipulation , author=. International Conference on Learning Representations (

  36. [44]

    2410.06158 , archivePrefix=

    Cheang, Chi-Lam and Chen, Guangzeng and Jing, Ya and Kong, Tao and Li, Hang and Li, Yifeng and others , year=. 2410.06158 , archivePrefix=

    arXiv

  37. [45]

    International Conference on Machine Learning (

    Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representations , author=. International Conference on Machine Learning (

  38. [46]

    International Conference on Learning Representations (

    Predictive Inverse Dynamics Models Are Scalable Learners for Robotic Manipulation , author=. International Conference on Learning Representations (

  39. [47]

    Wu, Jialong and Yin, Shaofeng and Feng, Ningya and He, Xu and Li, Dong and Hao, Jianye and Long, Mingsheng , booktitle=

  40. [48]

    International Conference on Machine Learning (

    Genie: Generative Interactive Environments , author=. International Conference on Machine Learning (

  41. [49]

    2025 , eprint=

    Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasets , author=. 2025 , eprint=

    2025

  42. [50]

    2506.21539 , archivePrefix=

    Cen, Jun and Yu, Chaohui and others , year=. 2506.21539 , archivePrefix=

    arXiv

  43. [51]

    Medical Image Computing and Computer Assisted Intervention (

    Endora: Video Generation Models as Endoscopy Simulators , author=. Medical Image Computing and Computer Assisted Intervention (

  44. [52]

    Chen, Tong and others , booktitle=

  45. [53]

    Yang, Yousef and others , booktitle=

  46. [54]

    2024 , eprint=

    Bora: Biomedical Generalist Video Generation Model , author=. 2024 , eprint=

    2024

  47. [55]

    Medical Image Computing and Computer Assisted Intervention (

    Neural Rendering for Stereo 3D Reconstruction of Deformable Tissues in Robotic Surgery , author=. Medical Image Computing and Computer Assisted Intervention (

  48. [56]

    2401.12561 , archivePrefix=

    Liu, Yifan and Li, Chenxin and Yang, Chen and Yuan, Yixuan , year=. 2401.12561 , archivePrefix=

    arXiv

  49. [57]

    Yang, Shuojue and Li, Qian and Shen, Daiyun and Gong, Bingchen and Dou, Qi and Jin, Yueming , booktitle=

  50. [58]

    Seenivasan, Lalith and Islam, Mobarakol and Kannan, Gokul and Ren, Hongliang , booktitle=

  51. [59]

    2505.15206 , archivePrefix=

    Ng, Chi Kit and Bai, Long and Wang, Guankun and Ren, Hongliang and others , year=. 2505.15206 , archivePrefix=

    arXiv

  52. [60]

    2026 , eprint=

    SAW: Toward a Surgical Action World Model via Controllable and Scalable Video Generation , author=. 2026 , eprint=

    2026

  53. [61]

    2025 , eprint=

    How Far Are Surgeons from Surgical World Models? A Pilot Study on Zero-shot Surgical Video Generation with Expert Assessment , author=. 2025 , eprint=

    2025

  54. [62]

    International Conference on Learning Representations (

    Learning to Act without Actions , author=. International Conference on Learning Representations (

  55. [63]

    2505.06111 , archivePrefix=

    Bu, Qingwen and Yang, Yanting and Cai, Jisong and Gao, Shenyuan and Ren, Guanghui and Yao, Maoqing and Luo, Ping and Li, Hongyang , year=. 2505.06111 , archivePrefix=

    arXiv

  56. [64]

    International Conference on Computer Vision (

    Moto: Latent Motion Token as the Bridging Language for Learning Robot Manipulation from Videos , author=. International Conference on Computer Vision (

  57. [65]

    Baker, Bowen and Akkaya, Ilge and Zhokhov, Peter and Huizinga, Joost and Tang, Jie and Ecoffet, Adrien and Houghton, Brandon and Sampedro, Raul and Clune, Jeff , booktitle=. Video

  58. [66]

    Nair, Suraj and Rajeswaran, Aravind and Kumar, Vikash and Finn, Chelsea and Gupta, Abhinav , booktitle=

  59. [67]

    Ma, Yecheng Jason and Sodhani, Shagun and Jayaraman, Dinesh and Bastani, Osbert and Kumar, Vikash and Zhang, Amy , booktitle=

  60. [68]

    2022 , eprint=

    Masked Visual Pre-Training for Motor Control , author=. 2022 , eprint=

    2022

  61. [69]

    Robotics: Science and Systems (

    Language-Driven Representation Learning for Robotics , author=. Robotics: Science and Systems (

  62. [70]

    Advances in Neural Information Processing Systems (

    Where Are We in the Search for an Artificial Visual Cortex for Embodied Intelligence? , author=. Advances in Neural Information Processing Systems (

  63. [71]

    Wang, Chen and Fan, Linxi and Sun, Jiankai and Zhang, Ruohan and Fei-Fei, Li and Xu, Danfei and Zhu, Yuke and Anandkumar, Anima , booktitle=

  64. [72]

    Jain, Vidhi and others , booktitle=

  65. [73]

    Robotics: Science and Systems (

    Structured World Models from Human Videos , author=. Robotics: Science and Systems (

  66. [74]

    Bharadhwaj, Homanga and Mottaghi, Roozbeh and Gupta, Abhinav and Tulsiani, Shubham , booktitle=

  67. [75]

    Robotics: Science and Systems (

    Any-Point Trajectory Modeling for Policy Learning , author=. Robotics: Science and Systems (

  68. [76]

    Conference on Robot Learning (

    Flow as the Cross-Domain Manipulation Interface , author=. Conference on Robot Learning (

  69. [77]

    Conference on Robot Learning (

    General Flow as Foundation Affordance for Scalable Robot Learning , author=. Conference on Robot Learning (

  70. [78]

    Science Robotics , volume=

    Medical Robotics---Regulatory, Ethical, and Legal Considerations for Increasing Levels of Autonomy , author=. Science Robotics , volume=

  71. [79]

    Science Robotics , volume=

    A Decade Retrospective of Medical Robotics Research from 2010 to 2020 , author=. Science Robotics , volume=

    2010

  72. [80]

    Annual Review of Control, Robotics, and Autonomous Systems , volume=

    Autonomy in Surgical Robotics , author=. Annual Review of Control, Robotics, and Autonomous Systems , volume=

  73. [81]

    Surgical Endoscopy , volume=

    The Current State of Autonomous Suturing: A Systematic Review , author=. Surgical Endoscopy , volume=

  74. [82]

    International Journal of Computer Assisted Radiology and Surgery , volume=

    Surgical Robotics beyond Enhanced Dexterity Instrumentation: A Survey of Machine Learning Techniques and Their Role in Intelligent and Autonomous Surgical Actions , author=. International Journal of Computer Assisted Radiology and Surgery , volume=

  75. [83]

    O'Neill, Abby and Rehman, Abdul and Maddukuri, Abhinav and others , booktitle=. Open

  76. [84]

    Khazatsky, Alexander and Pertsch, Karl and Nair, Suraj and others , booktitle=

  77. [85]

    Walke, Homer and Black, Kevin and Lee, Abraham and Kim, Moo Jin and Du, Max and Zheng, Chongyi and Zhao, Tony and Hansen-Estruch, Philippe and Vuong, Quan and He, Andre and Myers, Vivek and Fang, Kuan and Finn, Chelsea and Levine, Sergey , booktitle=

  78. [86]

    Advances in Neural Information Processing Systems (

    Recurrent World Models Facilitate Policy Evolution , author=. Advances in Neural Information Processing Systems (

  79. [87]

    International Conference on Learning Representations (

    Dream to Control: Learning Behaviors by Latent Imagination , author=. International Conference on Learning Representations (

  80. [88]

    Brohan, Anthony and Brown, Noah and Carbajal, Justice and others , booktitle=

  81. [89]

    Kim, Moo Jin and Pertsch, Karl and Karamcheti, Siddharth and others , booktitle=

  82. [90]

    Robotics: Science and Systems (

    Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware , author=. Robotics: Science and Systems (

  83. [91]

    Robotics: Science and Systems (

    Diffusion Policy: Visuomotor Policy Learning via Action Diffusion , author=. Robotics: Science and Systems (

  84. [92]

    Advances in Neural Information Processing Systems (

    Behavior Transformers: Cloning k Modes with One Stone , author=. Advances in Neural Information Processing Systems (

  85. [93]

    Demonstration-Guided Reinforcement Learning with Efficient Exploration for Task Automation of Surgical Robot , author=

  86. [94]

    and Finn, Chelsea , booktitle=

    Fu, Zipeng and Zhao, Tony Z. and Finn, Chelsea , booktitle=. Mobile

  87. [95]

    and others , booktitle=

    Black, Kevin and Brown, Noah and Darpinian, James and Dhabalia, Karan and Driess, Danny and Esmail, Adnan and Equi, Michael Robert and Finn, Chelsea and Fusai, Niccolo and Galliker, Manuel Y. and others , booktitle=. _

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.