Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

SRT-H: A Hierarchical Framework for Autonomous Surgery via Language Conditioned Imitation Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A hierarchical language-conditioned policy is claimed to complete the clip-and-cut phase of cholecystectomy autonomously on unseen ex vivo tissue, with 100% success across eight gallbladders.

desk verdict A genuine engineering advance in long-horizon surgical imitation learning, but the 'fully autonomous without human intervention' claim is contradicted by the paper's own protocol and needs to be fixed before this can be taken at face value. read the letter →

arxiv 2505.10251 v3 pith:2OZR6R62 submitted 2025-05-15 cs.RO

Ji Woong Kim , Juo-Tung Chen , Pascal Hansen , Lucy X. Shi , Antony Goldenberg , Samuel Schmidgall , Paul Maria Scheikl , Anton Deguet
show 6 more authors
Brandon M. White De Ru Tsai Richard Cha Jeffrey Jopling Chelsea Finn Axel Krieger
This is my paper · ORCID
classification cs.RO
keywords autonomoussurgerycholecystectomyhierarchicalpolicylanguage-conditionedimitationlearningself-correctiondVRKexvivovalidationlong-horizonmanipulation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that a surgical robot can perform a real, multi-minute surgical step on its own, not just a scripted subtask in a controlled lab. Its proposed system, SRT-H, combines a high-level policy that plans in natural language with a low-level policy that turns each instruction into motions, and it is trained end-to-end from RGB video and language labels by imitation. On eight previously unseen ex vivo porcine gallbladders, the system completed all 17 clip-and-cut tasks of cholecystectomy with no human intervention, self-correcting its errors along the way. If this holds, it would be evidence that step-level autonomy—not just task-level automation—is within reach for soft-tissue surgery, and that language is a workable control interface for long-horizon manipulation.

What carries the argument

The load-bearing mechanism is the two-level SRT-H stack with natural language as the coupling medium. The high-level policy maps a short history of endoscope frames to three discrete outputs—the next task, a correction flag, and a directional corrective instruction—using a Swin-Transformer vision encoder and a Transformer decoder. The low-level policy is a decoder-only Transformer that fuses wrist-camera and endoscope images, encoded by EfficientNet, with a DistilBERT embedding of the incoming instruction through FiLM layers, and emits a chunk of hybrid-relative actions: translations relative to the endoscope tip, rotations relative to the end-effector, and jaw angles for both arms. Language is doing the decomposition work: it splits a long horizon into named phases, gives the high-level policy a compact way to say 'move the right arm to the right' when the low level drifts, and provides the channel through which DAgger-style human corrections are collected and replayed as training data.

What would settle it

Run SRT-H on a fresh batch of ex vivo gallbladders with fully functional clips and live scissor closure, and count whether all six clips fully occlude their tubes and both tubes are completely transected without leakage; the 100% claim stands only if every trial passes, and the first incomplete occlusion or partial cut would bound the real-world transfer.

Watch

Extended reading notes

Core claim

The central claim, stated on the paper's own terms, is that step-level surgical autonomy is achievable with a hierarchical policy trained purely by imitation from RGB images and language labels. The high-level policy watches endoscope video and issues one of 17 task instructions or one of 18 corrective motion instructions; the low-level policy turns whichever instruction it receives, together with endoscope and wrist-camera images, into a 2-second chunk of Cartesian motions for both arms. On eight ex vivo porcine gallbladders that were not used in training, the system completed all 17 clip-and-cut tasks—grabbing the gallbladder, placing six clips, and cutting the duct and artery twice each—without any human intervention, averaging 317 seconds of robot motion and about six self-corrections per case. The paper also reports that removing the corrective-language channel, the wrist cameras, or the high-level DAgger fine-tuning each degrades success, and that a flat end-to-end low-level-only policy succeeds only 33.3% of the time.

Load-bearing premise

The system's training data never exercises real clip latching or real scissor closure—clips are disabled and cuts are synthesized in post-processing—so the 100% success claim assumes that transferring to genuine clipping and cutting on unseen tissue requires no dynamics the simulated labels did not capture.

Editorial extensions

If this is right

  • If the 100% result holds, an RGB-only, language-conditioned imitation learning stack can handle contact-rich bimanual manipulation on deformable tissue, not just tabletop or navigation tasks.
  • The hierarchy's self-correction channel means the system can recover from its own errors mid-procedure, shifting the safety question from never making mistakes to detecting and correcting mistakes before tissue damage occurs.
  • The ablations imply that the corrective-instruction vocabulary, wrist-camera views, and high-level DAgger fine-tuning are each load-bearing; dropping any one measurably lowers success.
  • The data-scaling curve (66.7%, 77.8%, and 100% success at one-third, two-thirds, and full data) implies that collecting demonstrations from more organs and anatomies is a direct route to broader generalization.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: A natural next experiment is to replace the trained high-level policy with a fine-tuned vision-language model emitting the same instruction set; if success holds, the low-level policy is the main carrier of skill, and if it collapses, the high-level policy's learned temporal model of the procedure is essential.
  • Inference: The current 100% is defined over organs that were selected for their typical anatomy; a deployed system would additionally need to handle the roughly 10% of gallbladders excluded here for artery-over-duct or branched-artery anatomy, since those are common enough to appear in practice.
  • Inference: Because the high-level policy samples every 3 seconds and the low level commits to 20-30 step action chunks, the framework could be adapted to other multi-phase soft-tissue procedures by changing only the instruction vocabulary and demonstration data, with little architectural change.
  • Inference: The surgeon comparison suggests SRT-H is smoother but slower; a clinically viable system would need to close that time gap or automate the clip loading and tool changes that currently require a human assistant, since the reported 317 seconds excludes those pauses.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces SRT-H, a hierarchical framework for robotic surgery in which a high-level policy issues language instructions (task-level and corrective) and a low-level policy generates Cartesian-space trajectories from RGB images and wrist-camera views. The framework is evaluated on the clip-and-cut phase of ex vivo cholecystectomy: 16,000 demonstration trajectories from 34 porcine gallbladders are used to train the policies, and eight unseen gallbladders are used for the core evaluation. The authors report a 100% success rate across all 17 tasks with an average duration of 317 seconds, along with ablations on the hierarchical design, wrist cameras, HL DAgger fine-tuning, and an end-to-end baseline, plus a comparison with an expert surgeon.

Significance. If the technical claims can be supported after revision, this is a meaningful advance: it is one of the first demonstrations of a single learned policy executing a long-horizon (17-task) surgical step sequence with self-correction in a realistic ex vivo setting, rather than a single tabletop subtask. The dataset and code are made available, and the ablations provide useful evidence that the hierarchical design and language-based corrective interface contribute to robustness. The comparison with GPT-4o as a planner and the surgeon comparison are useful additional datapoints. The main contribution is the engineering and empirical demonstration of a hierarchical vision-language-action policy for a multi-step surgical procedure, which is timely and relevant to the community.

major comments (4)
  1. [Abstract; Core experiment results; Materials and Methods, Experiment design; Low-level policy, Inference] The central claim "fully autonomously without human intervention" is not supported by the protocol described in the manuscript. The operator loads each clip, changes tools between the clip applier and scissors, and resumes the system after tool changes; the dVRK had to be reinitialized three times during manual tool changes; and the reported 317-second duration explicitly "excludes the time of reloading the clips and making tool changes performed by the operator." This is an internal inconsistency, not a matter of interpretation. The abstract and Movie 1 should be revised to state that the system autonomously executes the learned manipulation and strategy-selection steps, with human assistance for clip reloading and tool changes; the LoA IV classification should also be re-examined, since physical assistance with tool management is not merely supervision.
  2. [Materials and Methods, Data collection, paragraph 2] The 100% success rate is conditional on excluding approximately 10% of gallbladders with atypical anatomy (artery crossing over the duct or branching). The claim "generalizing across anatomies" in the introduction and Movie 1 is therefore limited to the subset of anatomies seen in the training and evaluation sets. The paper should state this exclusion explicitly wherever the 100% rate and the generalizability claims appear, and should temper the clinical-generalization framing accordingly.
  3. [Materials and Methods, Data collection, paragraph 3; Comparison with expert surgeon] The training data for cutting motions were collected without actually closing the scissors: the motion of placing the scissors was performed, and the kinematics were post-processed to simulate cutting. The policy is therefore never trained on real tool-tissue interaction during cutting. The test-time evidence of successful cuts is anecdotal ("if the cut is not successful, which was very rare"), and in the surgeon comparison the robot was stopped before closing its grippers to avoid damaging tissue. To support the 100% success rate of the cutting tasks, the paper should provide a systematic verification of cut success (e.g., post-procedure surgical inspection, leakage testing, or video-based evidence that the duct and artery were cleanly severed), and discuss the sim-to-real gap arising from the simulated cutting labels.
  4. [High-level policy ablation studies; Table S2] The HL policy's corrective-instruction prediction accuracy is reported as approximately 70% (F1 = 56.37%) in Table S2, which is notably lower than the task-instruction accuracy of 97%. The paper should discuss whether this moderate accuracy is sufficient for reliable recovery behavior, and how the 18-way corrective instruction classifier interacts with the LL policy during the reported self-corrections.
minor comments (5)
  1. [Discussion, Limitations, Making SRT-H safer] "A further extension fo this work" contains a typo; it should read "A further extension of this work."
  2. [Low-level policy, Inference] The statement that "the robot autonomously requests for a tool change" is implemented by a logic-based state machine rather than by the learned policy; this distinction should be clarified so that the autonomy claim is attributed precisely to the learned components.
  3. [Table 1 caption and Core experiment results] Table 1 lists 'Duration (s)' but the text clarifies this excludes clip reloading and tool changes; the caption should state this caveat explicitly to avoid misreading.
  4. [Figure 2 caption] Figure 2 is dense; labeling the HL output heads (task, correction flag, corrective instruction) and the selection switch in the panel would improve readability.
  5. [High-level policy ablation studies] The sentence referring to corrective instructions as "out of 18 possible motion classes (see Supplementary Methods" is missing a closing parenthesis and should reference the list of corrective instructions in the supplementary text for completeness.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the central 100% success result is an external evaluation on unseen gallbladders, not a fitted or self-referential quantity.

full rationale

The derivation chain is self-contained. Training data are expert demonstrations on 34 porcine gallbladders; the high-level policy minimizes a weighted cross-entropy objective (Eq. 1) and the low-level policy minimizes L1 action loss (Eq. 3), both over held-out supervised labels. Corrective language labels for the LL policy are generated from future action-chunk motion trends, a preprocessing rule rather than a quantity derived from the model's own outputs. The headline 100% success rate is measured on eight unseen gallbladders not used in training or parameter fitting, so it is an external result rather than a constructed one. The paper does cite same-group prior work for components: the hybrid-relative action representation from SRT [26], the image-history and prediction-offset ideas from Shi et al. [54], and the HL DAgger procedure [54]. These are adopted methods, not load-bearing uniqueness theorems or definitions that make the evaluation self-fulfilling; the main claim does not reduce to them. The internal inconsistency between 'fully autonomously without human intervention' and the protocol requiring an operator to load clips, change tools, and resume the system, as well as the training-time simulation of cutting motions by post-processing kinematics, are external-validity and correctness concerns, not circular reductions: no equation is equivalent to its own input, and no fitted parameter is relabeled as a prediction. I therefore find no circular step, with score 1 reflecting only the presence of minor, non-load-bearing self-citations.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

This is an empirical robotics paper, so the ledger lists the experimental and modeling assumptions that the 100% success claim depends on, plus the hand-tuned hyperparameters of the learned policies. The system introduces no new physical entities; the relevant 'free parameters' are training choices that were not derived from the data.

free parameters (7)
  • HL policy loss weights = w_p=0.4, w_c=0.3, w_m=0.3
    Chosen to prioritize task instruction accuracy; the paper does not report a systematic sweep, and these weights affect the learned HL policy that drives the procedure.
  • LL execution horizons = 30 timesteps for grabbing, 20 for other phases
    Selected based on preliminary experiments to reduce hesitation during grabbing; directly affects the success of the low-level policy.
  • Corrective instruction auto-label chunk size = 10 timesteps
    Used to generate corrective language labels from future action trends; larger or smaller chunks change the corrective vocabulary and training signal.
  • HL prediction offset = 0.5 s
    The HL policy is trained to predict the task 0.5 s into the future; chosen without reported ablation.
  • Center crop region = inner 50% width, lower 80% height
    Fixed crop added as an extra HL input; the region is a design choice not derived from data.
  • Recovery-mode training fraction = 60% of sequences start in recovery mode
    Hand-set to expose the policy to recovery scenarios; affects the HL policy's corrective behavior.
  • LL action chunk size = k=60 (2 s horizon)
    Horizon for predicting action sequences; chosen to balance precision and replanning frequency.
assumptions (5)
  • domain assumption Ex vivo porcine gallbladder in an open CAD-designed abdominal dome setup is a valid proxy for evaluating cholecystectomy autonomy.
    The experiments are bench-top, not in vivo; the paper acknowledges the absence of breathing motion, bleeding, and port constraints (Discussion, Limitations).
  • domain assumption Training on clips with disabled latches and on simulated cutting trajectories (scissors not closed, kinematics extended post hoc) transfers to real clip placement and cutting at test time.
    Materials and Methods, Data collection; the paper notes the policy sometimes re-cuts if the duct remains intact, acknowledging the transfer is imperfect.
  • domain assumption Operator-mediated clip loading, tool changes, and system pauses do not count as human intervention in the 'fully autonomous' claim.
    The protocol requires a human to load clips, change tools, and resume after pauses; the autonomy claim rests on excluding these from the definition of intervention.
  • domain assumption Binary completion of the 17 predefined tasks (grab, six clips, two cuts with retractions) is an adequate measure of surgical success.
    No clinical outcome measures, tissue damage scoring, or independent surgical grading are used; success is assessed from images of clip placement and cuts.
  • domain assumption Wrist cameras mounted on the instruments provide sufficient perception despite their size and the open setup; existing dVRK kinematics errors are handled by the hybrid-relative action representation from prior work.
    The wrist-camera configuration is non-clinical (too large for ports), and the hybrid-relative actions are adopted from the self-cited SRT paper [26].

how reviews work

0 comments
Cite this review

Pith. "Pith review of SRT-H: A Hierarchical Framework for Autonomous Surgery via Language Conditioned Imitation Learning." pith.science (2026). https://pith.science/paper/2OZR6R62

@misc{pith2026250510251,
  author       = {Pith},
  title        = {Pith review of: SRT-H: A Hierarchical Framework for Autonomous Surgery via Language Conditioned Imitation Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2OZR6R62}},
  note         = {Machine review of arXiv:2505.10251}
}
read the original abstract

Research on autonomous surgery has largely focused on simple task automation in controlled environments. However, real-world surgical applications demand dexterous manipulation over extended durations and generalization to the inherent variability of human tissue. These challenges remain difficult to address using existing logic-based or conventional end-to-end learning approaches. To address this gap, we propose a hierarchical framework for performing dexterous, long-horizon surgical steps. Our approach utilizes a high-level policy for task planning and a low-level policy for generating robot trajectories. The high-level planner plans in language space, generating task-level or corrective instructions that guide the robot through the long-horizon steps and correct for the low-level policy's errors. We validate our framework through ex vivo experiments on cholecystectomy, a commonly-practiced minimally invasive procedure, and conduct ablation studies to evaluate key components of the system. Our method achieves a 100\% success rate across eight unseen ex vivo gallbladders, operating fully autonomously without human intervention. This work demonstrates step-level autonomy in a surgical procedure, marking a milestone toward clinical deployment of autonomous surgical systems.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Surgical WAM: A World-Action Model for Data-Efficient Surgical Robot Learning

    cs.RO 2026-08 conditional novelty 5.0 of 10

    Video-only pretraining of a world-action model improves closed-loop dVRK manipulation on SurRoL from 63.5% to 77.8% average success under a fixed action-labeled budget.

Reference graph

Works this paper leans on

66 extracted references · 59 canonical work pages · cited by 1 Pith paper

  1. [1]

    The growing global burden of gallstone disease.World Gastroenterology News, 17(4):6–9, 2012

    Monica Acalovschi and Frank Lammert. The growing global burden of gallstone disease.World Gastroenterology News, 17(4):6–9, 2012

    work page 2012

  2. [2]

    Sloboda, Siraj Husain, Nawaid Usmani, and Mahdi Tavakoli

    Mehrnoosh Afshar, Jay Carriere, Tyler Meyer, Ron S. Sloboda, Siraj Husain, Nawaid Usmani, and Mahdi Tavakoli. A model-based multi-point tissue manipulation for enhancing breast brachytherapy. IEEE Transactions on Medical Robotics and Bionics, 4(4):1046–1056, 2022

    work page 2022

  3. [3]

    Angelopoulos and Stephen Bates

    Anastasios N. Angelopoulos and Stephen Bates. Conformal prediction: A gentle introduction. Foundations and Trends in Machine Learning, 16(4):494–591, 2023. ISSN 1935-8237

    work page 2023

  4. [4]

    Single-shot synthetic wavelength imaging: Sub-mm precision tof sensing with conventional cmos sensors

    Manuel Ballester, Heming Wang, Jiren Li, Oliver Cossairt, and Florian Willomitzer. Single-shot synthetic wavelength imaging: Sub-mm precision tof sensing with conventional cmos sensors. Optics and Lasers in Engineering, 178:108165, 2024

    work page 2024

  5. [5]

    Rt-1: Robotics transformer for real-world control at scale

    Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, et al. Rt-1: Robotics transformer for real-world control at scale. InRobotics: Science and Systems, 2022

    work page 2022

  6. [6]

    Albumentations: fast and flexible image augmentations.Information, 11(2):125, 2020

    Alexander Buslaev, Vladimir I Iglovikov, Eugene Khvedchenya, Alex Parinov, Mikhail Druzhinin, and Alexandr A Kalinin. Albumentations: fast and flexible image augmentations.Information, 11(2):125, 2020

    work page 2020

  7. [7]

    Q-transformer: Scalable offline reinforcement learning via autoregressive q-functions

    Yevgen Chebotar, Quan Vuong, Karol Hausman, Fei Xia, Yao Lu, Alex Irpan, Aviral Kumar, Tianhe Yu, Alexander Herzog, Karl Pertsch, et al. Q-transformer: Scalable offline reinforcement learning via autoregressive q-functions. InConference on Robot Learning, pp. 3909–3928. PMLR, 2023

    work page 2023

  8. [8]

    Funk, Ryan K

    Zih-Yun Chiu, Florian Richter, Emily K. Funk, Ryan K. Orosco, and Michael C. Yip. Bimanual regrasping for suture needles using reinforcement learning for rapid motion planning. In2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 7737–7743, 2021

    work page 2021

Show all 66 references
  1. [9]

    Clearcam – the first and only robotic scope cleaner.https://www.clearcam-med

    ClearCam. Clearcam – the first and only robotic scope cleaner.https://www.clearcam-med. com/, 2025. Accessed: 2025-04-01. 21 SRT-H: A Hierarchical Framework for Autonomous Surgery via Language-Conditioned Imitation Learning

    2025

  2. [10]

    Clickclean – lens cleaning device for minimally invasive surgery

    ClickClean. Clickclean – lens cleaning device for minimally invasive surgery. https:// clickclean-medeon.com/, 2025. Accessed: 2025-04-01

    2025

  3. [11]

    Open X-Embodiment Collaboration, Abhishek Padalkar, Acorn Pooley, Ajinkya Jain, Alex Bewley, Alex Herzog, Alex Irpan, Alexander Khazatsky, Anant Rai, Anikait Singh, Anthony Brohan, Antonin Raffin, Ayzaan Wahid, Ben Burgess-Limerick, Beomjoon Kim, Bernhard Schölkopf, Brian Icht...

    2023

  4. [12]

    Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V

    Ekin D. Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V. Le. Randaugment: Practical automated data augmentation with a reduced search space. In2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 3008–3017, 2020

    2020

  5. [13]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In2009 IEEE conference on computer vision and pattern recognition, pp. 248–255. Ieee, 2009

    2009

  6. [14]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. InNorth American Chapter of the Association for Computational Linguistics, 2019

    2019

  7. [15]

    Autonomous robotic intracardiac catheter navigation using haptic vision.Science robotics, 4(29):eaaw1977, 2019

    Georgios Fagogenis, Margherita Mencattelli, Zurab Machaidze, Benoit Rosa, Karl Price, F Wu, V Weixler, Mossab Saeed, John E Mayer, and Pierre E Dupont. Autonomous robotic intracardiac catheter navigation using haptic vision.Science robotics, 4(29):eaaw1977, 2019

    2019

  8. [16]

    Rohit Gupta, Anil Kumar, Chinniahnapalaya P Hariprasad, and Manoj Kumar. Anatomical variations of cystic artery, cystic duct, and gall bladder and their associated intraoperative and 22 SRT-H: A Hierarchical Framework for Autonomous Surgery via Language-Conditioned Imitation L...

    2023

  9. [17]

    Action contextualiza- tion: Adaptive task planning and action tuning using large language models.IEEE Robotics and Automation Letters, 9(11):9407–9414, 2024

    Sthithpragya Gupta, Kunpeng Yao, Loïc Niederhauser, and Aude Billard. Action contextualiza- tion: Adaptive task planning and action tuning using large language models.IEEE Robotics and Automation Letters, 9(11):9407–9414, 2024. doi: 10.1109/LRA.2024.3460408

    2024

  10. [18]

    Autonomy for surgical robots: Concepts and paradigms.IEEE Transactions on Medical Robotics and Bionics, 1(2):65–76, 2019

    Tamás Haidegger. Autonomy for surgical robots: Concepts and paradigms.IEEE Transactions on Medical Robotics and Bionics, 1(2):65–76, 2019

    2019

  11. [19]

    Mustafa Haiderbhai, Radian Gondokaryono, Andrew Wu, and Lueder A. Kahrs. Sim2real rope cutting with a surgical robot using vision-based reinforcement learning.IEEE Transactions on Automation Science and Engineering, pp. 1–12, 2024

    2024

  12. [20]

    Vision-based manipulators need to also see from their hands

    Kyle Hsu, Moo Jin Kim, Rafael Rafailov, Jiajun Wu, and Chelsea Finn. Vision-based manipulators need to also see from their hands. InInternational Conference on Learning Representations, 2022

    2022

  13. [21]

    Dogar, and Pietro Valdastri

    Junlei Hu, Dominic Jones, Mehmet R. Dogar, and Pietro Valdastri. Occlusion-robust autonomous robotic manipulation of human soft tissues with 3-d surface feedback.IEEE Transactions on Robotics, 40:624–638, 2024

    2024

  14. [22]

    Towardgeneral-purposerobotsviafoundationmodels: Asurveyandmeta-analysis

    Yafei Hu, Quanting Xie, Vidhi Jain, Jonathan Francis, Jay Patrikar, Nikhil Keetha, Seungchan Kim, Yaqi Xie, Tianyi Zhang, Shibo Zhao, Yu-Quan Chong, Chen Wang, Katia Sycara, Matthew Johnson-Roberson, Dhruv Batra, Xiaolong Wang, Sebastian Scherer, Zsolt Kira, Fei Xia, and Yonat...

    2023

  15. [23]

    Efficiently calibrating cable-driven surgical robots with rgbd fiducial sensing and recurrent neural networks.IEEE Robotics and Automation Letters, 5 (4):5937–5944, 2020

    Minho Hwang, Brijen Thananjeyan, Samuel Paradis, Daniel Seita, Jeffrey Ichnowski, Danyal Fer, Thomas Low, and Ken Goldberg. Efficiently calibrating cable-driven surgical robots with rgbd fiducial sensing and recurrent neural networks.IEEE Robotics and Automation Letters, 5 (4)...

    2020

  16. [24]

    Minho Hwang, Jeffrey Ichnowski, Brijen Thananjeyan, Daniel Seita, Samuel Paradis, Danyal Fer, Thomas Low, and Ken Goldberg. Automating surgical peg transfer: Calibration with deep learning can exceed speed, accuracy, and consistency of humans.IEEE Transactions on Automation Sc...

    2022

  17. [25]

    Hg- dagger: Interactive imitation learning with human experts

    Michael Kelly, Chelsea Sidrane, Katherine Driggs-Campbell, and Mykel J Kochenderfer. Hg- dagger: Interactive imitation learning with human experts. In2019 International Conference on Robotics and Automation (ICRA), pp. 8077–8083, 2019

    2019

  18. [26]

    Zhao, Samuel Schmidgall, Anton Deguet, Marin Kobilarov, Chelsea Finn, and Axel Krieger

    Ji Woong Kim, Tony Z. Zhao, Samuel Schmidgall, Anton Deguet, Marin Kobilarov, Chelsea Finn, and Axel Krieger. Surgical robot transformer (SRT): Imitation learning for surgical subtasks. In 8th Annual Conference on Robot Learning, 2024

    2024

  19. [27]

    Autonomous medical needle steering in vivo.Science Robotics, 8(82):eadf7614, 2023

    Alan Kuntz, Maxwell Emerson, Tayfun Efe Ertop, Inbar Fried, Mengyu Fu, Janine Hoelscher, Margaret Rox, Jason Akulian, Erin A Gillaspie, Yueh Z Lee, et al. Autonomous medical needle steering in vivo.Science Robotics, 8(82):eadf7614, 2023

    2023

  20. [28]

    Cyberknife: A new paradigm in radiotherapy, 2010

    Gopalakrishna Kurup. Cyberknife: A new paradigm in radiotherapy, 2010

    2010

  21. [29]

    Medic: Autonomous surgical robotic assistance to maximizing exposure for dissection and cautery

    Xiao Liang, Chung-Pang Wang, Nikhil Uday Shinde, Fei Liu, Florian Richter, and Michael Yip. Medic: Autonomous surgical robotic assistance to maximizing exposure for dissection and cautery. arXiv preprint arXiv:2409.14287, 2024. 23 SRT-H: A Hierarchical Framework for Autonomous...

    2024 arXiv

  22. [30]

    Authoritytransferaccording to a driver intervention intention considering coexistence of communication delay.Computers, 12(11):228, 2023

    TaeyoonLim, MyeonghwanHwang, EugeneKim, andHyunrokCha. Authoritytransferaccording to a driver intervention intention considering coexistence of communication delay.Computers, 12(11):228, 2023

    2023

  23. [31]

    Llava-next: Improved reasoning, ocr, and world knowledge, January 2024

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasoning, ocr, and world knowledge, January 2024. URLhttps: //llava-vl.github.io/blog/2024-01-30-llava-next/

    2024

  24. [32]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 10012–10022, October 2021

    2021

  25. [33]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. InInternational Conference on Learning Representations, 2019

    2019

  26. [34]

    How do i do it: laparoscopic cholecys- tectomy

    Arnab Majumder, Maria S Altieri, and L Michael Brunt. How do i do it: laparoscopic cholecys- tectomy. Annals of Laparoscopic and Endoscopic Surgery, 5, 2020

    2020

  27. [35]

    The zeus robotic system: experimental and clinical applications

    Jacques Marescaux and Francesco Rubino. The zeus robotic system: experimental and clinical applications. Surgical Clinics, 83(6):1305–1315, 2003

    2003

  28. [36]

    Nagy and Tamás Haidegger

    Tamás D. Nagy and Tamás Haidegger. Performance and capability assessment in surgical subtask automation.Sensors, 22(7), 2022. ISSN 1424-8220. doi: 10.3390/s22072501. URL https://www.mdpi.com/1424-8220/22/7/2501

    2022 doi

  29. [37]

    A simple solution to lens fogging during robotic and laparoscopic surgery

    Camran Nezhat and Vadim Morozov. A simple solution to lens fogging during robotic and laparoscopic surgery. JSLS: Journal of the Society of Laparoendoscopic Surgeons, 12(4):431, Oct–Dec 2008

    2008

  30. [38]

    Yafei Ou and Mahdi Tavakoli. Sim-to-real surgical robot learning and autonomous planning for internal tissue points manipulation using reinforcement learning.IEEE Robotics and Automation Letters, 8(5):2502–2509, 2023

    2023

  31. [39]

    Film: Visual reasoning with a general conditioning layer

    Ethan Perez, Florian Strub, Harm De Vries, Vincent Dumoulin, and Aaron Courville. Film: Visual reasoning with a general conditioning layer. InProceedings of the AAAI conference on artificial intelligence, volume 32, 2018

    2018

  32. [40]

    Profile: Veebot drawing blood faster and more safely than a human can, 2013

    Tekla S Perry. Profile: Veebot drawing blood faster and more safely than a human can, 2013

    2013

  33. [41]

    Learning from demonstrations for autonomous soft-tissue retraction

    Ameya Pore, Eleonora Tagliabue, Marco Piccinelli, Diego Dall’Alba, Alicia Casals, and Paolo Fior- ini. Learning from demonstrations for autonomous soft-tissue retraction. In2021 International Symposium on Medical Robotics (ISMR), pp. 1–7, 2021

    2021

  34. [42]

    Using robotics to move a neurosurgeon’s hands to the tip of their endoscope.Science Robotics, 8(82):eadg6042, 2023

    Karl Price, Joseph Peine, Margherita Mencattelli, Yash Chitalia, David Pu, Thomas Looi, Scellig Stone, James Drake, and Pierre E Dupont. Using robotics to move a neurosurgeon’s hands to the tip of their endoscope.Science Robotics, 8(82):eadg6042, 2023

    2023

  35. [43]

    A generalist agent.Transactions on Machine Learning Research, 2022

    Scott Reed, Konrad Zolna, Emilio Parisotto, Sergio Gómez Colmenarejo, Alexander Novikov, Gabriel Barth-maron, Mai Giménez, Yury Sulsky, Jackie Kay, Jost Tobias Springenberg, Tom Eccles, Jake Bruce, Ali Razavi, Ashley Edwards, Nicolas Heess, Yutian Chen, Raia Hadsell, Oriol Vin...

    2022

  36. [44]

    Allen Z. Ren, Anushri Dixit, Alexandra Bodrova, Sumeet Singh, Stephen Tu, Noah Brown, Peng Xu, Leila Takayama, Fei Xia, Jake Varley, Zhenjia Xu, Dorsa Sadigh, Andy Zeng, and Anirudha Majumdar. Robots that ask for help: Uncertainty alignment for large language model planners. I...

    2023

  37. [45]

    A reduction of imitation learning and struc- tured prediction to no-regret online learning

    Stephane Ross, Geoffrey Gordon, and Drew Bagnell. A reduction of imitation learning and struc- tured prediction to no-regret online learning. In Geoffrey Gordon, David Dunson, and Miroslav Dudík (eds.),Proceedings of the Fourteenth International Conference on Artificial Intell...

    2011

  38. [46]

    Autonomous robotic laparoscopic surgery for intestinal anastomosis

    Hamed Saeidi, Justin D Opfermann, Michael Kam, Shuwen Wei, Simon Léonard, Michael H Hsieh, Jin U Kang, and Axel Krieger. Autonomous robotic laparoscopic surgery for intestinal anastomosis. Science robotics, 7(62):eabj2908, 2022

    2022

  39. [47]

    DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter

    Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. In5th Workshop on Energy Efficient Machine Learning and Cognitive Computing @ NeurIPS 2019, 2019

    2019

  40. [48]

    Optics for music: a new (sub) millimeter camera for the caltech submillimeter observatory

    Jack Sayers, Nicole G Czakon, Peter K Day, Thomas P Downes, Ran P Duan, Jiansong Gao, Jason Glenn, Sunil R Golwala, Matt I Hollister, Henry G LeDuc, et al. Optics for music: a new (sub) millimeter camera for the caltech submillimeter observatory. InMillimeter, Submillimeter, a...

    2010

  41. [49]

    Lapgym - an open source framework for reinforcement learning in robot-assisted laparoscopic surgery.Journal of Machine Learning Research, 24(368): 1–42, 2023

    Paul Maria Scheikl, Balázs Gyenes, Rayan Younis, Christoph Haas, Gerhard Neumann, Franziska Mathis-Ullrich, and Martin Wagner. Lapgym - an open source framework for reinforcement learning in robot-assisted laparoscopic surgery.Journal of Machine Learning Research, 24(368): 1–42, 2023

    2023

  42. [50]

    Sim-to-real transfer for visual reinforcement learning of deformable object manipulation for robot-assisted surgery.IEEE Robotics and Automation Letters, 8(2):560–567, 2023

    Paul Maria Scheikl, Eleonora Tagliabue, Balázs Gyenes, Martin Wagner, Diego Dall’Alba, Paolo Fiorini, and Franziska Mathis-Ullrich. Sim-to-real transfer for visual reinforcement learning of deformable object manipulation for robot-assisted surgery.IEEE Robotics and Automation ...

    2023

  43. [51]

    Movement primitive diffusion: Learning gentle robotic manipulation of deformable objects.IEEE Robotics and Automation Letters, 9(6):5338– 5345, 2024

    Paul Maria Scheikl, Nicolas Schreiber, Christoph Haas, Niklas Freymuth, Gerhard Neumann, Rudolf Lioutikov, and Franziska Mathis-Ullrich. Movement primitive diffusion: Learning gentle robotic manipulation of deformable objects.IEEE Robotics and Automation Letters, 9(6):5338– 5345, 2024

    2024

  44. [52]

    General- purpose foundation models for increased autonomy in robot-assisted surgery.arXiv preprint arXiv:2401.00678, 2024

    Samuel Schmidgall, Ji Woong Kim, Alan Kuntz, Ahmed Ezzat Ghazi, and Axel Krieger. General- purpose foundation models for increased autonomy in robot-assisted surgery.arXiv preprint arXiv:2401.00678, 2024

    2024 arXiv

  45. [53]

    Supervised autonomous robotic soft tissue surgery.Science translational medicine, 8(337):337ra64–337ra64, 2016

    Azad Shademan, Ryan S Decker, Justin D Opfermann, Simon Leonard, Axel Krieger, and Peter CW Kim. Supervised autonomous robotic soft tissue surgery.Science translational medicine, 8(337):337ra64–337ra64, 2016

    2016

  46. [54]

    Zhao, Archit Sharma, Karl Pertsch, Jianlan Luo, Sergey Levine, and Chelsea Finn

    Lucy Xiaoyang Shi, Zheyuan Hu, Tony Z. Zhao, Archit Sharma, Karl Pertsch, Jianlan Luo, Sergey Levine, and Chelsea Finn. Yell at your robot: Improving on-the-fly from language corrections. In Robotics: Science and Systems, 2024. 25 SRT-H: A Hierarchical Framework for Autonomous...

    2024

  47. [55]

    Dutson, and Jacob Rosen

    Changyeob Shin, Peter Walker Ferguson, Sahba Aghajani Pedram, Ji Ma, Erik P. Dutson, and Jacob Rosen. Autonomous tissue manipulation via surgical robot using learning based model predictive control. In2019 International Conference on Robotics and Automation (ICRA), pp. 3875–3881, 2019

    2019

  48. [56]

    Toward teaching by demonstration for robot-assisted minimally invasive surgery

    Hang Su, Andrea Mariani, Salih Ertug Ovur, Arianna Menciassi, Giancarlo Ferrigno, and Elena De Momi. Toward teaching by demonstration for robot-assisted minimally invasive surgery. IEEE Transactions on Automation Science and Engineering, 18(2):484–494, 2021

    2021

  49. [57]

    Efficientnet: Rethinking model scaling for convolutional neural networks

    Mingxing Tan. Efficientnet: Rethinking model scaling for convolutional neural networks. In Proceedings of the International Conference on Machine Learning (ICML), 2019

    2019

  50. [58]

    Sequential robot imitation learning from observations.The International Journal of Robotics Research, 40 (10-11):1306–1325, 2021

    Ajay Kumar Tanwani, Andy Yan, Jonathan Lee, Sylvain Calinon, and Ken Goldberg. Sequential robot imitation learning from observations.The International Journal of Robotics Research, 40 (10-11):1306–1325, 2021

    2021

  51. [59]

    Multilateral surgical pattern cutting in 2d orthotropic gauze with deep reinforcement learningpoliciesfortensioning

    Brijen Thananjeyan, Animesh Garg, Sanjay Krishnan, Carolyn Chen, Lauren Miller, and Ken Goldberg. Multilateral surgical pattern cutting in 2d orthotropic gauze with deep reinforcement learningpoliciesfortensioning. In 2017IEEEInternationalConferenceonRoboticsandAutomation (ICR...

    2017

  52. [60]

    Attention is all you need.Advances in Neural Information Processing Systems, 2017

    A Vaswani. Attention is all you need.Advances in Neural Information Processing Systems, 2017

    2017

  53. [61]

    Surrol: An open-source reinforcement learning centered and dvrk compatible platform for surgical robot learning

    Jiaqi Xu, Bin Li, Bo Lu, Yun-Hui Liu, Qi Dou, and Pheng-Ann Heng. Surrol: An open-source reinforcement learning centered and dvrk compatible platform for surgical robot learning. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1821–1828....

    2021

  54. [62]

    Orbit- surgical: An open-simulation framework for learning surgical augmented dexterity.arXiv preprint arXiv:2404.16027, 2024

    Qinxi Yu, Masoud Moghani, Karthik Dharmarajan, Vincent Schorp, William Chung-Ho Pan- itch, Jingzhou Liu, Kush Hari, Huang Huang, Mayank Mittal, Ken Goldberg, et al. Orbit- surgical: An open-simulation framework for learning surgical augmented dexterity.arXiv preprint arXiv:240...

    2024 arXiv

  55. [63]

    Toward automated vehicle teleoperation: Vision, opportunities, and challenges

    Tao Zhang. Toward automated vehicle teleoperation: Vision, opportunities, and challenges. IEEE Internet of Things Journal, 7(12):11347–11354, 2020

    2020

  56. [64]

    Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn

    Tony Z. Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn. Learning fine-grained bimanual manipulation with low-cost hardware. InICML Workshop on New Frontiers in Learning, Control, and Dynamical Systems, 2023

    2023

  57. [65]

    On the continuity of rotation representations in neural networks

    Yi Zhou, Connelly Barnes, Lu Jingwan, Yang Jimei, and Li Hao. On the continuity of rotation representations in neural networks. InThe IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019

    2019

  58. [66]

    Imitation learning for surgical robots with kinematics errors using self-corrections

    Brianna Zitkovich, Tianhe Yu, Sichun Xu, Peng Xu, Ted Xiao, Fei Xia, Jialin Wu, Paul Wohlhart, Stefan Welker, Ayzaan Wahid, Quan Vuong, Vincent Vanhoucke, Huong Tran, Radu Soricut, Anikait Singh, Jaspiar Singh, Pierre Sermanet, Pannag R. Sanketi, Grecia Salazar, Michael S. Ryo...

    2023

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.