Pith. sign in

REVIEW 2 major objections 6 minor 58 references

SonoGym: High Performance Simulation for Challenging Surgical Tasks with Robotic Ultrasound

T0 review · 2 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read SonoGym is a scalable robotic-ultrasound simulator that trains deep RL and imitation-learning policies for navigation, surface reconstruction, and ultrasound-guided spine drilling.

desk verdict Genuinely useful open platform for robotic ultrasound research, but the 'out-of-domain' test varies only GAN seeds and the one true held-out patient test shows large drops, so the sim-to-real conclusion is overclaimed. read the letter →

arxiv 2507.01152 v1 pith:2LRYV3DB submitted 2025-07-01 cs.RO

Yunke Ao , Masoud Moghani , Mayank Mittal , Manish Prajapat , Luohong Wu , Frederic Giraud , Fabio Carrillo , Andreas Krause
show 1 more author
Philipp Fürnstahl
This is my paper · ORCID
classification cs.RO
keywords roboticultrasoundsimulationplatformdeepreinforcementlearningimitationimagespinesurgerypediclescrewplacementsubmodular
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Robotic ultrasound could make spinal procedures more reproducible, but deep reinforcement learning and imitation learning have not been used much there because no fast, realistic simulator existed. SonoGym attacks that gap: it renders ultrasound images from CT-derived patient models two ways, with a physics-based ray-tracing model and with a generative network, and it runs tens to hundreds of environments in parallel so agents can collect experience quickly. The paper's central claim is that with this combination, learning-based agents actually train well: PPO reaches close-to-expert performance on navigation and surgery, and submodular PPO and A2C beat a heuristic scanning path on bone-surface reconstruction. The authors also quantify where the approach is not yet reliable, especially when a policy trained on several patients is tested on a new patient, where the safe ratio in the drilling task drops to 52.86%. That mix of demonstrated training success and honest remaining gaps is the contribution: a platform that lets the community work on the hard parts instead of rebuilding simulators.

What carries the argument

The load-bearing mechanism is the ultrasound image-rendering pipeline: for each of dozens of parallel environments, the current end-effector pose defines an ultrasound image plane inside the patient coordinate frame, and the simulation slices the 3D CT volume and its segmentation map along that plane to produce a 2D CT slice and a label slice. From those inputs, either the model-based branch computes reflection and backscattering with a convolutional ray-tracing model, with acoustic impedance set from CT intensity, or the learning-based branch feeds the CT slice to a generative image-translation network. This batch pipeline renders a 200 by 150 image for 100 environments in 0.0089 seconds for the model-based branch and 0.1107 seconds for the learning-based branch on an RTX 3090 Ti, which is what makes PPO training feasible in about 2 hours for model-based and 10 hours for learning-based simulation. The task-specific MDP formulations are the second piece: a partially observable MDP for navigation, a submodular MDP with marginal-gain rewards for reconstruction, and a state-wise constrained MDP with a safety filter for surgery.

What would settle it

Run a policy trained in SonoGym on a real robotic ultrasound system with a cadaver or volunteer and measure the same metrics used in the paper, such as navigation position and rotation error or the surgery safe ratio; if the safe ratio falls near the paper's own held-out-patient value of 52.86% or navigation error exceeds the reported band of 16 mm, the central sim-to-real premise fails. A cheaper first check is to compute LPIPS or SSIM between SonoGym's generated images and real ultrasound images from a specimen not used in training: if the generative outputs are no closer to real ultrasound than the model-based outputs are, the learning-based branch loses its claimed advantage.

Watch

Extended reading notes

Core claim

SonoGym's thesis is that the missing piece for robot learning in robotic ultrasound is not the learning algorithm but the training environment. The paper shows that with real-time parallel ultrasound simulation—both a convolutional ray-tracing model and a generative network trained on paired CT-ultrasound data from seven ex-vivo spine specimens—PPO agents train stably and achieve close-to-expert performance, outperforming A2C on navigation and surgery, while submodular PPO and A2C surpass the heuristic open-loop trajectory used in prior reconstruction work. The environment encodes the three tasks as specialized MDPs: navigation as a partially observable MDP with ultrasound images as observations, reconstruction as a submodular MDP whose reward is the marginal gain in covered bone-surface area, and surgery as a state-wise constrained MDP with an unsafe-region cost. On generalization, the paper claims that training against several ultrasound noise networks keeps navigation errors within acceptable ranges (below 16 mm in position and 12 degrees in rotation) and keeps surgical performance roughly level, while generalization to a held-out patient remains a clear failure mode, with the safe ratio falling to 52.86%.

Load-bearing premise

The load-bearing premise is that ultrasound images generated from CT volumes—in particular those from a generative network trained on seven ex-vivo spine specimens—are realistic enough that policies trained in SonoGym would behave the same way on real ultrasound and real patients, an assumption the paper does not test outside simulation.

Editorial extensions

If this is right

  • PPO agents trained from ultrasound image observations can reach near-expert performance on navigation and surgery, so image-based policies are viable when the true anatomical pose is unknown.
  • Submodular rewards based on the marginal gain in covered bone-surface area let reconstruction agents beat the heuristic open-loop scanning path, producing higher coverage with lower rotation and path length.
  • Training with several ultrasound-generator models keeps navigation and surgery performance roughly in-domain, which the authors read as evidence that the imaging-domain sim-to-real gap is addressable.
  • Imitation-learned ACT and diffusion policies train successfully on navigation and ACT on surgery, but PPO with a safety filter is more safety-aware in the drilling task, while ACT achieves better insertion accuracy, so the two families trade off.
  • Inter-patient generalization remains unsolved: on a held-out sixth patient, PPO's safe ratio drops to 52.86% and side error rises to 27.3 mm, so patient diversity in the training set is a key bottleneck.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the out-of-domain imaging results carry to a real machine, sim-to-real transfer could be achieved by training against several simulated ultrasound styles rather than collecting real ultrasound data; the paper does not make this claim.
  • Adding anatomical variability to the training patients may improve held-out-patient performance more than improving image fidelity, because the sharpest reported failure is inter-patient, not inter-noise, generalization.
  • A testable extension is to train on both model-based and learning-based images at once, randomizing over simulation branches, which could improve generalization beyond either branch alone.
  • We infer that the submodular-MDP coverage formulation applies beyond ultrasound, for example to laparoscopic surface scanning, where coverage and path length compete in the same way.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper presents SonoGym, a parallel robotic ultrasound simulation platform that provides both physics-based and learning-based (pix2pix) ultrasound image generation from CT-derived patient models. It proposes three surgical tasks—navigation, bone surface reconstruction, and ultrasound-guided pedicle screw drilling—and formulates them within MDP, submodular MDP, and state-wise constrained MDP frameworks. The authors benchmark PPO, A2C, a SafeRPlan safety-filtered PPO, ACT, and Diffusion Policy across these tasks, reporting learning curves, quantitative task metrics, and timing measurements. The main claims are that the platform enables stable policy learning across tasks, is efficient enough for parallel training, and that generalization experiments suggest the potential of sim-to-real transfer over the ultrasound imaging domain while cross-patient generalization remains challenging.

Significance. SonoGym addresses a genuine gap: existing surgical simulation platforms focus on laparoscopic or soft-tissue manipulation and rarely include patient-specific intraoperative imaging modalities such as ultrasound. The open-source release, expert demonstration datasets, and benchmarks for multiple RL/IL algorithms are valuable community assets. The honest reporting of unsuccessful baselines (SAC, PPO-Lagrangian, Decision Transformer on surgery) strengthens the paper's credibility. The computational efficiency results support the title's 'high performance' claim. If the simulation realism and transfer claims are appropriately qualified, the platform is a useful contribution to robot learning for robotic ultrasound.

major comments (2)
  1. [Section 5.2 Q4; Fig. 6; Table 1] The condition labeled 'Out-of-Domain Test (ODT)' for the learning-based ultrasound simulation is not an out-of-domain test as described in Section 5 (Experiment setup): it trains agents on four pix2pix networks and tests on a fifth, where all five networks share the same paired ex-vivo CT-US dataset and differ only by random seed. This varies generator noise/texture, not the imaging domain (anatomy distribution, transducer, acquisition physics). Therefore the conclusions that the gaps between LB_ODT and LB 'demonstrate the potential of sim-to-real transfer over the ultrasound imaging domain' (Fig. 6 caption) and that training with multiple networks 'address the sim-to-real gap between images' (Section 5.2 Q4) are unsupported. The only genuinely held-out domain evaluation, the new-patient surgery test in Table 6, shows large degradation (PPO safe ratio 52.86%, side error 27.3 ± 36.4 mm vs. 5.42 ± 4.9 mm in the in-domain result in Table 5). The authors should re-label this condition (e.g., 'generator randomization') or, ideally, evaluate on real ultrasound images or a distinct data distribution before claiming evidence for sim-to-real transfer.
  2. [Section 5.1, Q1] The quantitative evaluation of ultrasound realism reports LPIPS 0.2415, SSIM 0.3940, and PSNR 15.96 as point estimates without standard deviations, without variance across the five trained networks, and without a quantitative comparison to the model-based simulator or to existing baselines such as [8], despite the claim that the values are 'close' to [8]. Because the realism of the learning-based simulation is a load-bearing premise for the transfer claims, the authors should report variance across networks and include a quantitative comparison with the model-based approach and prior ultrasound simulation methods.
minor comments (6)
  1. [Table 1] The arrow in the table header 'safe ratio ↓[%]' is inverted; higher safe ratio is better, so it should be '↑[%]'.
  2. [Abstract] The platform is consistently called 'SonoGym' except for the spelling 'Sonogym' in the abstract; please unify the spelling throughout.
  3. [Section 4.1, reward equation] The rotation error term in the navigation reward appears without the weight w1, although the text states that w1 balances position error (in mm) and rotation error (in rad); please clarify whether w1 multiplies both terms.
  4. [Section 5.1, timing results] The reported timings of 0.0089 s and 0.1107 s for 100 environments do not specify whether the measurement includes slice extraction plus rendering or only the ultrasound renderer, nor the GPU batch size; please state the measurement boundary.
  5. [Table 1, caption] The caption states 'comparable performance between ODT and IDT for LB' without significance tests or standard deviations; please provide error bars or statistical tests to support this claim, since the table otherwise omits standard deviations.
  6. [Abstract] The abstract refers to 'vision transformers' for IL agents, but the paper trains ACT and Diffusion Policy; ACT uses a transformer within a CVAE but is not a vision transformer in the standard sense, so more precise terminology is recommended.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: SonoGym's performance claims are measured in-simulation, and self-cited algorithms are used as off-the-shelf tools rather than as load-bearing evidence.

full rationale

SonoGym is an empirical systems paper, not a derivation chain. The central claims—that PPO/A2C/IL agents can be trained on the platform and outperform baselines—are supported by in-simulator measurements (learning curves, coverage ratios, errors, safe ratios) reported in the paper, not by any fitted parameter being renamed as a prediction. The paper cites the authors' own prior work for submodular RL [37] and SafeRPlan [3], but those are used as implemented algorithms and then independently evaluated in the new environments; the performance numbers do not reduce to those citations. The learning-based ultrasound simulator is a pix2pix model trained on a separately collected paired CT-US dataset, and the physics-based simulator follows external methods [38, 21]; neither is defined in terms of the downstream task results. The reconstruction objective F includes the same coverage quantity used in evaluation, but that is standard reward design and the heuristic baseline is measured under the same metric and is surpassed, so it is not circular. The paper's 'Out-of-Domain Test' (ODT) varies only random-seed-trained GAN networks on the same paired ex-vivo data, so the conclusion about 'sim-to-real transfer over the ultrasound imaging domain' is overstated; Table 6 itself shows large cross-patient degradation (safe ratio as low as 52.86%). This is an external-validity and experimental-design concern, not circularity: the ODT results are measurements, not outputs forced by construction or by a self-citation chain. No equation reduces to its own input, and no load-bearing uniqueness claim is imported from the authors' prior work.

Assumptions & free parameters 7 free parameters · 6 assumptions · 0 invented entities

The platform's central value depends on the realism of its ultrasound simulators. Both the physics-based and GAN parameters come from prior work, not from a derivation in this paper. The tunable reward weights and noise levels are hand-set. No new physical entities are postulated.

free parameters (7)
  • Reward weight w1 (navigation) = 0.045
    Balances position error (mm) and rotation error (rad); chosen by hand in Appendix C.1.
  • Reward weights w2, w3 (reconstruction) = 0.01, 1
    Balance surface coverage vs path length and rotation; chosen by hand in Appendix C.2.
  • Reward weights w4, w5, w6 (surgery) = 30, 5, 300
    Encourage reaching the skin point and minimizing side error during drilling; chosen by hand in Appendix C.3.
  • Missing probability for simulated bone segmentation (reconstruction) = 20%
    Simulates segmentation noise for each ground-truth surface point; chosen in Appendix C.2.
  • Safety margin delta (SafeRPlan safety filter) = not specified
    Conservative margin for cost threshold in surgery; introduced in Section 3.1.
  • Ultrasound probe position randomization threshold lambda = 5 mm in tangential axes
    Noise in ultrasound-based navigation for the surgery task; set in Appendix C.3.
  • Expert policy scaling parameter rho1 (navigation) = not specified (0 < rho1 < 1)
    Proportional control gain for collecting expert demonstrations; defined in Section 4.1.
assumptions (6)
  • domain assumption The physics-based ultrasound model (Salehi et al. 2015, Kutter et al. 2009) accurately approximates real ultrasound image formation for the spine.
    The platform's model-based ultrasound images are treated as realistic training inputs; Section 4 (Ultrasound simulation) and Section 5.1.
  • domain assumption The pix2pix GAN trained on seven ex-vivo spine specimens generalizes to in-vivo TotalSegmentator CT volumes.
    The learning-based ultrasound simulation applies the GAN to patient CTs not in the training distribution; Section 4 and Table 6.
  • domain assumption A low-level robot controller maintains probe-skin contact and perpendicular orientation during navigation.
    Explicitly assumed in Section 4.1; the planners only control tangential motion.
  • domain assumption Bone surface can be segmented from each 2D ultrasound image with a controllable missing-point rate.
    Assumed in Section 4.2; segmentation is simulated with 20% missing probability rather than learned.
  • standard math Submodular reward maximization via marginal gain (Prajapat et al. 2024) is a valid approximation for the reconstruction objective.
    Used in Section 4.2 to turn a set-function objective into per-step rewards.
  • domain assumption State-wise constrained MDP and the SafeRPlan safety filter provide valid safety guarantees in the surgery task.
    The modified SafeRPlan predicts cost and blocks actions; validity in simulation is assessed only by safe-ratio metrics, Section 3.1 and 5.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SonoGym: High Performance Simulation for Challenging Surgical Tasks with Robotic Ultrasound." pith.science (2026). https://pith.science/paper/2LRYV3DB

@misc{pith2026250701152,
  author       = {Pith},
  title        = {Pith review of: SonoGym: High Performance Simulation for Challenging Surgical Tasks with Robotic Ultrasound},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2LRYV3DB}},
  note         = {Machine review of arXiv:2507.01152}
}
read the original abstract

Ultrasound (US) is a widely used medical imaging modality due to its real-time capabilities, non-invasive nature, and cost-effectiveness. Robotic ultrasound can further enhance its utility by reducing operator dependence and improving access to complex anatomical regions. For this, while deep reinforcement learning (DRL) and imitation learning (IL) have shown potential for autonomous navigation, their use in complex surgical tasks such as anatomy reconstruction and surgical guidance remains limited -- largely due to the lack of realistic and efficient simulation environments tailored to these tasks. We introduce SonoGym, a scalable simulation platform for complex robotic ultrasound tasks that enables parallel simulation across tens to hundreds of environments. Our framework supports realistic and real-time simulation of US data from CT-derived 3D models of the anatomy through both a physics-based and a generative modeling approach. Sonogym enables the training of DRL and recent IL agents (vision transformers and diffusion policies) for relevant tasks in robotic orthopedic surgery by integrating common robotic platforms and orthopedic end effectors. We further incorporate submodular DRL -- a recent method that handles history-dependent rewards -- for anatomy reconstruction and safe reinforcement learning for surgery. Our results demonstrate successful policy learning across a range of scenarios, while also highlighting the limitations of current methods in clinically relevant environments. We believe our simulation can facilitate research in robot learning approaches for such challenging robotic surgery applications. Dataset, codes, and videos are publicly available at https://sonogym.github.io/.

Figures

Figures reproduced from arXiv: 2507.01152 by the authors.

Figure 1
Figure 1. Overview. SonoGym provides model-based and learning-based ultrasound (US) simulation using 3D label map and CT scans from real patient datasets. Tasks in SonoGym include US navigation, anatomy reconstruction, and US-guided robotic surgery. SonoGym enables benchmarking of various algorithms, including reinforcement learning (RL), safe RL, vision transformer, and diffusion policy. improve imaging efficiency [17]. Addi… view at source ↗
Figure 2
Figure 2. Efficient ultrasound simulation across a large number of environments. Given the current end-effector poses of the robot arms, we first compute the ultrasound image planes in the patient frames attached with 3D CT volumes and label maps. We then extract 2D CT and label slices as pixels on the plane. Ultrasound images are subsequently simulated based on these inputs using either physics-based models or neural network… view at source ↗
Figure 3
Figure 3. Tasks. The target anatomy (L4 vertebra) is colored dark blue. (a) Ultrasound (US) navigation: Move the ultrasound probe to the goal pose (green) based on the real-time ultrasound images. (b) Bone surface reconstruction: Efficiently scan the surface of the target vertebra (green) with a low path length. (c) Ultrasound-guided spinal surgery: Fix the ultrasound probe to track the target vertebra, and drill inside the v… view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Qualitative evaluation of ultrasound (US) simulation. (left) Evaluation with the testing dataset. The top row shows 2D CT slices of the human specimen. The corresponding real US and the generated US of Sonogym are shown in the rows below. (right) Examples of simulated …
Figure 5
Figure 5. Figure 5: Learning curves of reinforcement learning agents for all tasks. The shaded region represents the 1-sigma confidence interval across training runs with five different random seeds. Our modeling allows stable training of PPO agents, which can achieve close performance to…
Figure 6
Figure 6. Figure 6: Performance for navigation and reconstruction. Results are averaged over 100 trials, and error bars denote the standard deviation. The gaps between LB_ODT and LB are not significant, which demonstrates the potential of sim-to-real transfer over the ultrasound imaging d…
Figure 7
Figure 7. Figure 7: Learned and heuristic trajectories for the Reconstruction task. (left) The learned tra￾jectory exhibits a circular pattern around the target vertebra; (right) the fixed heuristic trajectory. Q2: How effective are the MDP formula￾tions and reward design for different ta…
Figure 8
Figure 8. Figure 8 [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 9
Figure 9. Figure 9: Patient anatomy data from TotalSegmentator dataset. We provide ultrasound simulation based on diverse real patient models, including CT volume and segmentation. The noise maps N0 and N1 are sampled over the 3D space at initialization and remain fixed throughout the onl…
Figure 10
Figure 10. Figure 10: Top-down view of region of manipulating the probe. Environment settings The initial 2D pose of the ultra￾sound probe is randomized within a 130 × 130 [mm2 ] region on the frontal plane of the patient, like the region shown in [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]
Figure 11
Figure 11. Figure 11: U-Net architecture used for learning-based ultrasound simulation. Dashed lines indicate residual connections within the U-Net. Convolutional (blue) and transposed convolutional (cyan) layers are annotated with (kernel size)×(kernel size)×(number of channels). The valu…
Figure 12
Figure 12. Figure 12: Network architecture for (a) navigation, (b) reconstruction and (c) surgery. The convolution layers are represented by (kernal size)×(kernal size)×(number of channels). The numbers in the blue block on the right is the numbers of strides [PITH_FULL_IMAGE:figures/full…
Figure 13
Figure 13. Figure 13: Additional qualitative results of ultrasound simulation. The top, middle, and down rows are CT slices, learning-based ultrasound simulation and model-based ultrasound simulation, respectively. These feature streams are then concatenated and processed by an additional …
Figure 14
Figure 14. Figure 14: Variation between ultrasound simulation models. The input CT slice is shown left, and the other 5 ultrasound images are generated from different models with the same CT input [PITH_FULL_IMAGE:figures/full_fig_p020_14.png]
Figure 15
Figure 15. Figure 15: Reconstructed surfaces. The reconstructed points and uncovered points on the bone surface are colored yellow and green, respectively. The DRL policy has higher coverage from the back and side views. What about other agents? We also trained Soft Actor-Critic (SAC) agen…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

58 extracted references · 41 canonical work pages

  1. [8]

    Diffusion as sound propagation: Physics-inspired model for ultrasound image generation

    Marina Domínguez, Yordanka Velikova, Nassir Navab, and Mohammad Farid Azampour. Diffusion as sound propagation: Physics-inspired model for ultrasound image generation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 613–623. Springer, 2024

    work page 2024

  2. [1]

    Solving rubik’s cube with a robot hand

    Ilge Akkaya, Marcin Andrychowicz, Maciek Chociej, Mateusz Litwin, Bob McGrew, Arthur Petron, Alex Paino, Matthias Plappert, Glenn Powell, Raphael Ribas, et al. Solving rubik’s cube with a robot hand. arXiv preprint arXiv:1910.07113, 2019

    arXiv 1910

  3. [2]

    Gan- based realistic bone ultrasound image and label synthesis for improved segmentation

    Ahmed Z Alsinan, Charles Rule, Michael Vives, Vishal M Patel, and Ilker Hacihaliloglu. Gan- based realistic bone ultrasound image and label synthesis for improved segmentation. InMedical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part VI 23, pages 795–804. Spri...

    work page 2020

  4. [3]

    Saferplan: Safe deep reinforcement learning for intraoperative planning of pedicle screw placement

    Yunke Ao, Hooman Esfandiari, Fabio Carrillo, Christoph J Laux, Yarden As, Ruixuan Li, Kaat Van Assche, Ayoob Davoodi, Nicola A Cavalcanti, Mazda Farshad, et al. Saferplan: Safe deep reinforcement learning for intraoperative planning of pedicle screw placement. Medical Image Analysis, 99:103345, 2025

    work page 2025

  5. [4]

    Real-time gpu-based ultrasound simulation using deformable mesh models

    Benny Burger, Sascha Bettinghausen, Matthias Radle, and Jürgen Hesser. Real-time gpu-based ultrasound simulation using deformable mesh models. IEEE transactions on medical imaging, 32(3):609–618, 2012

    work page 2012

  6. [5]

    Ultrasound image-to-video synthesis via latent dynamic diffusion models

    Tingxiu Chen, Yilei Shi, Zixuan Zheng, Bingcong Yan, Jingliang Hu, Xiao Xiang Zhu, and Lichao Mou. Ultrasound image-to-video synthesis via latent dynamic diffusion models. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 764–774. Springer, 2024

    work page 2024

  7. [6]

    Diffusion policy: Visuomotor policy learning via action diffusion

    Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. The International Journal of Robotics Research, page 02783649241273668, 2023

    2023

  8. [7]

    Pybullet, a python module for physics simulation for games, robotics and machine learning

    Erwin Coumans and Yunfei Bai. Pybullet, a python module for physics simulation for games, robotics and machine learning. http://pybullet.org, 2016–2019. 10

    work page 2016

Show all 58 references
  1. [9]

    Ultraray: Full-path ray tracing for enhancing realism in ultrasound simulation

    Felix Duelmer, Mohammad Farid Azampour, and Nassir Navab. Ultraray: Full-path ray tracing for enhancing realism in ultrasound simulation. arXiv preprint arXiv:2501.05828, 2025

    2025 arXiv

  2. [10]

    Sofa: A multi-model framework for interactive physical simulation

    François Faure, Christian Duriez, Hervé Delingette, Jérémie Allard, Benjamin Gilles, Stéphanie Marchesseau, Hugo Talbot, Hadrien Courtecuisse, Guillaume Bousquet, Igor Peterlik, et al. Sofa: A multi-model framework for interactive physical simulation. Soft tissue biomechanical...

    2012

  3. [11]

    Generative adversarial nets

    Ian J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014

    2014

  4. [12]

    Ultrasound-guided robotic navigation with deep reinforcement learning

    Hannes Hase, Mohammad Farid Azampour, Maria Tirindelli, Magdalini Paschali, Walter Simson, Emad Fatemizadeh, and Nassir Navab. Ultrasound-guided robotic navigation with deep reinforcement learning. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IR...

    2020

  5. [13]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020

    2020

  6. [14]

    Learning agile and dynamic motor skills for legged robots

    Jemin Hwangbo, Joonho Lee, Alexey Dosovitskiy, Dario Bellicoso, Vassilios Tsounis, Vladlen Koltun, and Marco Hutter. Learning agile and dynamic motor skills for legged robots. Science Robotics, 4(26):eaau5872, 2019

    2019

  7. [15]

    Image-to-image translation with conditional adversarial networks

    Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1125–1134, 2017

    2017

  8. [16]

    Mechanical and biological effects of ultrasound: a review of present knowledge

    Zahra Izadifar, Paul Babyn, and Dean Chapman. Mechanical and biological effects of ultrasound: a review of present knowledge. Ultrasound in medicine & biology, 43(6):1085–1104, 2017

    2017

  9. [17]

    Robotic ultrasound imaging: State-of-the-art and future perspectives

    Zhongliang Jiang, Septimiu E Salcudean, and Nassir Navab. Robotic ultrasound imaging: State-of-the-art and future perspectives. Medical image analysis, 89:102878, 2023

    2023

  10. [18]

    Scalable deep reinforcement learning for vision-based robotic manipulation

    Dmitry Kalashnikov, Alex Irpan, Peter Pastor, Julian Ibarz, Alexander Herzog, Eric Jang, Deirdre Quillen, Ethan Holly, Mrinal Kalakrishnan, Vincent Vanhoucke, et al. Scalable deep reinforcement learning for vision-based robotic manipulation. In Conference on robot learning, pa...

    2018

  11. [19]

    Generation of musculoskeletal ultrasound images with diffusion models

    Sofoklis Katakis, Nikolaos Barotsis, Alexandros Kakotaritis, Panagiotis Tsiganos, George Economou, Elias Panagiotopoulos, and George Panayiotakis. Generation of musculoskeletal ultrasound images with diffusion models. BioMedInformatics, 3(2):405–421, 2023

    2023

  12. [20]

    An open-source research kit for the da vinci ® surgical system

    Peter Kazanzides, Zihan Chen, Anton Deguet, Gregory S Fischer, Russell H Taylor, and Simon P DiMaio. An open-source research kit for the da vinci ® surgical system. In 2014 IEEE international conference on robotics and automation (ICRA), pages 6434–6439. IEEE, 2014

    2014

  13. [21]

    Visualization and gpu-accelerated simulation of medical ultrasound from ct images

    Oliver Kutter, Ramtin Shams, and Nassir Navab. Visualization and gpu-accelerated simulation of medical ultrasound from ct images. Computer methods and programs in biomedicine , 94(3):250–266, 2009

    2009

  14. [22]

    Realistic ultrasound synthesis based on diagnostic ct to facilitate ultrasound-guided robotic spine surgery

    Ang Li, Jiayi Han, Yongjian Zhao, Keyu Li, and Li Liu. Realistic ultrasound synthesis based on diagnostic ct to facilitate ultrasound-guided robotic spine surgery. IEEE Transactions on Medical Robotics and Bionics, 5(4):879–889, 2023

    2023

  15. [23]

    Autonomous navigation of an ultrasound probe towards standard scan planes with deep re- inforcement learning

    Keyu Li, Jian Wang, Yangxin Xu, Hao Qin, Dongsheng Liu, Li Liu, and Max Q-H Meng. Autonomous navigation of an ultrasound probe towards standard scan planes with deep re- inforcement learning. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 8302–8...

    2021

  16. [24]

    Robot-assisted ultrasound reconstruction for spine surgery: from bench-top to pre-clinical study

    Ruixuan Li, Ayoob Davoodi, Yuyu Cai, Kenan Niu, Gianni Borghesan, Nicola Cavalcanti, Aidana Massalimova, Fabio Carrillo, Christoph J Laux, Mazda Farshad, et al. Robot-assisted ultrasound reconstruction for spine surgery: from bench-top to pre-clinical study. International jour...

    2023

  17. [25]

    Ultrasound- based robot-assisted drilling for minimally invasive pedicle screw placement.IEEE Transactions on Medical Robotics and Bionics, 6(3):818–828, 2024

    Ruixuan Li, Ayoob Davoodi, Maikel Timmermans, Kaat Van Assche, Orçun Taylan, Lennart Scheys, Matthias Tummers, Gianni Borghesan, and Emmanuel Vander Poorten. Ultrasound- based robot-assisted drilling for minimally invasive pedicle screw placement.IEEE Transactions on Medical R...

    2024

  18. [26]

    Sketch guided and progressive growing gan for realistic and editable ultrasound image synthesis

    Jiamin Liang, Xin Yang, Yuhao Huang, Haoming Li, Shuangchi He, Xindi Hu, Zejian Chen, Wufeng Xue, Jun Cheng, and Dong Ni. Sketch guided and progressive growing gan for realistic and editable ultrasound image synthesis. Medical image analysis, 79:102461, 2022

    2022

  19. [27]

    Human-in-the-loop embodied intelligence with interactive simulation environment for surgical robot learning.IEEE Robotics and Automation Letters, 8(8):4441–4448, 2023

    Yonghao Long, Wang Wei, Tao Huang, Yuehao Wang, and Qi Dou. Human-in-the-loop embodied intelligence with interactive simulation environment for surgical robot learning.IEEE Robotics and Automation Letters, 8(8):4441–4448, 2023

    2023

  20. [28]

    Isaac gym: High performance gpu-based physics simulation for robot learning

    Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, Nikita Rudin, Arthur Allshire, Ankur Handa, et al. Isaac gym: High performance gpu-based physics simulation for robot learning. arXiv preprint arXiv:2108.10470, 2021

    2021 arXiv

  21. [29]

    Realistic ultrasound simulation of complex surface models using interactive monte-carlo path tracing

    Oliver Mattausch, Maxim Makhinya, and Orcun Goksel. Realistic ultrasound simulation of complex surface models using interactive monte-carlo path tracing. In Computer Graphics Forum, volume 37, pages 202–213. Wiley Online Library, 2018

    2018

  22. [30]

    Learning robust perceptive locomotion for quadrupedal robots in the wild

    Takahiro Miki, Joonho Lee, Jemin Hwangbo, Lorenz Wellhausen, Vladlen Koltun, and Marco Hutter. Learning robust perceptive locomotion for quadrupedal robots in the wild. Science robotics, 7(62):eabk2822, 2022

    2022

  23. [31]

    Orbit: A unified simulation framework for interactive robot learning environments

    Mayank Mittal, Calvin Yu, Qinxi Yu, Jingzhou Liu, Nikita Rudin, David Hoeller, Jia Lin Yuan, Ritvik Singh, Yunrong Guo, Hammad Mazhar, Ajay Mandlekar, Buck Babich, Gavriel State, Marco Hutter, and Animesh Garg. Orbit: A unified simulation framework for interactive robot learni...

    2023

  24. [32]

    Asynchronous methods for deep reinforce- ment learning

    V olodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. Asynchronous methods for deep reinforce- ment learning. In International conference on machine learning, pages 1928–1937. PmLR, 2016

    1928

  25. [33]

    Sufia-bc: Generating high qual- ity demonstration data for visuomotor policy learning in surgical subtasks

    Masoud Moghani, Nigel Nelson, Mohamed Ghanem, Andres Diaz-Pinto, Kush Hari, Mahdi Azizian, Ken Goldberg, Sean Huver, and Animesh Garg. Sufia-bc: Generating high qual- ity demonstration data for visuomotor policy learning in surgical subtasks. arXiv preprint arXiv:2504.14857, 2025

    2025 arXiv

  26. [34]

    Autonomic robotic ultrasound imaging system based on reinforcement learning

    Guochen Ning, Xinran Zhang, and Hongen Liao. Autonomic robotic ultrasound imaging system based on reinforcement learning. IEEE transactions on biomedical engineering, 68(9):2787– 2797, 2021

    2021

  27. [35]

    Fast and automatic bone segmentation and registration of 3d ultrasound to ct for the full pelvic anatomy: a comparative study

    Prashant Pandey, Pierre Guy, Antony J Hodgson, and Rafeef Abugharbieh. Fast and automatic bone segmentation and registration of 3d ultrasound to ct for the full pelvic anatomy: a comparative study. International journal of computer assisted radiology and surgery, 13:1515– 1524, 2018

    2018

  28. [36]

    Ultrasound- guided robotic surgical procedures: a systematic review

    Matteo Pavone, Barbara Seeliger, Elena Teodorico, Marta Goglia, Cristina Taliento, Nicolò Biz- zarri, Lise Lecointre, Cherif Akladios, Antonello Forgione, Giovanni Scambia, et al. Ultrasound- guided robotic surgical procedures: a systematic review. Surgical endoscopy, 38(5):23...

    2024

  29. [37]

    Submodular rein- forcement learning

    Manish Prajapat, Mojmir Mutny, Melanie Zeilinger, and Andreas Krause. Submodular rein- forcement learning. In The Twelfth International Conference on Learning Representations , 2024. 12

    2024

  30. [38]

    Patient-specific 3d ultrasound simulation based on convolutional ray-tracing and appearance optimization

    Mehrdad Salehi, Seyed-Ahmad Ahmadi, Raphael Prevost, Nassir Navab, and Wolfgang Wein. Patient-specific 3d ultrasound simulation based on convolutional ray-tracing and appearance optimization. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th Inter...

    2015

  31. [39]

    Lapgym-an open source framework for reinforce- ment learning in robot-assisted laparoscopic surgery

    Paul Maria Scheikl, Bal ˘A ˛ Azs Gyenes, Rayan Younis, Christoph Haas, Gerhard Neumann, Martin Wagner, and Franziska Mathis-Ullrich. Lapgym-an open source framework for reinforce- ment learning in robot-assisted laparoscopic surgery. Journal of Machine Learning Research, 24(36...

    2023

  32. [40]

    Surgical gym: A high-performance gpu-based platform for reinforcement learning with surgical robots

    Samuel Schmidgall, Axel Krieger, and Jason Eshraghian. Surgical gym: A high-performance gpu-based platform for reinforcement learning with surgical robots. arXiv preprint arXiv:2310.04676, 2023

    2023 arXiv

  33. [41]

    Proximal policy optimization algorithms, 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms, 2017

    2017

  34. [42]

    skrl: Modular and flexible library for reinforcement learning

    Antonio Serrano-Muñoz, Dimitrios Chrysostomou, Simon Bøgh, and Nestor Arana- Arexolaleiba. skrl: Modular and flexible library for reinforcement learning. Journal of Machine Learning Research, 24(254):1–9, 2023

    2023

  35. [43]

    Real-time simulation of medical ultrasound from ct images

    Ramtin Shams, Richard Hartley, and Nassir Navab. Real-time simulation of medical ultrasound from ct images. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2008: 11th International Conference, New York, NY, USA, September 6-10, 2008, Proceedings, Part II 1...

    2008

  36. [44]

    Learning structured output representation using deep conditional generative models

    Kihyuk Sohn, Honglak Lee, and Xinchen Yan. Learning structured output representation using deep conditional generative models. Advances in neural information processing systems, 28, 2015

    2015

  37. [45]

    S-cyclegan: Semantic segmentation enhanced ct-ultrasound image-to-image translation for robotic ultrasonography

    Yuhan Song and Nak Young Chong. S-cyclegan: Semantic segmentation enhanced ct-ultrasound image-to-image translation for robotic ultrasonography. In 2024 IEEE International Conference on Cyborg and Bionic Systems (CBS), pages 115–120. IEEE, 2024

    2024

  38. [46]

    Echo from noise: synthetic ultrasound image generation using diffusion models for real image segmentation

    David Stojanovski, Uxio Hermida, Pablo Lamata, Arian Beqiri, and Alberto Gomez. Echo from noise: synthetic ultrasound image generation using diffusion models for real image segmentation. In International Workshop on Advances in Simplifying Medical Ultrasound , pages 34–43. Spr...

    2023

  39. [47]

    Sim-to-real: Learning agile locomotion for quadruped robots

    Jie Tan, Tingnan Zhang, Erwin Coumans, Atil Iscen, Yunfei Bai, Danijar Hafner, Steven Bohez, and Vincent Vanhoucke. Sim-to-real: Learning agile locomotion for quadruped robots. arXiv preprint arXiv:1804.10332, 2018

    2018 arXiv

  40. [48]

    Mujoco: A physics engine for model-based control

    Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ international conference on intelligent robots and systems, pages 5026–5033. IEEE, 2012

    2012

  41. [49]

    Robotic path re-planning for us reconstruction of the spine

    Kaat Van Assche, Ruixuan Li, Ayoob Davoodi, Matthias Tummers, Mouloud Ourak, Gianni Borghesan, Nicola Cavalcanti, Philipp Fünstahl, and Emmanuel Vander Poorten. Robotic path re-planning for us reconstruction of the spine. IEEE Transactions on Medical Robotics and Bionics, 2025

    2025

  42. [50]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017

    2017

  43. [51]

    Follow the curve: Robotic ultrasound navigation with learning-based localization of spinous processes for scoliosis assessment

    Maria Victorova, Michael Ka-Shing Lee, David Navarro-Alarcon, and Yongping Zheng. Follow the curve: Robotic ultrasound navigation with learning-based localization of spinous processes for scoliosis assessment. IEEE access, 10:40216–40229, 2022. 13

    2022

  44. [52]

    Totalseg- mentator: robust segmentation of 104 anatomic structures in ct images

    Jakob Wasserthal, Hanns-Christian Breit, Manfred T Meyer, Maurice Pradella, Daniel Hinck, Alexander W Sauter, Tobias Heye, Daniel T Boll, Joshy Cyriac, Shan Yang, et al. Totalseg- mentator: robust segmentation of 104 anatomic structures in ct images. Radiology: Artificial Inte...

    2023

  45. [53]

    Ultrabones100k: A reliable automated labeling method and large-scale dataset for ultrasound-based bone surface extraction

    Luohong Wu, Nicola A Cavalcanti, Matthias Seibold, Giuseppe Loggia, Lisa Reissner, Jonas Hein, Silvan Beeler, Arnd Viehöfer, Stephan Wirth, Lilian Calvet, et al. Ultrabones100k: A reliable automated labeling method and large-scale dataset for ultrasound-based bone surface extr...

    2025 arXiv

  46. [54]

    Surrol: An open-source reinforcement learning centered and dvrk compatible platform for surgical robot learning

    Jiaqi Xu, Bin Li, Bo Lu, Yun-Hui Liu, Qi Dou, and Pheng-Ann Heng. Surrol: An open-source reinforcement learning centered and dvrk compatible platform for surgical robot learning. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 1821–18...

    2021

  47. [55]

    Orbit-surgical: An open-simulation framework for learning surgical augmented dexterity

    Qinxi Yu, Masoud Moghani, Karthik Dharmarajan, Vincent Schorp, William Chung-Ho Panitch, Jingzhou Liu, Kush Hari, Huang Huang, Mayank Mittal, Ken Goldberg, et al. Orbit-surgical: An open-simulation framework for learning surgical augmented dexterity. In 2024 IEEE International...

    2024

  48. [56]

    Learning fine-grained bimanual manipulation with low-cost hardware

    Tony Z Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn. Learning fine-grained bimanual manipulation with low-cost hardware. arXiv preprint arXiv:2304.13705, 2023

    2023 arXiv

  49. [57]

    State-wise constrained policy optimization, 2024

    Weiye Zhao, Rui Chen, Yifan Sun, Tianhao Wei, and Changliu Liu. State-wise constrained policy optimization, 2024

    2024

  50. [58]

    Unpaired image-to-image translation using cycle-consistent adversarial networks

    Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. InProceedings of the IEEE international conference on computer vision, pages 2223–2232, 2017. 14 A Dataset access Below are the links ...

    2017

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.