Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

EndoVLA: Dual-Phase Vision-Language-Action Model for Autonomous Tracking in Endoscopy

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A dual-phase fine-tuning strategy makes a vision-language-action model succeed at autonomous endoscopic tracking.

desk verdict Competent application of SFT+RFT to continuum endoscopy with a new dataset, but the headline centering metric uses a laxer threshold than the stated objective and the SOTA claim lacks external baselines. read the letter →

arxiv 2505.15206 v1 pith:XXCGVMBK submitted 2025-05-21 cs.RO cs.AI

Chi Kit Ng , Long Bai , Guankun Wang , Yupeng Wang , Huxin Gao , Kun Yuan , Chenhan Jin , Tieyong Zeng
show 1 more author
Hongliang Ren
This is my paper · ORCID
classification cs.ROcs.AI
keywords vision-language-actionendoscopictrackingdual-phasefine-tuningreinforcementcontinuumrobotszero-shotgeneralizationGRPO
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

EndoVLA argues that a general-purpose vision-language-action (VLA) model can be turned into an autonomous endoscopic tracker by fine-tuning it in two stages: first supervised fine-tuning (SFT) on a small dataset of image-action pairs, then reinforcement fine-tuning (RFT) with verifiable rewards. The paper reports that the combined pipeline succeeds in moving the endoscope toward polyps and abnormal regions in 100% of test trials, completes the circumferential cutting loop in 10% of trials, and transfers zero-shot to non-endoscopic scenes such as tracking characters and fruits. This matters because current endoscopic tracking systems require manually tuned detection and motion modules that do not generalize, while a single model that reads a surgeon's natural-language prompt could reduce cognitive burden without recalibration.

What carries the argument

The load-bearing mechanism is the dual-phase fine-tuning (DFT) protocol. Phase 1 is supervised fine-tuning with LoRA on the EndoVLA-Motion dataset, aligning the base model to endoscopic images and prompts. Phase 2 applies reinforcement fine-tuning with group-relative policy optimization (GRPO), using three verifiable rewards—IoU, motion-angle match, and format—so the model learns to output a tight bounding box and the correct one of five discrete actions (four diagonal motor increments or 'still') without a learned reward model. The discrete action space and the 'still' threshold ($\epsilon = 18$ pixels) define the control objective: move the target center into the focus region and then stop.

What would settle it

Run EndoVLA on a live or ex-vivo tissue model with peristalsis, specular reflections, and partial occlusion and measure the centering success rate; if it drops well below the phantom 100% moving-toward-target rate, the phantom-validated claim does not transfer. A simpler mechanical falsifier: hold the endoscope at a fixed bending angle and measure the target's pixel trajectory under repeated identical motor increments; if the trajectory is not linear in the motor increment (e.g., it curves or exhibits backlash), the control objective in Eq. (2) is invalid.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that the dual-phase fine-tuning strategy (SFT followed by RFT) resolves the data scarcity and domain-shift problems that block VLA models in endoscopy. The model is trained on EndoVLA-Motion, a curated 6k-pair vision-language-kinematic dataset, and refined with GRPO using three verifiable rewards: IoU of the predicted bounding box, exact match of the discrete motion action, and output format. This yields a model that not only localizes the target and emits the correct bending command but, uniquely among the three training variants, produces any successful full-circle completions in the circular cutting task and a 50% success rate on the complete CORL sequence in zero-shot general scenes. The authors interpret this as the RL phase sharpening localization and action selection beyond what SFT alone, or RFT alone, can achieve.

Load-bearing premise

The central assumption is that tracking quality is fully measured by centering a target's bounding box within 18 pixels of the image center using four fixed diagonal motor increments and a linear relation between motor rotation and image motion, in phantom environments that do not deform, occlude, or bleed.

Editorial extensions

If this is right

  • If the result holds, a single VLA model could replace modular detection–planning–control pipelines for endoscopic target tracking, removing manual recalibration across patients and anatomical sites.
  • The success of the dual-phase recipe suggests that pairing SFT with verifiable-reward RFT is a broadly effective way to adapt general MLLMs to safety-critical robotic tasks with small datasets.
  • The zero-shot generalization to non-endoscopic scenes implies that the model learns task semantics ('find the target, move it to center') rather than memorizing phantom-specific appearances, which encourages transfer to new instruments or modalities.
  • The 10% completion rate on the circular cutting loop shows a path toward autonomous ESD-type procedures, but also marks the loop as the current bottleneck.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the auto-labeling pipeline was only about 65% accurate before manual curation, the reported performance likely depends on that curation; a fully automatic data pipeline might not reproduce the same gains.
  • The linear mapping between motor rotation and image motion, plus the discrete four-direction action set, means the reported success rates are upper bounds for scenarios with nonlinear kinematics or actuation delay, which the paper's own limitations section acknowledges.
  • The 2 Hz inference speed is far below the 30 Hz video rate, so the model would struggle with fast-moving targets unless temporal context or faster hardware is added; the authors list this as future work.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces EndoVLA, a vision-language-action (VLA) model for autonomous tracking with continuum robotic endoscopes. The authors propose a dual-phase fine-tuning strategy consisting of supervised fine-tuning (SFT) followed by reinforcement fine-tuning (RFT) with verifiable rewards, and construct a new dataset, EndoVLA-Motion, of roughly 6,000 image-action pairs from phantom-based endoscopic videos. The model is evaluated on three tasks (polyp tracking, abnormal region tracking, and circular cutting marker following) on a real robotic endoscope, as well as zero-shot generalization to non-endoscopic scenes. The central claim is that the SFT+RFT model substantially outperforms single-phase SFT and RFT across all tasks and enables zero-shot generalization.

Significance. If the results hold, the paper contributes a practical recipe for adapting general-purpose VLA models to robotic endoscopy via a two-stage fine-tuning procedure, along with a new dataset that could support further research. The real-robot evaluation and the explicit listing of limitations are commendable. However, the evidence is entirely internal (no external baselines), the evaluation metric is inconsistent with the stated control objective, and the trial counts are small with no error bars. These issues currently limit the strength of the headline claims, including 'state-of-the-art' performance and 'precise centering'.

major comments (4)
  1. [§3.2, §4.2, Table 3] The success metric 'moving within FR (r)' used in Table 3 is based on the focus-region radius r = 20√2 ≈ 28.3 pixels defined in the labeling pipeline (Supplement A.2), whereas Eq. (2) in §4.2 defines the control objective with ε = 18 pixels. The reported centering success rates (63% for PP and 57% for AR) are therefore measured against a substantially looser threshold than the stated control target, and the model is trained (via the MA reward in RFT) to stop when the target is within the larger radius. Re-scoring at ε = 18 would likely lower these numbers, so the abstract's claim of 'precisely centering' is not supported by the current metric. Please rescore the results with a consistent threshold or explicitly redefine the evaluation metric to match the control objective.
  2. [§6, Contribution (3)] The conclusion claims 'state-of-the-art performance' across the three endoscopic tracking tasks, but all comparisons in Tables 2–4 are internal ablations (SFT vs. RFT vs. SFT+RFT). There are no external baselines, such as a classical detect-and-servo pipeline, a non-RL VLA model, or a simple rule-based controller. Without such comparisons, the 'state-of-the-art' claim is unsupported. Please add at least one external baseline or temper the claim to 'consistent improvements over our single-phase variants'.
  3. [§5.4, §5.5] The trial counts are small: 30 trials per condition for PP and AR, and only 10 trials for CC (Table 3b) and for the general-scene tasks (Table 4). No variance, confidence intervals, or statistical significance tests are reported. In particular, the CC completion rate of 10% corresponds to a single successful trial, so the statement that 'only the DFT model achieved any success' rests on very limited evidence. Please report the number of trials per condition and include error bars or a statistical test (e.g., a binomial confidence interval).
  4. [§4.2, §7] The evaluation is performed under the same linear-discrete actuation model (Eq. 1–2) that the authors themselves identify in Section 7 as ignoring nonlinear kinematics, actuation delays, and varying compliance. Because both the control objective and the success metric are defined within this simplified model, the reported tracking success does not yet demonstrate robustness to the conditions listed in the limitations. It would strengthen the paper to include a sensitivity analysis (even in simulation) showing how performance degrades as the linearity or discretization assumptions are relaxed.
minor comments (6)
  1. [§3.2 vs. Supplement A.2] The threshold for the focus region is described as '28 pixels' in Section 3.2 and as 'r = 20√2 pixels' (≈28.3) in Supplement A.2; please reconcile these values.
  2. [§4.3, Eq. (3)] The value function V_ψ is introduced in the GRPO advantage estimate but is never defined or described. GRPO typically uses a group-average baseline rather than a learned value function; please clarify the role of V_ψ or remove it.
  3. [Table 5] The abbreviations 'TS' and 'ES' are used in Table 5 but not defined in the caption; please define them (training set and evaluation set).
  4. [§1] The sentence 'Endoscopic procedures is the gold standard' should be 'Endoscopic procedures are the gold standard'.
  5. [Figure 2] The figure labels 'Vision-SFT' and 'Vision-RFT' are not used consistently in the text, which refers to a single dual-phase fine-tuning (DFT) strategy; please align the terminology.
  6. [§5.2, Table 1] The negative improvement (−31.3%) reported for CC/Ia indicates that localization instructions hurt performance in that condition; the general conclusion that 'localization instructions improve performance' should be qualified to note this exception.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: EndoVLA's dual-phase fine-tuning is evaluated on held-out real-robot trials and zero-shot scenes, with no prediction that reduces to a fitted input by construction.

full rationale

I find no circular step in the paper's derivation chain. The central claim is that SFT+RFT improves tracking over single-phase fine-tuning; this is supported by held-out real-robot trials (Table 3) and zero-shot generalization to non-endoscopic tasks (Table 4), not by reusing training labels as the success metric. The RFT rewards (IoU, MA, Format) are computed against external ground-truth bounding boxes and discrete action labels derived from bounding-box centers in the dataset, which is standard supervised reward design rather than a self-consistent definition of success. The action labels are a deterministic function of the ground-truth box center relative to the focus region, but the model must still learn to predict that function from raw images, and the real-robot evaluation measures physical target centering, not the label rule itself. References [7], [17], and [31] include authors of this paper, but they appear only in related-work context and none carries a load-bearing assumption or uniqueness claim. I also note the flagged evaluation inconsistency: Section 4.2 sets epsilon=18 px in Eq. 2, while Supplement A.2 assigns 'still' labels with r=20*sqrt(2) approx 28.3 px, so the 'moving within FR (r)' metric in Table 3 uses the looser labeling radius; this is a correctness/threshold concern, not circularity. The limitations section explicitly acknowledges the simplified linear-discrete actuation model and phantom-only data, which further confirms the results are empirical rather than definitionally forced.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central numerical claims depend on hand-set thresholds (epsilon, FR radius), the discrete action step size, and a linear actuation model. No new physical entities are introduced. The dataset filtering step removes hard examples, which should be kept in mind when interpreting success rates.

free parameters (5)
  • epsilon (stop threshold) = 18 pixels
    Hand-set in Section 4.2 Eq. 2 to define 'mission success'; changes the reported success rates if altered.
  • Focus Region radius r = 20*sqrt(2) pixels (supplement) or 28 pixels (Section 3.2)
    Hand-set threshold for labeling the 'still' action and for evaluation; the two values differ in the text.
  • discrete action increment delta_theta = Not specified numerically.
    Fixed motor increment per action in Eq. 1; the step size determines whether the target can be centered within epsilon.
  • linear mapping constant k = Not specified numerically.
    Assumed linear relation alpha_i = k theta_i between bending angle and motor rotation (Section 3.1 and Section 4.2).
  • GRPO group size = 4
    Hyperparameter chosen for RFT; affects stability and performance.
assumptions (5)
  • domain assumption Qwen2-VL-7B provides a sufficiently strong vision-language base for endoscopic images.
    The paper relies on the pretrained model's visual grounding without ablation against other backbones.
  • ad hoc to paper The mapping from motor rotations to image coordinates is linear (p_t = M(theta), alpha_i = k theta_i).
    Stated in Sections 3.1 and 4.2; the authors acknowledge it ignores nonlinear kinematics, actuation delays, and compliance.
  • ad hoc to paper A discrete set of five actions (four diagonal increments plus stop) is sufficient for tracking.
    Section 4.2 defines the action set; the limitations section notes coarse discretization limits precision.
  • domain assumption Phantom environments with clean visual conditions are representative enough to evaluate tracking ability.
    All data and tests use two stomach phantoms; the authors state in Limitations that in vivo conditions (bleeding, smoke, deformation) are untested.
  • domain assumption YOLOv5 auto-labels plus manual curation yield accurate ground truth.
    Supplement A.2-A.4; labeling accuracy was about 65% before curation, and erroneous samples were removed, implying selection toward easy examples.

how reviews work

0 comments
Cite this review

Pith. "Pith review of EndoVLA: Dual-Phase Vision-Language-Action Model for Autonomous Tracking in Endoscopy." pith.science (2026). https://pith.science/paper/XXCGVMBK

@misc{pith2026250515206,
  author       = {Pith},
  title        = {Pith review of: EndoVLA: Dual-Phase Vision-Language-Action Model for Autonomous Tracking in Endoscopy},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XXCGVMBK}},
  note         = {Machine review of arXiv:2505.15206}
}
read the original abstract

In endoscopic procedures, autonomous tracking of abnormal regions and following circumferential cutting markers can significantly reduce the cognitive burden on endoscopists. However, conventional model-based pipelines are fragile for each component (e.g., detection, motion planning) requires manual tuning and struggles to incorporate high-level endoscopic intent, leading to poor generalization across diverse scenes. Vision-Language-Action (VLA) models, which integrate visual perception, language grounding, and motion planning within an end-to-end framework, offer a promising alternative by semantically adapting to surgeon prompts without manual recalibration. Despite their potential, applying VLA models to robotic endoscopy presents unique challenges due to the complex and dynamic anatomical environments of the gastrointestinal (GI) tract. To address this, we introduce EndoVLA, designed specifically for continuum robots in GI interventions. Given endoscopic images and surgeon-issued tracking prompts, EndoVLA performs three core tasks: (1) polyp tracking, (2) delineation and following of abnormal mucosal regions, and (3) adherence to circular markers during circumferential cutting. To tackle data scarcity and domain shifts, we propose a dual-phase strategy comprising supervised fine-tuning on our EndoVLA-Motion dataset and reinforcement fine-tuning with task-aware rewards. Our approach significantly improves tracking performance in endoscopy and enables zero-shot generalization in diverse scenes and complex sequential tasks.

Figures

Figures reproduced from arXiv: 2505.15206 by the authors.

Figure 1
Figure 1. EndoVLA enables robust autonomous tracking in endoscopic procedures, demonstrating [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of the setup of robotic endoscope and the DFT architecture of EndoVLA [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Example of successful rollouts on the real-world endoscopic tasks (PP, AR and CC) on a [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Example of successful rollouts on a real-world robotic setup of tasks: CORL character [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: (a) The data is collected by two phantoms. (b) Robotic setup. [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: The examples for wrongly annotated the next tracking point in anti-clockwise direction. [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Qualitative results of bounding box prediction (green: GT, red: prediction) and motion [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. CapsDT: Diffusion-Transformer for Capsule Robot Manipulation

    cs.RO 2025-06 reject novelty 4.0 of 10

    A diffusion transformer for capsule endoscopy robots, fine-tuned on a small self-collected dataset, reports 26.25% average success across four stomach-simulator tasks.

Reference graph

Works this paper leans on

40 extracted references · 22 canonical work pages · cited by 1 Pith paper

  1. [1]

    H. Gao, X. Yang, X. Xiao, X. Zhu, T. Zhang, C. Hou, H. Liu, M. Q.-H. Meng, L. Sun, X. Zuo, et al. Transendoscopic flexible parallel continuum robotic mechanism for bimanual endoscopic submucosal dissection.The International Journal of Robotics Research, 43(3):281–304, 2024

    work page 2024

  2. [2]

    J. T. Maple, B. K. A. Dayyeh, S. S. Chauhan, J. H. Hwang, S. Komanduri, M. Manfredi, V . Konda, F. M. Murad, U. D. Siddiqui, and S. Banerjee. Endoscopic submucosal dissection. Gastrointestinal endoscopy, 81(6):1311–1325, 2015

    work page 2015

  3. [3]

    D.-H. Lee, B. Cheon, J. Kim, and D.-S. Kwon. easyendo robotic endoscopy system: Devel- opment and usability test in a randomized controlled trial with novices and physicians.The International Journal of Medical Robotics and Computer Assisted Surgery, 17(1):1–14, 2021

    work page 2021

  4. [4]

    Zhang, E

    X. Zhang, E. K. Ly, S. Nithyanand, R. J. Modayil, D. O. Khodorskiy, S. Neppala, S. Bhumi, M. DeMaria, J. L. Widmer, D. M. Friedel, et al. Learning curve for endoscopic submucosal dissection with an untutored, prevalence-based approach in the united states.Clinical Gas- troenterology and Hepatology, 18(3):580–588, 2020

    work page 2020

  5. [5]

    H. Lin, B. Li, X. Chu, Q. Dou, Y . Liu, and K. W. S. Au. End-to-end learning of deep visuomotor policy for needle picking. In2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 8487–8494. IEEE, 2023

    work page 2023

  6. [6]

    Hwang, D

    M. Hwang, D. Seita, B. Thananjeyan, J. Ichnowski, S. Paradis, D. Fer, T. Low, and K. Gold- berg. Applying depth-sensing to automated surgical manipulation with a da vinci robot. In 2020 international symposium on medical robotics (ISMR), pages 22–29. IEEE, 2020

    work page 2020

  7. [7]

    Islam, Y

    M. Islam, Y . Li, and H. Ren. Learning where to look while tracking instruments in robot- assisted surgery. InInternational Conference on Medical Image Computing and Computer- Assisted Intervention, pages 412–420. Springer, 2019

    work page 2019

  8. [8]

    Leonard, A

    S. Leonard, A. Sinha, A. Reiter, M. Ishii, G. L. Gallia, R. H. Taylor, and G. D. Hager. Eval- uation and stability analysis of video-based navigation system for functional endoscopic sinus surgery on in vivo clinical data.IEEE transactions on medical imaging, 37(10):2185–2195, 2018

    work page 2018

Show all 40 references
  1. [9]

    Z. Fu, Z. Jin, C. Zhang, Z. He, Z. Zha, C. Hu, T. Gan, Q. Yan, P. Wang, and X. Ye. The future of endoscopic navigation: a review of advanced endoscopic vision technology.IEEE Access, 9:41144–41167, 2021

    2021

  2. [10]

    Y . Long, W. Wei, T. Huang, Y . Wang, and Q. Dou. Human-in-the-loop embodied intelligence with interactive simulation environment for surgical robot learning.IEEE Robotics and Au- tomation Letters, 8(8):4441–4448, 2023

    2023

  3. [11]

    J. Xu, B. Li, B. Lu, Y .-H. Liu, Q. Dou, and P.-A. Heng. Surrol: An open-source reinforcement learning centered and dvrk compatible platform for surgical robot learning. In2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1821–1828. IEEE, 2021

    2021

  4. [12]

    Huang, K

    T. Huang, K. Chen, B. Li, Y .-H. Liu, and Q. Dou. Guided reinforcement learning with efficient exploration for task automation of surgical robot. In2023 IEEE International Conference on Robotics and Automation (ICRA), pages 4640–4647. IEEE, 2023. 9

    2023

  5. [13]

    S. Li, J. Wang, R. Dai, W. Ma, W. Y . Ng, Y . Hu, and Z. Li. Robonurse-vla: Robotic scrub nurse system based on vision-language-action model.arXiv preprint arXiv:2409.19590, 2024

    2024 arXiv

  6. [14]

    Moghani, L

    M. Moghani, L. Doorenbos, W. C.-H. Panitch, S. Huver, M. Azizian, K. Goldberg, and A. Garg. Sufia: language-guided augmented dexterity for robotic surgical assistants. In2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 6969–

  7. [15]

    Achiam, S

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    2023 arXiv

  8. [16]

    H. Liu, C. Li, Q. Wu, and Y . J. Lee. Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    2023

  9. [17]

    G. Wang, L. Bai, J. Wang, K. Yuan, Z. Li, T. Jiang, X. He, J. Wu, Z. Chen, Z. Lei, et al. Endochat: Grounded multimodal large language model for endoscopic surgery.arXiv preprint arXiv:2501.11347, 2025

    2025 arXiv

  10. [18]

    Zitkovich, T

    B. Zitkovich, T. Yu, S. Xu, P. Xu, T. Xiao, F. Xia, J. Wu, P. Wohlhart, S. Welker, A. Wahid, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning, pages 2165–2183. PMLR, 2023

    2023

  11. [19]

    J. Liu, M. Liu, Z. Wang, P. An, X. Li, K. Zhou, S. Yang, R. Zhang, Y . Guo, and S. Zhang. Robomamba: Efficient vision-language-action model for robotic reasoning and manipulation. Advances in Neural Information Processing Systems, 37:40085–40110, 2024

    2024

  12. [20]

    H. Arai, K. Miwa, K. Sasaki, K. Watanabe, Y . Yamaguchi, S. Aoki, and I. Yamamoto. Covla: Comprehensive vision-language-action dataset for autonomous driving. In2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 1933–1943. IEEE, 2025

    1933

  13. [21]

    M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, G. Lam, P. Sanketi, et al. Openvla: An open-source vision-language-action model.arXiv preprint arXiv:2406.09246, 2024

    2024 arXiv

  14. [22]

    R. Rudiman. Minimally invasive gastrointestinal surgery: from past to the future.Annals of Medicine and Surgery, 71:102922, 2021

    2021

  15. [23]

    K. Fan, Z. Chen, G. Ferrigno, and E. De Momi. Learn from safe experience: Safe reinforce- ment learning for task automation of surgical robot.IEEE Transactions on Artificial Intelli- gence, 5(7):3374–3383, 2024

    2024

  16. [24]

    Brohan, N

    A. Brohan, N. Brown, J. Carbajal, Y . Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Haus- man, A. Herzog, J. Hsu, et al. Rt-1: Robotics transformer for real-world control at scale.arXiv preprint arXiv:2212.06817, 2022

    2022 arXiv

  17. [25]

    O’Neill, A

    A. O’Neill, A. Rehman, A. Maddukuri, A. Gupta, A. Padalkar, A. Lee, A. Pooley, A. Gupta, A. Mandlekar, A. Jain, et al. Open x-embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collaboration 0. In2024 IEEE International Conference on Robotics and Automati...

    2024

  18. [26]

    Y . Hu, Q. Xie, V . Jain, J. Francis, J. Patrikar, N. Keetha, S. Kim, Y . Xie, T. Zhang, H.-S. Fang, et al. Toward general-purpose robots via foundation models: A survey and meta-analysis. arXiv preprint arXiv:2312.08782, 2023

    2023 arXiv

  19. [27]

    Schmidgall, J

    S. Schmidgall, J. W. Kim, A. Kuntz, A. E. Ghazi, and A. Krieger. General-purpose foundation models for increased autonomy in robot-assisted surgery.Nature Machine Intelligence, pages 1–9, 2024. 10

    2024

  20. [28]

    Shridhar, L

    M. Shridhar, L. Manuelli, and D. Fox. Perceiver-actor: A multi-task transformer for robotic manipulation. InConference on Robot Learning, pages 785–799. PMLR, 2023

    2023

  21. [29]

    Brohan, N

    A. Brohan, N. Brown, J. Carbajal, Y . Chebotar, X. Chen, K. Choromanski, T. Ding, D. Driess, A. Dubey, C. Finn, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control.arXiv preprint arXiv:2307.15818, 2023

    2023 arXiv

  22. [30]

    Black, N

    K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter, et al.π 0: A vision-language-action flow model for general robot control.arXiv preprint arXiv:2410.24164, 2024

    2024 arXiv

  23. [31]

    Qian and H

    C. Qian and H. Ren. Deep reinforcement learning in surgical robotics: enhancing the automa- tion level.Handbook of Robotic Surgery, pages 89–102, 2025

    2025

  24. [32]

    J. Liu, A. Andres, Y . Jiang, X. Luo, W. Shu, and S. A. Tsaftaris. Surgical task automa- tion using actor-critic frameworks and self-supervised imitation learning.arXiv preprint arXiv:2409.02724, 2024

    2024 arXiv

  25. [33]

    J. W. Kim, T. Z. Zhao, S. Schmidgall, A. Deguet, M. Kobilarov, C. Finn, and A. Krieger. Surgical robot transformer (srt): Imitation learning for surgical tasks.arXiv preprint arXiv:2407.12998, 2024

    2024 arXiv

  26. [34]

    Redmon, S

    J. Redmon, S. Divvala, R. Girshick, and A. Farhadi. You only look once: Unified, real- time object detection. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 779–788, 2016

    2016

  27. [35]

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, W. Chen, et al. Lora: Low-rank adaptation of large language models.ICLR, 1(2):3, 2022

    2022

  28. [36]

    Q. Dong, L. Li, D. Dai, C. Zheng, J. Ma, R. Li, H. Xia, J. Xu, Z. Wu, T. Liu, et al. A survey on in-context learning.arXiv preprint arXiv:2301.00234, 2022

    2022 arXiv

  29. [37]

    Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y . Li, Y . Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    2024 arXiv

  30. [38]

    M. H. Daniel Han and U. team. Unsloth, 2023. URLhttp://github.com/unslothai/ unsloth

    2023

  31. [39]

    H. Shen, P. Liu, J. Li, C. Fang, Y . Ma, J. Liao, Q. Shen, Z. Zhang, K. Zhao, Q. Zhang, R. Xu, and T. Zhao. Vlm-r1: A stable and generalizable r1-style large vision-language model.arXiv preprint arXiv:2504.07615, 2025

    2025 arXiv

  32. [40]

    sphere polyp

    P. Wang, S. Bai, S. Tan, S. Wang, Z. Fan, J. Bai, K. Chen, X. Liu, J. Wang, W. Ge, et al. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191, 2024. 7 Limitations Dataset Size and Diversity.The EndoVLA-Motion da...

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.