arxiv: 2604.03781 · v1 · submitted 2026-04-04 · 💻 cs.RO

Recognition: no theorem link

OpenRC: An Open-Source Robotic Colonoscopy Framework for Multimodal Data Acquisition and Autonomy Research

Siddhartha Kapuria , Mohammad Rafiee Javazm , Naruhiko Ikoma , Joga Ivatury , Mohammad Ali Nasseri , Nassir Navab , Farshid Alambeigi

Authors on Pith no claims yet

Pith reviewed 2026-05-13 16:52 UTC · model grok-4.3

classification 💻 cs.RO

keywords robotic colonoscopyopen-source frameworkmultimodal datasetsurgical autonomyteleoperated episodescolonoscope retrofittingmotion consistencyvision-language-action

0 comments

The pith

OpenRC retrofits standard colonoscopes with robotics to enable synchronized recording of video, commands, actuation, and tip pose for autonomy research.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces OpenRC, an open-source modular framework that adds robotic actuation to ordinary colonoscopes without altering standard clinical use. It captures aligned streams of scope video, operator inputs, robot state, and distal tip position during teleoperated sessions. The authors used the system to gather 1,894 episodes totaling roughly 19 hours across ten task types that include normal navigation, errors, and recoveries. A sympathetic reader would care because existing colonoscopy research lacks public, time-synchronized multimodal records that link human control to instrument motion and visual feedback. If the approach holds, it supplies a common starting point for closed-loop experiments in robotic endoscopy and vision-language-action models.

Core claim

OpenRC is an open-source modular robotic colonoscopy framework that retrofits conventional scopes while preserving clinical workflow. The framework supports simultaneous recording of video, operator commands, actuation state, and distal tip pose. The platform was validated for motion consistency and cross-modal latency. Using this platform, a multimodal dataset comprising 1,894 teleoperated episodes (~19 hours) across 10 structured task variations of routine navigation, failure events, and recovery behaviors was collected.

What carries the argument

The OpenRC retrofitting hardware and sensing stack that attaches to standard colonoscopes to produce time-aligned multimodal streams of video, commands, actuation, and tip pose.

If this is right

The released dataset supplies synchronized records for training and benchmarking vision-language-action models in endoscopic navigation.
Quantified cross-modal latency measurements allow direct testing of real-time control algorithms against actual hardware delays.
Open hardware and episode logs enable other labs to reproduce experiments without access to proprietary robotic scopes.
The structured coverage of failure and recovery behaviors supports study of robust autonomy under realistic error conditions.
Motion-consistency validation provides a baseline for comparing new robotic actuation methods against the recorded performance.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The dataset format could become a de facto reference for comparing autonomy algorithms across different robotic endoscopy platforms.
Pairing the recorded episodes with physics-based simulators would allow large-scale virtual testing before physical trials.
If the retrofitting approach proves stable, similar sensor stacks could be adapted to other flexible endoscopes used in gastroenterology.
Widespread use of the open logs might accelerate development of shared evaluation protocols for regulatory review of surgical robots.

Load-bearing premise

Retrofitting conventional scopes with the robotic system preserves clinical workflow and operator behavior without introducing meaningful changes to safety, usability, or data quality.

What would settle it

A controlled comparison showing that the added robotic components measurably alter operator hand movements, extend procedure time, or degrade image quality would falsify the claim that the retrofit preserves clinical workflow.

Figures

Figures reproduced from arXiv: 2604.03781 by Farshid Alambeigi, Joga Ivatury, Mohammad Ali Nasseri, Mohammad Rafiee Javazm, Naruhiko Ikoma, Nassir Navab, Siddhartha Kapuria.

**Figure 1.** Figure 1: Overview of the proposed OpenRC framework, including hardware interfaces, actuation, data acquisition, and sensing via EM tracking. 2 Methods As shown in [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

**Figure 2.** Figure 2: OpenRC components: (a) bending module, (b) feeding module, (c) ROS 2 graph showing data streams, and (d) experimental setup for data collection. for the native lock. An additional 1:2 reduction in the outer collet improves command resolution. Components are fabricated from nylon–carbon fiber composite and ABS plastic to balance strength and manufacturability. Feeding Module. As shown in [PITH_FULL_IMAGE:… view at source ↗

**Figure 3.** Figure 3: Results for characterization and synchronization showing: (a) sinusoidal system response characterization, and histograms of estimated residual lag distributions for (b) Operator Action vs State, and (c) State vs Tip Position. 3 System Characterization As described in Sec. 2.3, OpenRC integrates operator input, joint-space motor states, EM-based distal tip tracking, and colonoscope video, which operate th… view at source ↗

**Figure 4.** Figure 4: Example episode with synchronized multimodal recordings from the robotic colonoscopy dataset. From top to bottom: colonoscope video snapshots sampled from the same time axis as data streams; operator control actions; distal tip pose; robot state (normalized to [−1, 1] per axis for ease of visualization). formed within a colon phantom. Episodes include active manipulation, pauses, and adjustments, with dura… view at source ↗

**Figure 5.** Figure 5: Episode-level characteristics of the OpenRC Dataset showing distributions of (a) episode duration, (b) trajectory length, and (c) recorded task. Detailed task descriptions will be provided in the dataset repository. calibrated using the offsets from Sec. 3, with the video feed as reference. To validate post-alignment stability, we estimated the residual temporal lag τ ∗ between modality pairs (x, y), nam… view at source ↗

read the original abstract

Colorectal cancer screening critically depends on colonoscopy, yet existing platforms offer limited support for systematically studying the coupled dynamics of operator control, instrument motion, and visual feedback. This gap restricts reproducible closed-loop research in robotic colonoscopy, medical imaging, and emerging vision-language-action (VLA) learning paradigms. To address this challenge, we present OpenRC, an open-source modular robotic colonoscopy framework that retrofits conventional scopes while preserving clinical workflow. The framework supports simultaneous recording of video, operator commands, actuation state, and distal tip pose. We experimentally validated motion consistency and quantified cross-modal latency across sensing streams. Using this platform, we collected a multimodal dataset comprising 1,894 teleoperated episodes ~19 hours across 10 structured task variations of routine navigation, failure events, and recovery behaviors. By unifying open hardware and an aligned multimodal dataset, OpenRC provides a reproducible foundation for research in multimodal robotic colonoscopy and surgical autonomy.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

OpenRC offers a useful open platform and dataset for robotic colonoscopy, but its validation of preserved clinical workflow relies only on motion and latency checks.

read the letter

OpenRC gives the field an open-source robotic retrofit for conventional colonoscopes together with a multimodal dataset of nearly 1,900 episodes. This is the core contribution: hardware that supports simultaneous capture of video, commands, actuation state, and tip pose, plus structured tasks that include failures and recoveries. The paper does a solid job on the systems side by emphasizing modularity and openness, which lowers the barrier for others to experiment with autonomy in colonoscopy. The reported motion consistency and latency numbers provide a baseline for reproducibility. The weaker part is the claim about preserving clinical workflow. Validation stays at motion and cross-modal timing; there are no numbers on insertion forces, tip control under realistic loads, or changes in operator workload. If the retrofit adds friction or alters feedback, the collected data may not match real clinical episodes, which would reduce its value for training models meant to work in practice. This paper is for people in medical robotics and AI-driven surgical systems who need shared testbeds and data. It has enough new infrastructure to merit peer review, even if the validation section needs expansion. I'd send it to referees for a closer look at the hardware and dataset quality.

Referee Report

2 major / 1 minor

Summary. The manuscript presents OpenRC, an open-source modular robotic colonoscopy framework that retrofits conventional scopes while preserving clinical workflow. It enables simultaneous recording of video, operator commands, actuation state, and distal tip pose, validates motion consistency and cross-modal latency, and releases a multimodal dataset of 1,894 teleoperated episodes (~19 hours) across 10 structured task variations of navigation, failure events, and recovery behaviors to support research in multimodal robotic colonoscopy and surgical autonomy.

Significance. If the central claims hold, OpenRC would provide a valuable open hardware platform and aligned multimodal dataset for reproducible research in robotic colonoscopy, vision-language-action learning, and surgical autonomy, addressing the current lack of systematic data on coupled operator-instrument-visual dynamics. The open-source release and dataset contribution are explicit strengths for community use.

major comments (2)

[Abstract] Abstract: The claim that the retrofit 'preserves clinical workflow' is load-bearing for the dataset's value as a foundation for autonomy research, yet the reported validation covers only motion consistency and cross-modal latency quantification; no quantitative comparisons of insertion forces, tip control precision under load, haptic feedback, or endoscopist workload are described to support preservation of operator behavior and data fidelity.
[Experimental validation] Experimental validation section: The motion-consistency validation lacks error bars, statistical tests, or raw data release, which prevents independent verification of the reproducibility claims and leaves the soundness of the platform's fidelity assertions at the reported low level.

minor comments (1)

[Dataset] The manuscript should explicitly define the 10 task variations and episode selection criteria in the dataset section to strengthen reproducibility.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive feedback on our manuscript. We address each major comment point by point below, with proposed revisions where appropriate to strengthen the presentation of OpenRC.

read point-by-point responses

Referee: [Abstract] Abstract: The claim that the retrofit 'preserves clinical workflow' is load-bearing for the dataset's value as a foundation for autonomy research, yet the reported validation covers only motion consistency and cross-modal latency quantification; no quantitative comparisons of insertion forces, tip control precision under load, haptic feedback, or endoscopist workload are described to support preservation of operator behavior and data fidelity.

Authors: We acknowledge the referee's point that stronger quantitative evidence would better support the workflow-preservation claim. The claim is grounded in the retrofit's modular design, which attaches externally to conventional colonoscopes without modifying the scope's core manual handling, insertion technique, or operator interface, thereby maintaining standard clinical use. The reported validation (motion consistency and latency) confirms that the added actuation does not disrupt basic motion fidelity or data alignment. However, we did not perform the additional metrics mentioned (forces, precision under load, haptics, workload). In revision we will qualify the abstract claim to specify 'preserves clinical workflow in terms of operator handling and scope compatibility' and add an explicit limitations paragraph discussing the scope of current validation and the value of future studies on these metrics. The open-source release enables such extensions by the community. revision: partial
Referee: [Experimental validation] Experimental validation section: The motion-consistency validation lacks error bars, statistical tests, or raw data release, which prevents independent verification of the reproducibility claims and leaves the soundness of the platform's fidelity assertions at the reported low level.

Authors: We agree that the current presentation of motion-consistency results can be improved for reproducibility. In the revised manuscript we will add error bars (standard deviation) to all reported consistency metrics, include statistical tests (e.g., paired t-tests or ANOVA on repeated trials), and release the raw validation trial data in the public dataset repository alongside the 1,894 episodes. This will allow independent verification of the platform's fidelity claims. revision: yes

Circularity Check

0 steps flagged

No circularity: systems platform and dataset release with no derivation chain

full rationale

The paper describes an open-source robotic colonoscopy framework (OpenRC) that retrofits conventional scopes and collects a multimodal dataset of 1,894 episodes. Its central claim is that this unification provides a reproducible foundation for research. No equations, fitted parameters, predictions, or mathematical derivations are present in the abstract or described structure. Validation is limited to experimental motion consistency and latency checks, which do not reduce to self-referential definitions or self-citations. The contribution is the hardware/software platform and data release itself, not a derived result that collapses to its inputs by construction. This matches the default expectation for non-derivational systems papers.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The work is an engineering systems contribution. It relies on standard robotics assumptions about sensor accuracy and actuation repeatability but introduces no free parameters, new physical entities, or ad-hoc axioms beyond domain-standard ones.

axioms (1)

domain assumption Standard assumptions in robotics for distal tip pose estimation from integrated sensors
The framework records distal tip pose and assumes these measurements are sufficiently accurate for the intended research use cases.

pith-pipeline@v0.9.0 · 5490 in / 1175 out tokens · 35865 ms · 2026-05-13T16:52:33.714907+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

28 extracted references · 28 canonical work pages · 1 internal anchor

[1]

Medical engineering & physics38 2, 148–54 (2016)

Alazmani, A., Hood, A., Jayne, D.G., Neville, A., Culmer, P.: Quantitative as- sessment of colorectal morphology: Implications for robotic colonoscopy. Medical engineering & physics38 2, 148–54 (2016)

work page 2016
[2]

Scientific Data10(2022)

Azagra, P., Sostres, C., Ferrandez, A., Riazuelo, L., Tomasini, C., Barbed, O.L., Morlana, J., Recasens, D., Batlle, V.M., G’omez-Rodr’iguez, J.J., Elvira, R., L’opez, J., Oriol, C., Civera, J., Tard’os, J.D., Murillo, A.C., Lanas, Á., Mon- tiel, J.M.M.: Endomapper dataset of complete calibrated endoscopy procedures. Scientific Data10(2022)

work page 2022
[3]

Surgical Endoscopy38, 1096–1105 (2023)

Basha, S., Khorasani, M., Abdurahiman, N., Padhan, J., Baez, V.M., Al-Ansari, A.A., Tsiamyrtzis, P., Becker, A.T., Navkar, N.V.: A generic scope actua- tion system for flexible endoscopes. Surgical Endoscopy38, 1096–1105 (2023). https://doi.org/10.1007/s00464-023-10616-7

work page doi:10.1007/s00464-023-10616-7 2023
[4]

saliency maps from physicians

Bernal, J., Sánchez, F.J., Fernández-Esparrach, G., Gil, D., Rodríguez, C., Vilar- iño, F.: Wm-dova maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians. Computerized Medical Imaging and Graphics 43, 99–111 (2015)

work page 2015
[5]

Scientific Data11(2024)

Biffi, C., Antonelli, G., Bernhofer, S., Hassan, C., Hirata, D., Iwatate, M., Maieron, A., Salvagnini, P., Cherubini, A.: Real-colon: A dataset for developing real-world ai applications in colonoscopy. Scientific Data11(2024)

work page 2024
[6]

Scientific Data7(1), 283 (2020)

Borgli, H., Thambawita, V., Smedsrud, P.H., Hicks, S., Jha, D., Eskeland, S.L., Randel, K.R., Pogorelov, K., Lux, M., Nguyen, D.T.D., Johansen, D., Griwodz, C., Stensland, H.K., Garcia-Ceja, E., Schmidt, P.T., Hammer, H.L., Riegler, M.A., Halvorsen, P., de Lange, T.: Hyperkvasir, a comprehensive multi-class image and video dataset for gastrointestinal end...

work page doi:10.1038/s41597-020-00622-y 2020
[7]

Current Robotics Reports2(1), 43–54 (2021)

Boškoski, I., Orlandini, B., Papparella, L.G., Matteo, M.V., De Siena, M., Pon- tecorvi, V., Costamagna, G.: Robotics and artificial intelligence in gastrointestinal endoscopy: updated review of the literature and state of the art. Current Robotics Reports2(1), 43–54 (2021)

work page 2021
[8]

https://github.com/huggingface/lerobot (2024) 10 S

Cadene, R., Alibert, S., Soare, A., Gallouedec, Q., Zouitine, A., Palma, S., Kooij- mans, P., Aractingi, M., Shukor, M., Aubakirova, D., Russi, M., Capuano, F., Pas- cal, C., Choghari, J., Moss, J., Wolf, T.: Lerobot: State-of-the-art machine learning for real-world robotics in pytorch. https://github.com/huggingface/lerobot (2024) 10 S. Kapuria et al

work page 2024
[9]

Sci- entific Data12(2025)

Cold, K.M., Vamadevan, A., Heen, A., Vilmann, A.S., Bulut, M., Kovačević, B., Rasmussen, M., Konge, L., Svendsen, M.B.S.: Mapping the colon through the colonoscope’s coordinates – the copenhagen colonoscopy coordinate database. Sci- entific Data12(2025)

work page 2025
[10]

Open X-Embodiment: Robotic Learning Datasets and RT-X Models

Collaboration, O.X.E.: Open X-Embodiment: Robotic learning datasets and RT-X models. https://arxiv.org/abs/2310.08864 (2023)

work page internal anchor Pith review Pith/arXiv arXiv 2023
[11]

arXiv preprint arXiv:2506.24074 (2025)

Golhar, M.V., Fretes, L.S.G., Ayers, L., Akshintala, V.S., Bobrow, T.L., Durr, N.J.: C3vdv2–colonoscopy 3d video dataset with enhanced realism. arXiv preprint arXiv:2506.24074 (2025)

work page arXiv 2025
[12]

CA: A Cancer Journal for Clinicians52(2002)

Jemal, A., Thomas, A., Murray, T., Thun, M.J.: Cancer statistics, 2002. CA: A Cancer Journal for Clinicians52(2002)

work page 2002
[13]

In: MultiMedia Modeling: 26th International Conference, MMM 2020, Daejeon, South Korea, January 5–8, 2020, Proceedings, Part II 26

Jha, D., Smedsrud, P.H., Riegler, M.A., Halvorsen, P., de Lange, T., Johansen, D., Johansen, H.D.: Kvasir-seg: A segmented polyp dataset. In: MultiMedia Modeling: 26th International Conference, MMM 2020, Daejeon, South Korea, January 5–8, 2020, Proceedings, Part II 26. pp. 451–462 (2020)

work page 2020
[14]

Gastroenterology (2020)

Kaltenbach, T., Anderson, J.C., Burke, C.A., Dominitz, J.A., Gupta, S., Lieber- man, D.A., Robertson, D.J., Shaukat, A., Syngal, S., Rex, D.K.: Endoscopic re- moval of colorectal lesions-recommendations by the us multi-society task force on colorectal cancer. Gastroenterology (2020)

work page 2020
[15]

Annals of Biomedical Engineering51, 1499–1512 (2023)

Kara, O.C., Venkatayogi, N., Ikoma, N., Alambeigi, F.: A reliable and sensitive framework for simultaneous type and stage detection of colorectal cancer polyps. Annals of Biomedical Engineering51, 1499–1512 (2023)

work page 2023
[16]

Gastrointestinal endoscopy clinics of North America20 4, 659–71 (2010)

Ko, C.W., Dominitz, J.A.: Complications of colonoscopy: magnitude and manage- ment. Gastrointestinal endoscopy clinics of North America20 4, 659–71 (2010)

work page 2010
[17]

Endoscopy47, 815–819 (2015)

Kume, K., Sakai, N., Goto, T.: Development of a novel endoscopic manipula- tion system: the endoscopic operation robot ver.3. Endoscopy47, 815–819 (2015). https://doi.org/10.1055/s-0034-1391938

work page doi:10.1055/s-0034-1391938 2015
[18]

The International Journal of Medical Robotics and Computer Assisted Surgery17(3), e2168 (2020)

Lee, D.H., Cheon, B., Kim, J., Kwon, D.S.: easyendo robotic endoscopy system: Development and usability test in a randomized controlled trial with novices and physicians. The International Journal of Medical Robotics and Computer Assisted Surgery17(3), e2168 (2020). https://doi.org/10.1002/rcs.2168

work page doi:10.1002/rcs.2168 2020
[19]

In: Proceedings of the 7th international joint conference on Artificial intelligence-Volume 2

Lucas, B.D., Kanade, T.: An iterative image registration technique with an appli- cation to stereo vision. In: Proceedings of the 7th international joint conference on Artificial intelligence-Volume 2. pp. 674–679. Morgan Kaufmann Publishers Inc. (1981)

work page 1981
[20]

Journal of Imaging10(11), 275 (2024)

Nie, Z., Xu, M., Wang, Z., Lu, X., Song, W.: A review of application of deep learning in endoscopic image processing. Journal of Imaging10(11), 275 (2024)

work page 2024
[21]

Medical image analysis71, 102058 (2021)

Ozyoruk, K.B., Gokceler, G.I., Bobrow, T.L., Coskun, G., Incetan, K., Almalioglu, Y., Mahmood, F., Curto, E., Perdigoto, L., Oliveira, M., et al.: Endoslam dataset and an unsupervised monocular visual odometry and depth estimation approach for endoscopic videos. Medical image analysis71, 102058 (2021)

work page 2021
[22]

Medical Image Analysis96, 103195 (2024)

Rau, A., Bano, S., Jin, Y., Azagra, P., Morlana, J., Kader, R., Sanderson, E., Matuszewski, B.J., Lee, J.Y., Lee, D.J., et al.: Simcol3d—3d reconstruction during colonoscopy challenge. Medical Image Analysis96, 103195 (2024)

work page 2024
[23]

IEEE Transactions on Medical Robotics and Bionics5(4), 978–989 (2023)

Rau, A., Bhattarai, B., Agapito, L., Stoyanov, D.: Bimodal camera pose prediction for endoscopy. IEEE Transactions on Medical Robotics and Bionics5(4), 978–989 (2023)

work page 2023
[24]

CA: A Cancer Journal for Clinicians 71, 209 – 249 (2021) OpenRC: Open-Source Robotic Colonoscopy Framework 11

Sung, H., Ferlay, J., Siegel, R.L., Laversanne, M., Soerjomataram, I., Jemal, A., Bray,F.:Globalcancerstatistics2020:Globocanestimatesofincidenceandmortal- ity worldwide for 36 cancers in 185 countries. CA: A Cancer Journal for Clinicians 71, 209 – 249 (2021) OpenRC: Open-Source Robotic Colonoscopy Framework 11

work page 2021
[25]

Annals of Coloproctology 39(5), 385 (2023)

Tham, S., Koh, F.H., Ladlad, J., Chue, K.M., Centre, S.E., Lin, C.L., Teo, E.K., Foo, F.J.: The imitation game: a review of the use of artificial intelligence in colonoscopy, and endoscopists’ perceptions thereof. Annals of Coloproctology 39(5), 385 (2023)

work page 2023
[26]

In: Conference on Robot Learning (CoRL) (2023)

Walke, H., Black, K., Lee, A., Kim, M.J., Du, M., Zheng, C., Zhao, T., Hansen- Estruch, P., Vuong, Q., He, A., Myers, V., Fang, K., Finn, C., Levine, S.: Bridge- data v2: A dataset for robot learning at scale. In: Conference on Robot Learning (CoRL) (2023)

work page 2023
[27]

Translational Oncology14(10), 101174 (2021)

Xi, Y., Xu, P.: Global colorectal cancer burden in 2020 and projections to 2040. Translational Oncology14(10), 101174 (2021)

work page 2020
[28]

Gastroenterology156 6, 1661–1674.e11 (2019)

Zhao, S.B., Wang, S., Pan, P., Xia, T., Chang, X., Yang, X., Guo, L., Meng, Q.Q., Yang, F., Qian, W., Xu, Z., Wang, Y., Wang, Z., Gu, L., Wang, R., Jia, F., Yao, J., Li, Z., Bai, Y.: Magnitude, risk factors, and factors associated with adenoma miss rate of tandem colonoscopy: A systematic review and meta-analysis. Gastroenterology156 6, 1661–1674.e11 (2019)

work page 2019