Pith. sign in

REVIEW 4 cited by

Data Splits and Metrics for Method Benchmarking on Surgical Action Triplet Datasets

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2204.05235 v2 pith:HPGTSSQP submitted 2022-04-11 cs.CV

Chinedu Innocent Nwoye , Nicolas Padoy This is my paper · ORCID
classification cs.CV
keywords datadatasetmetricssplitssurgicalcholect50evaluationresearch
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

In addition to generating data and annotations, devising sensible data splitting strategies and evaluation metrics is essential for the creation of a benchmark dataset. This practice ensures consensus on the usage of the data, homogeneous assessment, and uniform comparison of research methods on the dataset. This study focuses on CholecT50, which is a 50 video surgical dataset that formalizes surgical activities as triplets of <instrument, verb, target>. In this paper, we introduce the standard splits for the CholecT50 and CholecT45 datasets and show how they compare with existing use of the dataset. CholecT45 is the first public release of 45 videos of CholecT50 dataset. We also develop a metrics library, ivtmetrics, for model evaluation on surgical triplets. Furthermore, we conduct a benchmark study by reproducing baseline methods in the most predominantly used deep learning frameworks (PyTorch and TensorFlow) to evaluate them using the proposed data splits and metrics and release them publicly to support future research. The proposed data splits and evaluation metrics will enable global tracking of research progress on the dataset and facilitate optimal model selection for further deployment.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Current validation practice undermines surgical AI development

    q-bio.OT 2025-11 conditional novelty 7.0 of 10

    A multi-stage Delphi consensus with 92 experts catalogs widespread validation pitfalls in surgical AI video analysis across data, metrics, and reporting, supported by a systematic review and empirical experiments.

  2. LAVIFT: Latent-Action-Guided Vision Fine-Tuning for Surgical Interaction Recognition

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Latent-action modeling (inverse dynamics plus forward world model) with a patch-level anti-collapse regularizer improves surgical action-triplet recognition and makes encoder change features land more on instrument-ti...

  3. Recognize Any Surgical Object: Unleashing the Power of Weakly-Supervised Data

    cs.CV 2025-01 conditional novelty 6.0 of 10

    RASO recognizes surgical instruments and anatomy in images and video using a weakly supervised training pipeline built from automatically generated tag-image-text pairs from surgical lecture videos.

  4. Large-scale Self-supervised Video Foundation Model for Intelligent Surgery

    cs.CV 2025-06 conditional novelty 5.0 of 10

    SurgVISTA is a masked-reconstruction surgical video foundation model whose joint spatiotemporal pretraining plus expert distillation outperforms image-level and natural-video pretrained models on 13 surgical benchmarks.

Pith tools