Pith. sign in

REVIEW 4 major objections 4 minor 52 references

SurgPIS: Surgical-instrument-level Instances and Part-level Semantics for Weakly-supervised Part-aware Instance Segmentation

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SurgPIS is the first part-aware instance segmentation model for surgical instruments, predicting each instrument's identity and its parts together.

desk verdict Solid architecture and results, but the weak-supervision loss is undefined as written, so the central claim needs a major revision before it can be checked. read the letter →

arxiv 2507.19592 v1 pith:WBBJVWI2 submitted 2025-07-25 cs.CV

Meng Wei , Charlie Budd , Oluwatosin Alabi , Miaojing Shi , Tom Vercauteren This is my paper · ORCID
classification cs.CV
keywords part-awareinstancesegmentationsurgicalinstrumentweakly-supervisedlearningteacher-studentconsistencymaskclassificationtransformerqueriesrobot-assistedsurgeryEndoVis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SurgPIS is proposed as the first part-aware instance segmentation (PIS) model for surgical instruments: it outputs, for every instrument, its class, its instance mask, and the masks of its parts (shaft, wrist, clasper) in a single forward pass. The paper's central claim is that these three levels should be learned together, and that a single transformer-based model can do so without requiring a dataset that has all three annotations. To get around the absence of large PIS-labelled surgical datasets, SurgPIS is trained on disjoint datasets that carry only instance-level labels or only part-level labels, using a teacher-student strategy plus mask aggregation to turn its predictions into the available label type. If the claims hold, one model replaces separate instrument-instance, instrument-semantic, and part-semantic segmenters, and it reaches state-of-the-art PartPQ of 77.92 on EndoVis2018 with a Swin-B backbone.

What carries the argument

The load-bearing mechanism is the part-specific query transformation: an MLP maps each of the $N_q$ instrument-level queries $Q$ to $C_{\mathrm{part}}$ part-specific queries $Q_{\mathrm{part}} \in \mathbb{R}^{(C_{\mathrm{part}}\times N_q)\times C_\epsilon}$, which are multiplied with the high-resolution feature map and passed through a sigmoid to produce the part masks $\hat m_{j,k}$. This makes every part mask inherit the instrument instance's identity. Two supporting mechanisms carry the weakly supervised stage: part-aware bipartite matching adds a part-level mask loss $L_{\mathrm{pm}}$ to the matching cost, and part-semantic mask aggregation builds soft part-level maps $\rho_k$ by weighting each predicted part mask by the probability of its argmax instrument class (Eq. 9), then renormalizing pixel-wise to a probability distribution. The teacher-student loop with EMA and Dice-based pseudo-label filtering supplies consistent PIS pseudo-labels when only partial annotations exist.

What would settle it

Train SurgPIS on EndoVis2018 PIS labels plus SAR-RARP50 PSS labels twice: once with Eq. (9) as written, and once with $\rho_k$ built from ground-truth instrument classes (or from a classifier known to be correct), then compare PartPQ on EndoVis2018. If the oracle-assisted version is materially better, the argmax assumption is load-bearing; if the numbers are equal, the weak-supervision path is robust to instrument-class errors. A simpler observation: on a PSS-only image where the model visibly mislabels an instrument's class, check whether the part-semantic loss $L_{\mathrm{pss}}^{\mathrm{wks}}$ rises even though the part masks are correct.

Watch

Extended reading notes

Core claim

The paper introduces SurgPIS, built by extending Mask2Former with a part-specific query transformation: after instrument-level object queries are processed by a transformer decoder, an MLP transforms each of them into a fixed set of part-specific queries, so each predicted part mask is explicitly attached to its parent instrument instance. Training with full PIS labels uses a part-aware bipartite matching whose cost matrix includes a part-level mask loss, improving assignment. For weakly labelled data, the student model's PIS predictions are aggregated into part-semantic maps (for PSS-only data) or instrument-instance maps (for IIS-only data), and losses are computed against those partial labels, while an EMA teacher generates pseudo PIS labels filtered by a Dice-confidence threshold. On EndoVis2018 with a Swin-B backbone the model achieves 77.92 PartPQ and outperforms the Mask2Former-based BPSS⊕BIIS baseline by 11.07 percentage points in the fully supervised setting and 14.49 in the weakly supervised combined-dataset setting; similar margins hold on EndoVis2017.

Load-bearing premise

When training on part-level-only data, the method assumes that the argmax of the predicted instrument-class probability for each predicted part mask is the correct instrument class; if that class is wrong, the part masks are accumulated under the wrong part label and the weakly supervised part loss is computed against corrupted targets.

Editorial extensions

If this is right

  • A single SurgPIS model can output PIS, IIS, PSS, and ISS predictions by aggregating its predicted masks, replacing separate task-specific models.
  • Combining PSS-only and IIS-only datasets during the weakly supervised stage improves PartPQ beyond training on PIS labels alone.
  • On EndoVis2018 with a Swin-B backbone, SurgPIS reaches 77.92 PartPQ, outperforming the Mask2Former-based BPSS⊕BIIS baseline by 11.07 pp fully supervised and 14.49 pp weakly supervised; similar margins hold on EndoVis2017.
  • With a ResNet-50 backbone, SurgPIS runs at roughly 30 FPS on 512×512 inputs, making real-time use plausible.
  • On the unseen GraSP test set, it keeps competitive instrument-level PQ and exceeds the PIS baselines, indicating cross-dataset generalization.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Replacing the argmax over predicted instrument classes in Eq. (9) with oracle classes would likely produce cleaner soft part maps and stronger PSS weak supervision; the paper does not ablate this, so the reported gains may understate the aggregation's ceiling.
  • The part-specific query transformation should transfer to other domains where distinct object categories share identical part types and look alike, such as industrial or endoscopic tools, where the paper shows natural-image PIS models fail.
  • The paper reports a drop when PSS-only data is removed, but does not separate the effect of extra data from the effect of the teacher-student consistency loss, leaving the marginal contribution of each weakly labelled dataset as an open question.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes SurgPIS, a Mask2Former-based model that jointly predicts surgical instrument instances, their instrument classes, and per-instance part masks. The two claimed contributions are (i) a part-specific query transformation that derives part queries from instrument-level queries, and (ii) a two-stage weakly supervised training scheme that first trains on PIS-labelled data and then additionally trains on disjoint datasets labelled only for IIS or PSS, using mask aggregation and a student-teacher consistency mechanism. Results are reported on EndoVis2018, EndoVis2017, SAR-RARP50, and a PIS-annotated GraSP test set, with PartPQ, PQ, and PartIoU gains over a Mask2Former-based baseline (BPSS⊕BIIS) and over TAPPS, together with ablations of the query transformation, bipartite matching, pseudo-label filtering, and augmentation strategies. The paper claims to be the first to address part-aware instance segmentation for surgical instruments, and its main empirical claim is that SurgPIS outperforms prior single-task and part-aware panoptic models across PIS, IIS, ISS, and PSS tasks.

Significance. The problem of learning from datasets with disjoint annotation granularities is important for surgical vision, where no large PIS dataset exists, and the paper is among the first to tackle it in this domain. The empirical evaluation is broad: four datasets, multiple backbones, cross-dataset generalization on GraSP, and ablations of the two main architectural components. If the weak-supervision mechanism were fully specified and reproducible, the contribution would be valuable and the reported PartPQ improvements over the strong Mask2Former-based baseline (11.07–14.49 pp) would be noteworthy. However, the central weak-supervision loss in Eq. (13) is not implementable as written, and the pseudo-label filter appears to require ground-truth part masks that do not exist for the weakly labelled datasets. Because this mechanism carries the paper's main claim of learning from disjoint PSS/IIS data, the current manuscript cannot be independently checked or reproduced despite the otherwise strong empirical scope.

major comments (4)
  1. [III-C, Eq. (13)] The total weakly supervised loss Lwks is written as Lwks = L_sup_teach + {Lwks_pss or Lwks_iis}, but L_sup_teach is never defined anywhere in the text. Section III-C1 only describes the EMA update in Eq. (7) and a Dice-based filtering criterion. Without a precise formula for L_sup_teach, including how teacher PIS masks are matched to student predictions and which losses are applied, the weak-supervision objective is not executable, and the central claim that SurgPIS can be trained on disjoint PSS/IIS datasets cannot be checked from the manuscript. Please add the missing definition and all associated matching and normalization details.
  2. [III-C1, pseudo-label filtering] The filtering condition is written as Dice(ˆm_teach_{τi,k}, m_{i,k})_{i≠0} > Tresh_Dice, where m_{i,k} in Eq. (1) denotes ground-truth part masks. For DPSS and DIIS, those ground-truth part masks do not exist by construction; if m_{i,k} is instead taken to be the teacher's own pseudo-label, then Dice(teacher, itself)=1, making the filter vacuous. The threshold Tresh_Dice is also not reported. Please clarify what m_{i,k} is in the weakly supervised setting and how the filter is actually computed, and report the threshold value.
  3. [IV-B and Fig. 7] The implementation details state that the EMA decay rate is set to α=0.99, while the sensitivity analysis in Fig. 7 shows that a decay rate around 0.995 consistently yields the best performance on all evaluated metrics. This is an internal contradiction. Either adopt the value suggested by the authors' own sensitivity analysis or explain why α=0.99 is retained; as written, the final model uses a hyperparameter that the paper itself identifies as suboptimal.
  4. [III-C2, Eq. (9)] The construction of the soft part-level semantic map ρk relies on γ̂_j = arg max over predicted instrument classes. If the instrument-class prediction is wrong, the predicted part masks are accumulated under the wrong part semantic class, and Lwks_pss in Eq. (11) is then computed against corrupted targets. This error path is load-bearing for the claimed benefit of PSS-only weak supervision, but it is not quantitatively analyzed or ablated. Please provide evidence on the accuracy of γ̂_j on weakly labelled data, or a mechanism that limits the damage caused by instrument-class misclassification.
minor comments (4)
  1. [IV-E4, Tables] There are two tables labeled 'Table II' in the manuscript: one for ISS comparison and one for the GraSP evaluation. The later table should be renumbered.
  2. [All tables] No error bars or confidence intervals are reported for any of the quantitative results. Given the small test sets and stochastic training, please report variance across multiple runs or at least provide the number of runs.
  3. [Section III-C title and III-C3] The title refers to 'PSS and ISS datasets,' but the formal notation and Eq. (13) use IIS (instance-level instance segmentation) for the instrument-only labels. Please align the terminology.
  4. [III-C1, notation] The threshold is written inconsistently as 'TreshDice' and 'Tresh_Dice' across the text and figures. Please use a single notation throughout.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation found: SurgPIS's main results rest on held-out evaluation against external baselines; the weak-supervision gaps are reproducibility issues, not self-referential reductions.

full rationale

The paper's central derivations are not circular. The architecture (instrument-level queries transformed into part-specific queries, hierarchical bipartite matching with part-level mask loss) is defined independently of the PIS predictions it produces; the supervised loss Lsup = Lic + Lim + Lpm compares predictions to external ground-truth PIS labels. The weakly supervised path aggregates student PIS outputs into PSS maps via Eqs. (8)-(10) and IIS outputs via Eq. (12), then computes losses against the available partial ground truth; this is a projection of predictions onto the label space, not a fit of the target from itself. Evaluation is held out on EndoVis2018/2017, SAR-RARP50 and GraSP against external baselines, so the headline PartPQ numbers do not reduce to training inputs. The paper does cite prior work by overlapping authors (e.g., Refs. [10], [11], [23]), but none of these citations carries a load-bearing uniqueness or derivation claim; they are background and dataset references. Two textual gaps are worth flagging but are not circularity: Eq. (13) references L_sup_teach without giving its formula, and the Dice filter in Sec. III-C1 is written against mi,k, which does not exist for PSS/IIS-only data; if mi,k is instead taken to be the teacher's own output, the filter would be identity. These are implementability and validation omissions, not demonstrations that a prediction equals its input by construction. Accordingly, no circular step is established and the circularity score is 0.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

No physical entities are introduced. The central claim rests on dataset assumptions (fixed part taxonomy, correct manual label merges) and on the unvalidated reliability of teacher pseudo-labels and argmax-based part aggregation.

free parameters (6)
  • EMA decay rate alpha = 0.99 (sensitivity suggests 0.995)
    Set in Section IV-B for the teacher update in Eq. (7); Section IV-E6 and Fig. 7 report 0.995 as optimal, so the chosen value is not the best according to the paper's own analysis.
  • Instrument-level classification loss weight alpha_class = 5.0
    Set in Section IV-B; Fig. 6 indicates a range of 4 to 6 is optimal.
  • Instrument/part mask loss weight alpha_mask = 5.0
    Set in Section IV-B; Fig. 6 indicates a range of 4 to 6 is optimal.
  • Weak supervision loss weights for PSS and IIS data = 3.0 and 2.0
    Set in Section IV-B for Eq. (13); no sensitivity analysis is reported for these weights.
  • Pseudo-label filtering threshold Tresh_Dice = not reported
    Section III-C1 uses a Dice-based mask scoring filter with a threshold, but no value is given anywhere in the paper.
  • Maximum number of queries Nq = not specified
    Assumed upper bound on instrument instances in Section III-A; the value is not stated.
assumptions (5)
  • domain assumption Each instrument instance is exactly the union of its part masks, and part masks are disjoint.
    Stated as a consistency property in Section III-A (Eq. 1). This is true for the surgical tools in the chosen datasets but is a modelling restriction.
  • domain assumption Instrument classes and the fixed part taxonomy are shared across EndoVis2017, EndoVis2018, SAR-RARP50, and GraSP after relabelling.
    Section IV-A recategorizes GraSP's laparoscopic grasper as grasping retractor; if this identity is wrong, cross-dataset evaluation is affected.
  • domain assumption Manually constructed PIS labels for EndoVis2017 by combining IIS and PSS annotations are correct.
    Section IV-A: the authors created IIS annotations and merged them with existing PSS annotations; errors in this merge would corrupt the fully supervised first stage on that dataset.
  • ad hoc to paper Teacher pseudo-labels from the EMA model are reliable training targets after Dice filtering.
    Section III-C1: the student is trained against the teacher's pseudo PIS labels; the threshold Tresh_Dice is not reported, so this reliability is assumed rather than demonstrated.
  • ad hoc to paper The argmax over predicted instrument classes in Eq. (9) assigns part masks to the correct semantic part classes.
    The PSS aggregation in Section III-C2 depends on this argmax; incorrect instrument class predictions would corrupt the part-level semantic map and the loss in Eq. (11).

how reviews work

0 comments
Cite this review

Pith. "Pith review of SurgPIS: Surgical-instrument-level Instances and Part-level Semantics for Weakly-supervised Part-aware Instance Segmentation." pith.science (2026). https://pith.science/paper/WBBJVWI2

@misc{pith2026250719592,
  author       = {Pith},
  title        = {Pith review of: SurgPIS: Surgical-instrument-level Instances and Part-level Semantics for Weakly-supervised Part-aware Instance Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WBBJVWI2}},
  note         = {Machine review of arXiv:2507.19592}
}
read the original abstract

Consistent surgical instrument segmentation is critical for automation in robot-assisted surgery. Yet, existing methods only treat instrument-level instance segmentation (IIS) or part-level semantic segmentation (PSS) separately, without interaction between these tasks. In this work, we formulate a surgical tool segmentation as a unified part-aware instance segmentation (PIS) problem and introduce SurgPIS, the first PIS model for surgical instruments. Our method adopts a transformer-based mask classification approach and introduces part-specific queries derived from instrument-level object queries, explicitly linking parts to their parent instrument instances. In order to address the lack of large-scale datasets with both instance- and part-level labels, we propose a weakly-supervised learning strategy for SurgPIS to learn from disjoint datasets labelled for either IIS or PSS purposes. During training, we aggregate our PIS predictions into IIS or PSS masks, thereby allowing us to compute a loss against partially labelled datasets. A student-teacher approach is developed to maintain prediction consistency for missing PIS information in the partially labelled data, e.g., parts of the IIS labelled data. Extensive experiments across multiple datasets validate the effectiveness of SurgPIS, achieving state-of-the-art performance in PIS as well as IIS, PSS, and instrument-level semantic segmentation.

Figures

Figures reproduced from arXiv: 2507.19592 by the authors.

Figure 1
Figure 1. SurgPIS is the first unified model for surgical instru￾ment segmentation, capable of predicting both instrument-level instances (i.e. IIS and ISS) and part-aware instances (i.e. PIS and PSS) based on a query-based transformation approach. It leverages weak PIS supervision from disjoint PSS and ISS datasets to learn from partially labelled data in different granularities. support downstream tasks such as tool trackin… view at source ↗
Figure 2
Figure 2. SurgPIS extends Mask2Former (grey) by a novel query transformation module to get part-specific queries, and part￾aware bipartite matching (blue) to calculate the losses at the instrument-level and part-level. can be calculated by averaging the contribution of each pair: L ic(ˆc, c) = 1 Nx X i<Nx ℓCE(ˆcτi , ci) (4) L im(ˆy, y) = 1 Nx X i<Nx ℓM(ˆyτi , yi) (5) Similarly, the part-level mask loss is given as: L pm( ˆm, … view at source ↗
Figure 3
Figure 3. Weak supervision training for SurgPIS contains student and teacher SurgPIS models: The teacher SurgPIS generates pseudo-ground-truth PIS masks for comparison with the student PIS predictions. The grey part is the shared process, while the green part is the separated process where the student PIS outputs are aggregated specifically for comparison against either IIS labels or PSS labels. We note that [ρ0, . . . , ρCpa… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Visualization comparing our SurgPIS model with our proposed strong baseline (BPSS [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Visualization examples of our SurgPIS model trained [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Sensitivity analysis for mask loss weight [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: EMA decay rate sensitivity analysis for En [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

52 extracted references · 45 canonical work pages

  1. [1]

    Robotic assisted minimally invasive surgery,

    J. H. Palep, “Robotic assisted minimally invasive surgery,” Journal of minimal access surgery , vol. 5, no. 1, pp. 1–7, 2009

    work page 2009

  2. [2]

    2018 robotic scene segmentation challenge,

    M. Allan, S. Kondo, S. Bodenstedt, S. Leger, R. Kadkhodamohammadi, I. Luengo, F. Fuentes, E. Flouty, A. Mohammed et al. , “2018 robotic scene segmentation challenge,” arXiv preprint, 2020

    work page 2018

  3. [3]

    A kinematic bottleneck approach for pose regression of flexible surgical instruments directly from images,

    L. Sestini, B. Rosa, E. De Momi, G. Ferrigno, and N. Padoy, “A kinematic bottleneck approach for pose regression of flexible surgical instruments directly from images,” IEEE Robotics and Automation Letters, vol. 6, no. 2, pp. 2938–2945, 2021

    work page 2021

  4. [4]

    Recognition of instrument-tissue interactions in endo- scopic videos via action triplets,

    C. I. Nwoye, C. Gonzalez, T. Yu, P. Mascagni, D. Mutter, J. Marescaux, and N. Padoy, “Recognition of instrument-tissue interactions in endo- scopic videos via action triplets,” in MICCAI. Springer, 2020, pp. 364–374

    work page 2020

  5. [5]

    Deep learning assisted robotic magnetic anchored and guided endoscope for real-time instrument tracking,

    T. Cheng, W. Li, W. Y . Ng, Y . Huang, J. Li, C. S. H. Ng, P. W. Y . Chiu, and Z. Li, “Deep learning assisted robotic magnetic anchored and guided endoscope for real-time instrument tracking,” IEEE Robotics and Automation Letters, vol. 6, no. 2, pp. 3979–3986, 2021

    work page 2021

  6. [6]

    Co-optimizing robot, environ- ment, and tool design via joint manipulation planning,

    M. Toussaint, J.-S. Ha, and O. S. Oguz, “Co-optimizing robot, environ- ment, and tool design via joint manipulation planning,” in 2021 IEEE International Conference on Robotics and Automation . IEEE, 2021, pp. 6600–6606

    work page 2021

  7. [7]

    Cholectriplet2021: A benchmark challenge for surgical action triplet recognition,

    C. I. Nwoye, D. Alapatt, T. Yu, A. Vardazaryan, F. Xia, Z. Zhao, T. Xia, F. Jia, Y . Yang et al. , “Cholectriplet2021: A benchmark challenge for surgical action triplet recognition,” Medical Image Analysis , vol. 86, p. 102803, 2023

    work page 2023

  8. [8]

    Toward image-guided automated suture grasping under com- plex environments: A learning-enabled and optimization-based holistic framework,

    B. Lu, B. Li, W. Chen, Y . Jin, Z. Zhao, Q. Dou, P.-A. Heng, and Y . Liu, “Toward image-guided automated suture grasping under com- plex environments: A learning-enabled and optimization-based holistic framework,” IEEE Transactions on Automation Science and Engineer- ing, vol. 19, no. 4, pp. 3794–3808, 2021

    work page 2021

Show all 52 references
  1. [9]

    Incorporating temporal prior from motion flow for instrument segmentation in minimally invasive surgery video,

    Y . Jin, K. Cheng, Q. Dou, and P.-A. Heng, “Incorporating temporal prior from motion flow for instrument segmentation in minimally invasive surgery video,” in MICCAI, 2019

    2019

  2. [10]

    Text promptable surgical instrument segmentation with vision-language models,

    Z. Zhou, O. Alabi, M. Wei, T. Vercauteren, and M. Shi, “Text promptable surgical instrument segmentation with vision-language models,” Ad- vances in Neural Information Processing Systems , vol. 36, pp. 28 611– 28 623, 2023. 10 SUBMITTED TO PREPRINT ARCHIVE

    2023

  3. [11]

    Enhancing surgical instrument segmentation: integrating vision transformer insights with adapter,

    M. Wei, M. Shi, and T. Vercauteren, “Enhancing surgical instrument segmentation: integrating vision transformer insights with adapter,” International Journal of Computer Assisted Radiology and Surgery , pp. 1–8, 2024

    2024

  4. [12]

    2017 robotic instrument segmentation challenge,

    M. Allan, A. Shvets, T. Kurmann, Z. Zhang, R. Duggal, Y .-H. Su, N. Rieke, I. Laina, N. Kalavakonda et al. , “2017 robotic instrument segmentation challenge,” arXiv preprint, 2019

    2017

  5. [13]

    Automatic instrument segmentation in robot-assisted surgery using deep learning,

    A. A. Shvets, A. Rakhlin, A. A. Kalinin, and V . I. Iglovikov, “Automatic instrument segmentation in robot-assisted surgery using deep learning,” in ICMLA, 2018

    2018

  6. [14]

    SAR-RARP50: Segmentation of surgical instrumentation and action recognition on robot-assisted radical prostatectomy challenge,

    D. Psychogyios, E. Colleoni, B. Van Amsterdam, C.-Y . Li, S.-Y . Huang, Y . Li, F. Jia, B. Zou, G. Wang et al., “SAR-RARP50: Segmentation of surgical instrumentation and action recognition on robot-assisted radical prostatectomy challenge,” arXiv preprint arXiv:2401.00496 , 2023

    2023 arXiv

  7. [15]

    ISINet: an instance- based approach for surgical instrument segmentation,

    C. Gonz ´alez, L. Bravo-S ´anchez, and P. Arbelaez, “ISINet: an instance- based approach for surgical instrument segmentation,” in MICCAI, 2020

    2020

  8. [16]

    TraSeTR: track-to-segment transformer with contrastive query for instance-level instrument segmentation in robotic surgery,

    Z. Zhao, Y . Jin, and P.-A. Heng, “TraSeTR: track-to-segment transformer with contrastive query for instance-level instrument segmentation in robotic surgery,” in IEEE International Conference on Robotics and Automation, 2022

    2022

  9. [17]

    From forks to forceps: A new framework for instance segmentation of surgical instruments,

    B. Baby, D. Thapar, M. Chasmai, T. Banerjee, K. Dargan, A. Suri, S. Banerjee, and C. Arora, “From forks to forceps: A new framework for instance segmentation of surgical instruments,” in WACV, 2023

    2023

  10. [18]

    Instances as queries,

    Y . Fang, S. Yang, X. Wang, Y . Li, C. Fang, Y . Shan, B. Feng, and W. Liu, “Instances as queries,” in Proceedings of the IEEE/CVF international conference on computer vision , 2021, pp. 6910–6919

    2021

  11. [19]

    Articulated multi-instrument 2-d pose estimation using fully convolutional networks,

    X. Du, T. Kurmann, P.-L. Chang, M. Allan, S. Ourselin, R. Sznitman, J. D. Kelly, and D. Stoyanov, “Articulated multi-instrument 2-d pose estimation using fully convolutional networks,” IEEE transactions on medical imaging, vol. 37, no. 5, pp. 1276–1287, 2018

    2018

  12. [20]

    Task-aligned part-aware panoptic seg- mentation through joint object-part representations,

    D. de Geus and G. Dubbelman, “Task-aligned part-aware panoptic seg- mentation through joint object-part representations,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 3174–3183

    2024

  13. [21]

    Masked-attention mask transformer for universal image segmentation,

    B. Cheng, I. Misra, A. G. Schwing, A. Kirillov, and R. Girdhar, “Masked-attention mask transformer for universal image segmentation,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 1290–1299

    2022

  14. [22]

    Robust medical instrument segmentation challenge 2019,

    T. Ross, A. Reinke, P. M. Full, M. Wagner, H. Kenngott, M. Apitz, H. Hempe, D. M. Filimon, P. Scholz et al., “Robust medical instrument segmentation challenge 2019,” arXiv preprint, 2020

    2019

  15. [23]

    CholecInstanceSeg: A tool instance segmentation dataset for laparoscopic surgery,

    O. Alabi, K. K. Z. Toe, Z. Zhou, C. Budd, N. Raison, M. Shi, and T. Ver- cauteren, “CholecInstanceSeg: A tool instance segmentation dataset for laparoscopic surgery,” arXiv preprint arXiv:2406.16039 , 2024

    2024 arXiv

  16. [24]

    Pixel-wise recognition for holistic surgical scene understanding,

    N. Ayobi and et al., “Pixel-wise recognition for holistic surgical scene understanding,” arXiv preprint, 2024

    2024

  17. [25]

    MATIS: Masked-attention transformers for surgical instrument segmentation,

    N. Ayobi, A. P ´erez-Rond´on, S. Rodr ´ıguez, and P. Arbel ´aez, “MATIS: Masked-attention transformers for surgical instrument segmentation,” arXiv preprint, 2023

    2023

  18. [26]

    Part-aware panoptic segmentation,

    D. de Geus, P. Meletis, C. Lu, X. Wen, and G. Dubbelman, “Part-aware panoptic segmentation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2021, pp. 5485–5494

    2021

  19. [27]

    Panoptic segmentation,

    A. Kirillov, K. He, R. Girshick, C. Rother, and P. Doll ´ar, “Panoptic segmentation,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2019, pp. 9404–9413

    2019

  20. [28]

    Aiparsing: Anchor- free instance-level human parsing,

    S. Zhang, X. Cao, G.-J. Qi, Z. Song, and J. Zhou, “Aiparsing: Anchor- free instance-level human parsing,” IEEE Transactions on Image Pro- cessing, vol. 31, pp. 5599–5612, 2022

    2022

  21. [29]

    Contextual instance decoupling for instance- level human analysis,

    D. Wang and S. Zhang, “Contextual instance decoupling for instance- level human analysis,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 8, pp. 9520–9533, 2023

    2023

  22. [30]

    Cp-net: Instance-aware part segmentation network for biological cell parsing,

    W. Chen, H. Song, C. Dai, Z. Huang, A. Wu, G. Shan, H. Liu, A. Jiang, X. Liu et al. , “Cp-net: Instance-aware part segmentation network for biological cell parsing,” Medical Image Analysis , vol. 97, p. 103243, 2024

    2024

  23. [31]

    Multi-task fusion for efficient panoptic-part segmentation,

    S. K. Jagadeesh, R. Schuster, and D. Stricker, “Multi-task fusion for efficient panoptic-part segmentation,” arXiv preprint arXiv:2212.07671 , 2022

    2022 arXiv

  24. [32]

    Panoptic- PartFormer: Learning a unified model for panoptic part segmentation,

    X. Li, S. Xu, Y . Yang, G. Cheng, Y . Tong, and D. Tao, “Panoptic- PartFormer: Learning a unified model for panoptic part segmentation,” in European Conference on Computer Vision. Springer, 2022, pp. 729– 747

    2022

  25. [33]

    Panoptic-PartFormer++: A unified and decoupled view for panoptic part segmentation,

    X. Li, S. Xu, Y . Yang, H. Yuan, G. Cheng, Y . Tong, Z. Lin, M.-H. Yang, and D. Tao, “Panoptic-PartFormer++: A unified and decoupled view for panoptic part segmentation,” IEEE transactions on pattern analysis and machine intelligence, vol. 46, 2024

    2024

  26. [34]

    Weakly supervised instance segmentation using class peak response,

    Y . Zhou, Y . Zhu, Q. Ye, Q. Qiu, and J. Jiao, “Weakly supervised instance segmentation using class peak response,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 3791– 3800

    2018

  27. [35]

    Boxinst: High-performance instance segmentation with box annotations,

    Z. Tian, C. Shen, X. Wang, and H. Chen, “Boxinst: High-performance instance segmentation with box annotations,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2021, pp. 5443–5452

    2021

  28. [36]

    Boxteacher: Exploring high-quality pseudo labels for weakly supervised instance seg- mentation,

    T. Cheng, X. Wang, S. Chen, Q. Zhang, and W. Liu, “Boxteacher: Exploring high-quality pseudo labels for weakly supervised instance seg- mentation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 3145–3154

    2023

  29. [37]

    Pointly-supervised instance seg- mentation,

    B. Cheng, O. Parkhi, and A. Kirillov, “Pointly-supervised instance seg- mentation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 2617–2626

    2022

  30. [38]

    Discobox: Weakly supervised instance segmentation and semantic correspondence from box supervision,

    S. Lan, Z. Yu, C. Choy, S. Radhakrishnan, G. Liu, Y . Zhu, L. S. Davis, and A. Anandkumar, “Discobox: Weakly supervised instance segmentation and semantic correspondence from box supervision,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp...

    2021

  31. [39]

    Beyond semantic to instance segmentation: Weakly-supervised instance segmentation via semantic knowledge transfer and self-refinement,

    B. Kim, Y . Yoo, C. E. Rhee, and J. Kim, “Beyond semantic to instance segmentation: Weakly-supervised instance segmentation via semantic knowledge transfer and self-refinement,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 4278–4287

    2022

  32. [40]

    Class-incremental continual learning for instance segmentation with image-level weak supervision,

    Y .-H. Hsieh, G.-S. Chen, S.-X. Cai, T.-Y . Wei, H.-F. Yang, and C.-S. Chen, “Class-incremental continual learning for instance segmentation with image-level weak supervision,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 1250–1261

    2023

  33. [41]

    Learning joint segmentation of tis- sues and brain lesions from task-specific hetero-modal domain-shifted datasets,

    R. Dorent, T. Booth, W. Li, C. H. Sudre, S. Kafiabadi, J. Cardoso, S. Ourselin, and T. Vercauteren, “Learning joint segmentation of tis- sues and brain lesions from task-specific hetero-modal domain-shifted datasets,” Medical image analysis , vol. 67, p. 101862, 2021

    2021

  34. [42]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 770–778

    2016

  35. [43]

    Swin transformer: Hierarchical vision transformer using shifted windows,

    Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 10 012–10 022

    2021

  36. [44]

    Panoptic feature pyramid networks,

    A. Kirillov, R. Girshick, K. He, and P. Doll ´ar, “Panoptic feature pyramid networks,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2019, pp. 6399–6408

    2019

  37. [45]

    Feature pyramid networks for object detection,

    T.-Y . Lin, P. Doll´ar, R. Girshick, K. He, B. Hariharan, and S. Belongie, “Feature pyramid networks for object detection,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2017, pp. 2117–2125

    2017

  38. [46]

    End-to-end object detection with transformers,

    N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko, “End-to-end object detection with transformers,” in European conference on computer vision . Springer, 2020, pp. 213– 229

    2020

  39. [47]

    Pointrend: Image segmen- tation as rendering,

    A. Kirillov, Y . Wu, K. He, and R. Girshick, “Pointrend: Image segmen- tation as rendering,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2020, pp. 9799–9808

    2020

  40. [48]

    Polite teacher: Semi-supervised instance segmentation with mutual learning and pseudo-label thresholding,

    D. Filipiak, A. Zapała, P. Tempczyk, A. Fensel, and M. Cygan, “Polite teacher: Semi-supervised instance segmentation with mutual learning and pseudo-label thresholding,” IEEE Access, 2024

    2024

  41. [49]

    Mean teachers are better role mod- els: Weight-averaged consistency targets improve semi-supervised deep learning results,

    A. Tarvainen and H. Valpola, “Mean teachers are better role mod- els: Weight-averaged consistency targets improve semi-supervised deep learning results,” Advances in neural information processing systems , vol. 30, 2017

    2017

  42. [50]

    Segment anything annotator,

    H. He, “Segment anything annotator,” https://github.com/ haochenheheda/segment-anything-annotator, 2024

    2024

  43. [51]

    Segment anything,

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg et al. , “Segment anything,” arXiv preprint, 2023

    2023

  44. [52]

    Dinov2: Learning robust visual features without supervision,

    M. Oquab, T. Darcet, T. Moutakanni, H. V o, M. Szafraniec, V . Khalidov, P. Fernandez, D. Haziza, F. Massa et al. , “Dinov2: Learning robust visual features without supervision,” Transactions on Machine Learning Research Journal, pp. 1–31, 2024

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.