Pith. sign in

REVIEW 3 major objections 6 minor 55 references

ZISVFM: Zero-Shot Object Instance Segmentation in Indoor Robotic Environments with Vision Foundation Models

T0 review · 3 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A training-free pipeline combining SAM, DINOv2, and K-Medoids point prompts can segment unseen objects in indoor scenes, matching or beating trained methods without any synthetic-data training.

desk verdict A solid training-free UOIS pipeline with a genuinely useful new dataset; the evaluation has test-set tuning that should be fixed before the numbers are taken at face value. read the letter →

arxiv 2502.03266 v1 pith:PAAKNKUP submitted 2025-02-05 cs.CV cs.RO

Ying Zhang , Maoliang Yin , Wenfu Bi , Haibao Yan , Shaohan Bian , Cui-Hua Zhang , Changchun Hua This is my paper · ORCID
classification cs.CVcs.RO
keywords unseenobjectinstancesegmentationzero-shotSegmentAnythingModelDINOv2RGB-DrobotperceptionK-Medoidspointpromptshierarchicalindoorscenessim-to-realgap
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that unseen-object instance segmentation in indoor robotics can be solved without any training, purely by composing two off-the-shelf vision foundation models. Its pipeline runs SAM on colorized depth images to propose object-agnostic masks, uses DINOv2's self-supervised attention to filter out background masks, and re-prompts SAM with K-Medoids cluster points for refined RGB segmentation. If this works as reported, service robots could segment and grasp unfamiliar objects in new, cluttered and hierarchical environments while avoiding the simulation-to-reality gap that limits synthetic-trained models. The authors report F-measures of 89.2 on OCID and 78.1 on OSD, competitive with trained state-of-the-art methods on OCID, and 90.5 on their new HIOD dataset, substantially above trained baselines.

What carries the argument

The load-bearing mechanism is the background-similarity filter built from DINOv2's final-layer attention: for each attention map the entropy $E(a_i)$ is computed, heads with lower entropy receive weight $\omega_i = -\log(E(a_i)/\sum_j E(a_j))$, and the patch with minimum weighted attention becomes the background reference; masks whose average cosine similarity to that reference in the weighted feature space exceeds a threshold $\tau$ are removed. The other two stages support it: SAM on viridis-colored depth produces the candidate masks, and K-Medoids clustering supplies three positive point prompts per proposal that let SAM re-segment each object from RGB.

What would settle it

Run ZISVFM on a densely cluttered scene with no visible background, such as a tabletop completely covered by objects or a filled drawer, and check whether recall collapses because the minimum-attention patch lands on an object and true masks are filtered out; comparing F-measure on such scenes against scenes with clear background would settle the role of this assumption.

Watch

Extended reading notes

Core claim

ZISVFM establishes that a training-free pipeline can perform UOIS competitively with methods trained on synthetic data. The key claim is that depth-first proposals plus self-supervised saliency filtering plus point-prompt refinement are jointly sufficient: viridis colorization makes geometric object boundaries legible to SAM, so its automatic masks are object-centric rather than texture-driven; entropy-weighted DINOv2 attention localizes salient objects and lets the pipeline identify the least-attended patch as background, removing masks whose mean cosine similarity to it exceeds a set threshold; and K-Medoids cluster centres inside the surviving proposals give SAM precise positive-point prompts that yield cleaner boundaries and a precision of 92.5% on OCID, 86.4% on OSD, and 91.3% on HIOD.

Load-bearing premise

Everything hinges on assuming the least-attended image patch is background and that masks resembling it are non-objects; in a scene where every patch lies on an object, that assumption can erase real foreground.

Editorial extensions

If this is right

  • Robots can segment unknown objects in new indoor scenes immediately, with no collection of synthetic or real training data for the target environment.
  • Improvements to SAM or to self-supervised ViTs should transfer directly, since the pipeline only composes their outputs and has no learned parameters of its own.
  • The method extends segmentation beyond single-plane tabletops: the reported HIOD results cover cabinets, drawers, sofa-desktop combinations, and handheld objects.
  • The full pipeline is robot-ready: the authors demonstrate grasping of unknown objects with a Fetch robot by feeding ZISVFM masks to a grasp-pose network.
  • The precision-recall trade-off implies the method is conservative: high precision but occasional missed small or low-attention objects, so it suits manipulation tasks where false positives are costlier than misses.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper does not test is temporal propagation: because prompts are just points, the K-Medoids points from one frame could seed the next frame, turning the method into a lightweight video object segmenter.
  • The entropy-weighting principle is general: any self-supervised attention model could be used in place of DINOv2, suggesting a plug-and-play filter that could improve other prompt-based segmentation pipelines.
  • Since the proposal stage relies on depth, the method likely inherits depth-sensor weaknesses; a testable extension is to fuse RGB-derived proposals as a fallback when depth is noisy or on reflective or transparent materials, which the paper's own failure cases hint at.
  • The single-background-patch assumption is fragile in fully cluttered scenes; replacing it with multiple reference patches and a clustered background model is a concrete variant worth testing.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes ZISVFM, a training-free three-stage pipeline for zero-shot object instance segmentation in indoor robotic environments. Stage 1 runs SAM on viridis-colorized depth images to obtain object-agnostic mask proposals; Stage 2 filters non-object proposals using DINOv2 attention features, weighting heads by information entropy, selecting the minimum-attention image patch as a background reference, and removing masks whose average cosine similarity to that patch exceeds a threshold τ; Stage 3 uses K-Medoids cluster centers within each remaining proposal as point prompts for SAM on the RGB image to refine boundaries. The method is evaluated on OCID, OSD, and a new self-collected HIOD dataset, and is demonstrated in a Fetch robot grasping experiment. The authors report superior performance over untrained baselines and comparable or better performance than trained SOTA methods on the new dataset.

Significance. If the claims hold, ZISVFM provides a practical training-free alternative to synthetic-data-trained UOIS models, with particular value in hierarchical scenes such as cabinets, drawers, and handheld objects. The paper's strengths include a clear modular design, public code, multiple ablation studies, and a real-robot demonstration. The principal weaknesses are the selection of the critical threshold τ on the test set, an unvalidated background-patch assumption that is load-bearing for the method, and the small self-collected dataset used for the main superiority claim. These concerns affect the interpretation of the reported quantitative results but are addressable with additional experiments and analysis.

major comments (3)
  1. [Section IV-E2, Fig. 4] The threshold τ is selected by sweeping on the OCID dataset, which is also the test benchmark in Tables I and III. This constitutes test-set tuning and makes the reported OCID numbers optimistic. The authors should evaluate with a pre-specified τ or use a validation split, and report results at the chosen value on held-out data.
  2. [Section III-D, Eqs. (3)-(5)] The method assumes that the least-attended patch l is background. In cluttered or hierarchical scenes, the minimum-attention patch can lie on an object, and the cosine-similarity filter in Eq. (5) would then label true foreground as background and remove legitimate masks. The failure cases in Section IV-F (unsegmented pen, stacked flat boxes) are consistent with this failure mode. The ablation in Fig. 4 only varies τ and does not test alternative background-reference selections; please validate the assumption (e.g., by measuring how often l falls on a ground-truth object in OCID or HIOD) or compare against alternative background-reference strategies.
  3. [Section IV-B and Table II] The HIOD dataset contains only 74 sets, and the paper does not specify how UOIS-Net-3D and MSMFormer+ were evaluated on it, including which checkpoints were used, whether the models were re-trained, and what evaluation protocol (matching, thresholds) was applied. Given the small dataset size, confidence intervals or per-scene breakdowns are needed to support the claim of significant superiority on HIOD. The provenance of the SOTA numbers in Table III (prior publications vs. re-runs) should also be stated explicitly.
minor comments (6)
  1. [Section III-D] The paragraph after Eq. (2) refers to attention maps indexed by i=1,...,Np; these should be indexed by the number of heads Nh, not the number of patches Np.
  2. [Section III-B] The section title 'The overall of proposed ZISFVM' contains a typo; it should read 'ZISVFM'.
  3. [Section III-E] The phrase 'are obtaind from' should be 'are obtained from'.
  4. [Eq. (4)] The variable F' is used without definition; it should be defined as the reshaped weighted feature matrix F_omega.
  5. [Section IV-A] The number of K-Medoids clusters is set to three; please justify this choice or specify how it is adapted for objects of very different sizes.
  6. [Section IV-B] The boundary F-measure is used but not defined in detail; a brief description of how boundaries are extracted and matched would improve reproducibility.

Circularity Check

1 steps flagged · score 4.0 of 10

Mostly self-contained derivation; the OCID comparison is partially fitted because the only tuned threshold τ is chosen on OCID and then evaluated on OCID.

  1. fitted input called prediction [Section IV.E.2 (τ ablation, Fig. 4), Section IV.A (Implementation Details), Table III]
    "Fig. 4 presents ablation studies on both object and boundary metrics, illustrating the model's sensitivity to variations in τ. These experiments, conducted on the OCID dataset without point prompts, reveal distinct performance patterns across different threshold ranges. ... The F-measure, which harmonically balances precision and recall, reaches its optimal value at τ ≈ 0.47."

    The implementation sets the background similarity threshold to τ = 0.47, which is exactly the value identified as optimal by the ablation on the OCID dataset. The same OCID dataset is then used in Table III to report ZISVFM's headline competitive F-measure (89.2 Overlap F, 87.0 F@.75). The OCID result is therefore not an independent prediction of the method's performance; it is partially the product of selecting the threshold on that same test set. This is a test-set model-selection loop for the OCID benchmark. It does not make the pipeline self-definitional, and it does not compromise the HIOD result, since τ was not tuned on HIOD; the central hierarchical-environment claim retains independent content.

full rationale

The core derivation chain is not circular. ZISVFM combines external, independently pre-trained components: SAM produces object-agnostic proposals from colorized depth, DINOv2 attention and features supply the background-similarity filter, and K-Medoids provides point prompts for SAM. Each stage consumes the previous stage's output rather than being defined in terms of the final ground-truth masks, and all quantitative results are compared against externally annotated masks (OCID, OSD, HIOD). No load-bearing step relies on a self-citation, and no uniqueness theorem is imported from prior work by the same authors. The only circularity-adjacent practice is the choice of τ: the ablation in Fig. 4 selects τ ≈ 0.47 by optimizing F-measure on OCID, and Table III then reports OCID performance using that same τ. This makes the OCID comparison partially fitted rather than purely predicted. The HIOD evaluation, which is the paper's distinguishing claim, uses the same threshold without having tuned it on HIOD, so it still provides independent evidence. The background-patch assumption (Eq. 3) is an untested heuristic in fully cluttered scenes, but it is an assumption about an external model's behavior, not a circular reduction of the target result.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the frozen behaviors of SAM and DINOv2 and on several heuristic modeling choices: the viridis depth colorization, the entropy weighting of attention heads, and the selection of the least-attended patch as background. The only parameter actually fitted to data in the paper is the background similarity threshold tau, chosen by test-set ablation. No new physical or learned entities are introduced.

free parameters (4)
  • tau (background similarity threshold) = 0.47
    Tuned on OCID test set by sweeping (Fig. 4); used for all reported results.
  • theta (IoU threshold in Algorithm 1) = 0.8
    Default value for union-mask removal; not optimized.
  • kmax (max combination size) = 3
    Default value in Algorithm 1; not optimized.
  • num_kmedoids_points = 3
    Number of cluster centers used as SAM point prompts; fixed by design.
assumptions (5)
  • domain assumption SAM generates object-agnostic mask proposals that contain an independent mask for each object when applied to viridis-colorized depth images.
    Stage one relies on this; the paper notes failures in cases of occlusion, stacking, and poor depth quality (Section IV-F).
  • domain assumption DINOv2 attention maps, weighted by information entropy, highlight foreground objects and allow reliable background identification.
    Stage two relies on this; the paper provides qualitative visualizations but no quantitative guarantee.
  • domain assumption The patch with minimum weighted attention is a valid background reference for all masks in the image.
    Section III-D, Eq. (3); if this patch lies on an object, background similarity filtering fails.
  • domain assumption Cosine similarity between weighted patch features and the background patch is a valid measure of backgroundness for entire masks.
    Section III-D, Eq. (4)-(5).
  • domain assumption Standard evaluation metrics and the Hungarian matching protocol from [51] are appropriate for UOIS comparison.
    Section IV-B; the protocol is standard in the field.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ZISVFM: Zero-Shot Object Instance Segmentation in Indoor Robotic Environments with Vision Foundation Models." pith.science (2026). https://pith.science/paper/PAAKNKUP

@misc{pith2026250203266,
  author       = {Pith},
  title        = {Pith review of: ZISVFM: Zero-Shot Object Instance Segmentation in Indoor Robotic Environments with Vision Foundation Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PAAKNKUP}},
  note         = {Machine review of arXiv:2502.03266}
}
read the original abstract

Service robots operating in unstructured environments must effectively recognize and segment unknown objects to enhance their functionality. Traditional supervised learningbased segmentation techniques require extensive annotated datasets, which are impractical for the diversity of objects encountered in real-world scenarios. Unseen Object Instance Segmentation (UOIS) methods aim to address this by training models on synthetic data to generalize to novel objects, but they often suffer from the simulation-to-reality gap. This paper proposes a novel approach (ZISVFM) for solving UOIS by leveraging the powerful zero-shot capability of the segment anything model (SAM) and explicit visual representations from a selfsupervised vision transformer (ViT). The proposed framework operates in three stages: (1) generating object-agnostic mask proposals from colorized depth images using SAM, (2) refining these proposals using attention-based features from the selfsupervised ViT to filter non-object masks, and (3) applying K-Medoids clustering to generate point prompts that guide SAM towards precise object segmentation. Experimental validation on two benchmark datasets and a self-collected dataset demonstrates the superior performance of ZISVFM in complex environments, including hierarchical settings such as cabinets, drawers, and handheld objects. Our source code is available at https://github.com/Yinmlmaoliang/zisvfm.

Figures

Figures reproduced from arXiv: 2502.03266 by the authors.

Figure 1
Figure 1. Overview of the proposed ZISVFM methodology. This approach employs two vision foundation models: SAM [19] for segmentation and ViT trained [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Visualization of self-attention maps obtained with the six different [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Comparison of ZISVFM with baseline, SOTA methods on OCID and HIOD datasets. The baseline method, SAM, utilized RGB and depth images [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: Common failure cases of our proposed method in OCID [47] and OSD [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: Demonstrating manipulation of unknown objects in the cluttered [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

55 extracted references · 43 canonical work pages

  1. [1]

    Unseen object instance segmentation for robotic environments,

    C. Xie, Y . Xiang, A. Mousavian, and D. Fox, “Unseen object instance segmentation for robotic environments,” IEEE Transactions on Robotics, vol. 37, no. 5, pp. 1343–1359, 2021

    work page 2021

  2. [2]

    Taylor neural network for unseen object instance segmentation in hierarchical grasping,

    K. Fu, X. Dang, and Y . Zhang, “Taylor neural network for unseen object instance segmentation in hierarchical grasping,” IEEE/ASME Transactions on Mechatronics, 2024, to be published, DOI: 10.1109/TMECH.2023.3347558

    work page arXiv 2024

  3. [3]

    Segmentation of unknown objects in indoor environments,

    A. Richtsfeld, T. M ¨orwald, J. Prankl, M. Zillich, and M. Vincze, “Segmentation of unknown objects in indoor environments,” in 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems , 2012, pp. 4791–4796

    work page 2012

  4. [4]

    Learn fast, segment well: Fast object segmentation learning on the icub robot,

    F. Ceola, E. Maiettini, G. Pasquale, G. Meanti, L. Rosasco, and L. Na- tale, “Learn fast, segment well: Fast object segmentation learning on the icub robot,” IEEE Transactions on Robotics , vol. 38, no. 5, pp. 3154– 3172, 2022

    work page 2022

  5. [5]

    Learning rgb-d feature embeddings for unseen object instance segmentation,

    Y . Xiang, C. Xie, A. Mousavian, and D. Fox, “Learning rgb-d feature embeddings for unseen object instance segmentation,” in Conference on Robot Learning , 2021, pp. 461–470

    work page 2021

  6. [6]

    Stow: Discrete-frame segmentation and tracking of unseen objects for warehouse picking robots,

    Y . Li, M. Zhang, M. Grotz, K. Mo, and D. Fox, “Stow: Discrete-frame segmentation and tracking of unseen objects for warehouse picking robots,” in Proceedings of the Conference on Robot Learning (CoRL) , 2023

    work page 2023

  7. [7]

    Mask r-cnn,

    K. He, G. Gkioxari, P. Doll ´ar, and R. Girshick, “Mask r-cnn,” in Proceedings of the IEEE International Conference on Computer Vision , 2017, pp. 2961–2969

    work page 2017

  8. [8]

    Image segmentation using deep learning: A survey,

    S. Minaee, Y . Boykov, F. Porikli, A. Plaza, N. Kehtarnavaz, and D. Ter- zopoulos, “Image segmentation using deep learning: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 44, no. 7, pp. 3523–3542, 2022

    work page 2022

Show all 55 references
  1. [9]

    Imagenet large scale visual recognition challenge,

    O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein et al., “Imagenet large scale visual recognition challenge,” International Journal of Computer Vision, vol. 115, pp. 211–252, 2015

    2015

  2. [10]

    Microsoft coco: Common objects in context,

    T.-Y . Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Doll ´ar, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in European Conference Computer Vision , 2014, pp. 740–755

    2014

  3. [11]

    Unseen object amodal instance segmentation via hierarchical occlusion mod- eling,

    S. Back, J. Lee, T. Kim, S. Noh, R. Kang, S. Bak, and K. Lee, “Unseen object amodal instance segmentation via hierarchical occlusion mod- eling,” in 2022 International Conference on Robotics and Automation (ICRA), 2022, pp. 5085–5092

    2022

  4. [12]

    Unknown object segmentation from stereo images,

    M. Durner, W. Boerdijk, M. Sundermeyer, W. Friedl, Z.-C. M ´arton, and R. Triebel, “Unknown object segmentation from stereo images,” in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2021, pp. 4823–4830

    2021

  5. [13]

    The best of both modes: Separately leveraging rgb and depth for unseen object instance segmentation,

    C. Xie, Y . Xiang, A. Mousavian, and D. Fox, “The best of both modes: Separately leveraging rgb and depth for unseen object instance segmentation,” in Conference on Robot Learning , 2020, pp. 1369–1378

    2020

  6. [14]

    Segmenting unknown 3d objects from real depth images using mask r-cnn trained on synthetic data,

    M. Danielczuk, M. Matl, S. Gupta, A. Li, A. Lee, J. Mahler, and K. Goldberg, “Segmenting unknown 3d objects from real depth images using mask r-cnn trained on synthetic data,” in 2019 International Conference on Robotics and Automation (ICRA) , 2019, pp. 7283–7290

    2019

  7. [15]

    Unseen object instance segmentation with fully test-time rgb-d embeddings adaptation,

    L. Zhang, S. Zhang, X. Yang, H. Qiao, and Z. Liu, “Unseen object instance segmentation with fully test-time rgb-d embeddings adaptation,” in 2023 IEEE International Conference on Robotics and Automation (ICRA), 2023, pp. 4945–4952

    2023

  8. [16]

    Self-supervised unseen object instance segmentation via long-term robot interaction,

    Y . Lu, N. Khargonkar, Z. Xu, C. Averill, K. Palanisamy, K. Hang, Y . Guo, N. Ruozzi, and Y . Xiang, “Self-supervised unseen object instance segmentation via long-term robot interaction,” arXiv preprint arXiv:2302.03793, 2023

    2023 arXiv

  9. [17]

    Self-supervised interactive object segmentation through a singulation-and-grasping approach,

    H. Yu and C. Choi, “Self-supervised interactive object segmentation through a singulation-and-grasping approach,” in European Conference on Computer Vision , 2022, pp. 621–637. THIS ARTICLE HAS BEEN ACCEPTED FOR PUBLICATION IN A FUTURE ISSUE OF IEEE TRANSACTIONS ON ROBOTICS . 13

    2022

  10. [18]

    Self-supervised transfer learning for instance segmentation through physical interaction,

    A. Eitel, N. Hauff, and W. Burgard, “Self-supervised transfer learning for instance segmentation through physical interaction,” in 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , 2019, pp. 4020–4026

    2019

  11. [19]

    Segment anything,

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Loet al., “Segment anything,” arXiv preprint arXiv:2304.02643 , 2023

    2023 arXiv

  12. [20]

    Seggpt: Segmenting everything in context,

    X. Wang, X. Zhang, Y . Cao, W. Wang, C. Shen, and T. Huang, “Seggpt: Segmenting everything in context,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , 2023, pp. 1130– 1140

    2023

  13. [21]

    Semantic segment anything,

    J. Chen, Z. Yang, and L. Zhang, “Semantic segment anything,” https: //github.com/fudan-zvg/Semantic-Segment-Anything, 2023

    2023

  14. [22]

    Segment everything everywhere all at once,

    X. Zou, J. Yang, H. Zhang, F. Li, L. Li, J. Gao, and Y . J. Lee, “Segment everything everywhere all at once,” in Proceedings of the 37th International Conference on Neural Information Processing Systems , 2023, pp. 19 769–19 782

    2023

  15. [23]

    Matplotlib: A 2d graphics environment,

    J. D. Hunter, “Matplotlib: A 2d graphics environment,” Computing in science & engineering , vol. 9, no. 03, pp. 90–95, 2007

    2007

  16. [24]

    Dinov2: Learning robust visual features without supervision,

    M. Oquab, T. Darcet, T. Moutakanni, H. V o, M. Szafraniec, V . Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby et al. , “Dinov2: Learning robust visual features without supervision,” Transactions on Machine Learning Research , 2024

    2024

  17. [25]

    Rice: Refining instance masks in cluttered environments with graph neural networks,

    C. Xie, A. Mousavian, Y . Xiang, and D. Fox, “Rice: Refining instance masks in cluttered environments with graph neural networks,” in Con- ference on Robot Learning , 2022, pp. 1655–1665

    2022

  18. [26]

    3d model-based zero-shot pose estimation pipeline,

    J. Chen, M. Sun, T. Bao, R. Zhao, L. Wu, and Z. He, “3d model-based zero-shot pose estimation pipeline,” arXiv preprint arXiv:2305.17934 , 2023

    2023 arXiv

  19. [27]

    Cnos: A strong baseline for cad-based novel object segmentation,

    V . N. Nguyen, T. Groueix, G. Ponimatkin, V . Lepetit, and T. Hodan, “Cnos: A strong baseline for cad-based novel object segmentation,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 2134–2140

    2023

  20. [28]

    Panoptic seg- mentation,

    A. Kirillov, K. He, R. Girshick, C. Rother, and P. Doll ´ar, “Panoptic seg- mentation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Pecognition , 2019, pp. 9404–9413

    2019

  21. [29]

    Yolact: Real-time instance segmentation,

    D. Bolya, C. Zhou, F. Xiao, and Y . J. Lee, “Yolact: Real-time instance segmentation,” in Proceedings of the IEEE/CVF International Confer- ence on Computer Vision , 2019, pp. 9157–9166

    2019

  22. [30]

    Fully convolutional networks for semantic segmentation,

    J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2015, pp. 3431–3440

    2015

  23. [31]

    Foundational models defining a new era in vision: A survey and outlook,

    M. Awais, M. Naseer, S. Khan, R. M. Anwer, H. Cholakkal, M. Shah, M.-H. Yang, and F. S. Khan, “Foundational models defining a new era in vision: A survey and outlook,” arXiv preprint arXiv:2307.13721 , 2023

    2023 arXiv

  24. [32]

    Accuracy of segment- anything model (sam) in medical image segmentation tasks,

    S. He, R. Bao, J. Li, P. E. Grant, and Y . Ou, “Accuracy of segment- anything model (sam) in medical image segmentation tasks,” arXiv preprint arXiv:2304.09324, 2023

    2023 arXiv

  25. [33]

    Segment and track anything,

    Y . Cheng, L. Li, Y . Xu, X. Li, Z. Yang, W. Wang, and Y . Yang, “Segment and track anything,” arXiv preprint arXiv:2305.06558 , 2023

    2023 arXiv

  26. [34]

    Inpaint anything: Segment anything meets image inpainting,

    T. Yu, R. Feng, R. Feng, J. Liu, X. Jin, W. Zeng, and Z. Chen, “Inpaint anything: Segment anything meets image inpainting,” arXiv preprint arXiv:2304.06790, 2023

    2023 arXiv

  27. [35]

    All-in-sam: from weak annotation to pixel-wise nuclei segmentation with prompt-based finetuning,

    C. Cui, R. Deng, Q. Liu, T. Yao, S. Bao, L. W. Remedios, Y . Tang, and Y . Huo, “All-in-sam: from weak annotation to pixel-wise nuclei segmentation with prompt-based finetuning,” Journal of Physics: Con- ference Series, vol. 2722, no. 1, p. 012012, 2024

    2024

  28. [36]

    Uvosam: A mask- free paradigm for unsupervised video object segmentation via segment anything model,

    Z. Zhang, Z. Wei, S. Zhang, Z. Dai, and S. Zhu, “Uvosam: A mask- free paradigm for unsupervised video object segmentation via segment anything model,” arXiv preprint arXiv:2305.12659 , 2023

    2023 arXiv

  29. [37]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017, pp. 6000–6010

    2017

  30. [38]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly et al. , “An image is worth 16x16 words: Transformers for image recognition at scale,” in 9th International Conference on Learning Representations , 2021

    2021

  31. [39]

    Cross-level multi-modal features learning with transformer for rgb-d object recognition,

    Y . Zhang, M. Yin, H. Wang, and C. Hua, “Cross-level multi-modal features learning with transformer for rgb-d object recognition,” IEEE Transactions on Circuits and Systems for Video Technology , vol. 33, no. 12, pp. 7121–7130, 2023

    2023

  32. [40]

    An empirical study of training self- supervised vision transformers,

    X. Chen, S. Xie, and K. He, “An empirical study of training self- supervised vision transformers,” in Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision (ICCV) , 2021, pp. 9640–9649

    2021

  33. [41]

    Emerging properties in self-supervised vision transformers,

    M. Caron, H. Touvron, I. Misra, H. J ´egou, J. Mairal, P. Bojanowski, and A. Joulin, “Emerging properties in self-supervised vision transformers,” in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 9650–9660

    2021

  34. [42]

    Mst: Masked self-supervised transformer for visual representation,

    Z. Li, Z. Chen, F. Yang, W. Li, Y . Zhu, C. Zhao, R. Deng, L. Wu, R. Zhao, M. Tang et al. , “Mst: Masked self-supervised transformer for visual representation,” Advances in Neural Information Processing Systems, vol. 34, pp. 13 165–13 176, 2021

    2021

  35. [43]

    Bert: Pre-training of deep bidirectional transformers for language understanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologi...

    2019

  36. [44]

    Beit: Bert pre-training of image transformers,

    H. Bao, L. Dong, S. Piao, and F. Wei, “Beit: Bert pre-training of image transformers,” in The International Conference on Learning Representations (ICLR), 2022

    2022

  37. [45]

    Masked au- toencoders are scalable vision learners,

    K. He, X. Chen, S. Xie, Y . Li, P. Doll ´ar, and R. Girshick, “Masked au- toencoders are scalable vision learners,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 16 000–16 009

    2022

  38. [46]

    Deep vit features as dense visual descriptors,

    S. Amir, Y . Gandelsman, S. Bagon, and T. Dekel, “Deep vit features as dense visual descriptors,” in European Conference on Computer Vision , vol. 2, no. 3, 2022, p. 4

    2022

  39. [47]

    Easylabel: A semi- automatic pixel-wise object annotation tool for creating robotic rgb-d datasets,

    M. Suchi, T. Patten, D. Fischinger, and M. Vincze, “Easylabel: A semi- automatic pixel-wise object annotation tool for creating robotic rgb-d datasets,” in 2019 International Conference on Robotics and Automation (ICRA), 2019, pp. 6678–6684

    2019

  40. [48]

    A simple and fast algorithm for k-medoids clustering,

    H.-S. Park and C.-H. Jun, “A simple and fast algorithm for k-medoids clustering,” Expert Systems With Applications , vol. 36, no. 2, pp. 3336– 3341, 2009

    2009

  41. [49]

    Vision transformers need registers,

    T. Darcet, M. Oquab, J. Mairal, and P. Bojanowski, “Vision transformers need registers,” in The Twelfth International Conference on Learning Representations (ICLR), 2024

    2024

  42. [50]

    Pytorch: An imperative style, high-performance deep learning library,

    A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga et al. , “Pytorch: An imperative style, high-performance deep learning library,” Advances in neural information processing systems , vol. 32, 2019

    2019

  43. [51]

    Towards segmenting anything that moves,

    A. Dave, P. Tokmakov, and D. Ramanan, “Towards segmenting anything that moves,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops , Oct 2019

    2019

  44. [52]

    Mean shift mask transformer for unseen object instance segmentation,

    Y . Lu, Y . Chen, N. Ruozzi, and Y . Xiang, “Mean shift mask transformer for unseen object instance segmentation,” arXiv preprint arXiv:2211.11679, 2022

    2022 arXiv

  45. [53]

    Contact- graspnet: Efficient 6-dof grasp generation in cluttered scenes,

    M. Sundermeyer, A. Mousavian, R. Triebel, and D. Fox, “Contact- graspnet: Efficient 6-dof grasp generation in cluttered scenes,” in 2021 IEEE International Conference on Robotics and Automation (ICRA) , 2021, pp. 13 438–13 444

    2021

  46. [54]

    Safe and efficient robot manipulation: Task-oriented environment modeling and object pose estimation,

    Y . Zhang, G. Tian, and X. Shao, “Safe and efficient robot manipulation: Task-oriented environment modeling and object pose estimation,” IEEE Transactions on Instrumentation and Measurement , vol. 70, pp. 1–12, 2021

    2021

  47. [55]

    Moveit![ros topics],

    S. Chitta, I. Sucan, and S. Cousins, “Moveit![ros topics],” IEEE Robotics & Automation Magazine , vol. 19, no. 1, pp. 18–19, 2012. Ying Zhang (Senior Member, IEEE) received the Ph.D. degree in Control Theory and Control Engi- neering from Shandong University, Jinan, China, in ...

    2012

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.