Pith. sign in

REVIEW 3 major objections 5 minor 2 cited by

LoftUp: Learning a Coordinate-Based Feature Upsampler for Vision Foundation Models

T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A coordinate-based cross-attention upsampler trained with self-distilled pseudo-groundtruth recovers full-resolution features from vision foundation models, yielding 10–20% gains across dense prediction tasks and nearly 50% on video…

desk verdict A solid, useful feature upsampler with a clear typo and an under-specified pseudo-GT; deserves review, needs a few targeted fixes. read the letter →

arxiv 2504.14032 v1 pith:6GCAB457 submitted 2025-04-18 cs.CV cs.AIcs.LGeess.IV

Haiwen Huang , Anpei Chen , Volodymyr Havrylov , Andreas Geiger , Dan Zhang This is my paper · ORCID
classification cs.CVcs.AIcs.LGeess.IV
keywords featureupsamplingvisionfoundationmodelsDINOv2coordinate-basedtransformercross-attentionself-distillationpseudo-groundtruthdenseprediction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the coarse feature maps produced by vision foundation models can be upgraded to full-resolution, pixel-level features by a task-agnostic upsampler, and that two design choices matter most: the upsampler architecture and the training signal. The proposed upsampler, LoftUp, is a coordinate-based cross-attention transformer: each high-resolution pixel queries the low-resolution VFM feature map using its coordinates and RGB value, so details are synthesized with global semantic context rather than local kernels. Because no full-resolution feature ground truth exists, the paper constructs pseudo-groundtruth at image resolution by refining bicubic-upsampled features with class-agnostic masks and then self-distilling from a teacher upsampler that sees larger crops. Across semantic segmentation, depth and normal estimation, video object segmentation, and open-vocabulary and interactive segmentation, LoftUp reports consistent gains over prior upsamplers, with the largest on video object segmentation. If correct, this makes high-resolution dense prediction from frozen vision foundation models cheaper and more accurate than feeding larger images or training task-specific decoders.

What carries the argument

The load-bearing object is the coordinate-based cross-attention transformer. High-resolution pixel coordinates are encoded with sinusoidal positional encodings, concatenated with RGB values, projected by a 3×3 convolution, and used as queries in cross-attention blocks whose keys and values are the low-resolution VFM features, replacing fixed or locally predicted kernels with global content-aware attention. The training machinery is a two-stage pseudo-groundtruth construction: Stage 1 blends bicubic-upsampled features with per-mask mean features computed from class-agnostic SAM masks and trains with an L2 loss; Stage 2 runs a teacher upsampler on larger image crops, refines those features with the same mask blending, and distills them into the student with an affinity-matrix loss, with the teacher updated by exponential moving average. Together these ingredients supply full-resolution supervision that previous upsamplers lacked.

What would settle it

Run LoftUp on a benchmark where full-resolution features can be obtained as oracle ground truth, such as a vision model that natively outputs dense features at input resolution, and compare linear-probe segmentation or depth mIoU of the oracle features versus LoftUp's upsampled features under identical decoders. If LoftUp does not approach the oracle's performance, or if a LoftUp variant trained directly on oracle features substantially outperforms the pseudo-GT-trained version, then the pseudo-GT training objective is the bottleneck rather than the upsampler architecture.

Watch

Extended reading notes

Core claim

The paper's central claim is that the resolution bottleneck of vision foundation models is not fixed by the backbone but can be removed by a lightweight, task-agnostic upsampler, provided the upsampler uses global coordinate-based cross-attention and is trained against full-resolution pseudo-groundtruth features. Previous task-agnostic upsamplers either optimized per image at test time or supervised only at low resolution; LoftUp instead learns a direct coordinate-to-feature mapping and supervises it at image resolution, using SAM masks to enforce sharp boundaries and a self-distillation stage where a teacher upsampler processing high-resolution crops produces targets for the student. On the DINOv2-S/14 backbone with linear or lightweight decoders, the paper reports relative gains of 10–20% over the best previous upsamplers on most of six tasks, and nearly 50% on video object segmentation, along with consistent gains when the backbone is replaced by CLIP or RADIO.

Load-bearing premise

The load-bearing premise is that the pseudo-groundtruth features built from SAM-mask-smoothed bicubic features and from the teacher upsampler's high-resolution-crop outputs are accurate enough proxies for true full-resolution VFM features that training against them transfers to downstream tasks; if that surrogate is wrong, the reported downstream gains would not reflect genuine recovery of fine detail.

Editorial extensions

If this is right

  • Task-agnostic feature upsampling becomes a plug-and-play enhancement: any frozen vision foundation model (DINOv2, CLIP, RADIO) can be paired with LoftUp and a lightweight decoder instead of retraining a task-specific head on high-resolution inputs.
  • Because LoftUp maps coordinates directly, one trained model can produce features at arbitrary upsampling scales, and the paper shows it can beat a VFM baseline that processes 2× larger images at roughly four times the computational cost.
  • Boundary-sensitive tasks benefit most: video object segmentation, where contours matter, shows the largest gain, with the J&F score rising from 36.66 to 60.25.
  • The pseudo-groundtruth itself is reusable: the paper shows it improves other upsampler architectures such as resize-conv and FeatUp-JBU, not just LoftUp.
  • With less than 20% parameter overhead over the backbone and inference time comparable to bilinear upsampling, the method is practical for deployment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the pseudo-groundtruth is constructed without task labels, the same Stage 1 plus Stage 2 recipe could plausibly be applied to other dense feature targets, such as distilling finer features from a stronger vision model or from multi-frame video, without changing the upsampler.
  • The cross-attention query design suggests a natural extension to non-image modalities with coordinate-like structure, such as point clouds or voxel grids, where low-resolution semantic tokens are queried by high-resolution geometric coordinates.
  • The large video-object-segmentation gain hints that much of the improvement comes from boundary sharpness rather than global semantics; a testable prediction is that LoftUp's advantage over baselines will correlate with the boundary density of the evaluation dataset.
  • If the 10–20% gains hold broadly, they imply that frozen VFM features encode far more spatial detail than their native resolution exposes, meaning the sparse token grid is a representational bottleneck that can be decoded with the right point-wise queries.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes LoftUp, a coordinate-based cross-attention transformer that upsamples vision foundation model (VFM) features to full input resolution. The training pipeline has two stages: Stage 1 constructs pseudo-groundtruth features by bicubic upsampling and then refining them with class-agnostic SAM masks; Stage 2 uses self-distillation, where an EMA teacher upsampler, initialized from the Stage 1 model, processes high-resolution crops and supervises the student. The authors evaluate LoftUp on semantic segmentation, depth and normal estimation, video object segmentation, open-vocabulary segmentation, and interactive segmentation using DINOv2, with additional CLIP and RADIO experiments in the supplement. They report consistent improvements over bilinear upsampling, FeatUp, and LiFT, and argue that the coordinate-based architecture supports arbitrary upsampling scales at low computational overhead. Code is released.

Significance. If the central claims hold, LoftUp would be a practical, task-agnostic module for improving dense prediction from frozen VFMs, combining a cross-attention upsampler with a pseudo-GT training objective that avoids per-image optimization. The paper's strengths are its breadth of downstream evaluation, the architecture ablations in Table B.3, the inclusion of multiple VFM backbones in the supplement, and the release of code. The consistency of the reported gains across diverse tasks and backbones makes the core direction credible. However, the load-bearing pseudo-GT construction is under-specified as printed (Eq. (1) is self-cancelling), the Stage 2 target is bootstrapped from the student's own upsampler rather than an independent source of high-resolution feature evidence, and the LiFT comparison is conducted at an upsampling factor far outside LiFT's design range. These issues do not invalidate the approach, but they must be addressed before the paper's central claims can be considered established.

major comments (3)
  1. [Sec. 4.1, Eq. (1)] Equation (1) as printed defines FMask-Bicubic[m] as alpha * FBicubic[m] + (1-alpha) * FBicubic[m]; since both terms are identical, alpha cancels and the mask refinement has no effect. The surrounding text describes blending the mean feature inside each mask with the original features, so the equation is evidently missing a mean symbol or an analogous correction. This is a load-bearing issue because Eq. (1) defines the Stage 1 training target, which in turn initializes the Stage 2 teacher and student. Please replace the formula with the intended expression (e.g., alpha * mean(FBicubic[m]) + (1-alpha) * FBicubic) and update the text in Sec. 4.1 and the supplementary details accordingly.
  2. [Sec. 4.2, Stage 2 self-distillation] The Stage 2 target is not independent: the teacher is initialized from the Stage 1 student, updated as an EMA of the same student, and consumes the same frozen VFM, so the only external information in the entire pseudo-GT pipeline is the class-agnostic SAM mask prior. Consequently, the claim that LoftUp 'captures fine-grained details' beyond what mask-based boundary sharpening provides is not yet supported. I request targeted experiments: (i) compare against an oracle target obtained from genuinely high-resolution VFM features (e.g., features of a larger input image) on a holdout set; (ii) measure whether the learned upsampler improves similarity to such oracle features relative to bilinear and FeatUp; (iii) ablate Stage 2 by fixing the teacher after Stage 1 instead of EMA-updating it, and by turning the mask refinement on the teacher output on and off. Without such evidence, the observed downstream gains could be attributed to the mask prior and smoothing rather than to recovery of true high-frequency VFM features.
  3. [Sec. 5.1, Tables 3 and 4] LiFT is described in Sec. 4 as a method whose training objective uses 2x larger inputs, and previous work reports LiFT for 2x upsampling. Yet Tables 3 and 4 evaluate LiFT at 14x upsampling (and 8x for open-vocabulary segmentation). This is an unfair comparison when claiming 'previous SoTA' performance: a method trained for a 2x target cannot be expected to work well at 14x. Please either train LiFT for the target scale, evaluate it at its native 2x scale and discuss the mismatch, or clearly state that the comparison is out-of-distribution for LiFT. Additionally, Tables 3 and 4 report single runs without error bars or multiple seeds; given the small differences in some rows (e.g., depth and normal estimation in Table 3), please report mean and standard deviation over at least three seeds for the main tables.
minor comments (5)
  1. [Introduction and References] The Introduction contains the stray token 'schannen2025siglip2' and references [54] and [55] are duplicated for SigLIP 2; the reference list and in-text citations should be cleaned up.
  2. [Sec. 4.2, notation] The sentence describing the self-distillation loss states that the teacher output is downsampled to match crop(fstudent(I)) in R^{H/t x W/t}, which is dimensionally confusing because fstudent(I) is in R^{H x W}. Please define explicitly that the student output is spatially cropped to the region corresponding to the teacher's crop before the comparison.
  3. [Table 7] The parameter counts in Table 7 are reported in a way that is hard to interpret: bilinear is listed with 22.1M parameters, and FeatUp-Implicit's inference time is written as 54.302 without an explicit unit. Please state whether the VFM backbone parameters are included and format the timing column consistently.
  4. [Throughout] There are typographical errors such as 'illsustrated' (Sec. 3), 'segementation' (Table 4 caption), and 'pseduo-GT' (supplement); a careful proofread is needed.
  5. [Fig. 5 caption] The caption states 'We set alpha = 0.8 (in Eq. (1))', but Eq. (1) as printed has no effect of alpha; this cross-reference should be updated once Eq. (1) is corrected.

Circularity Check

1 steps flagged · score 4.0 of 10

One self-definitional tautology in the Stage-1 pseudo-GT equation, but the paper's external downstream benchmarks keep the central claim independently grounded.

  1. self definitional [Section 4.1, Eq. (1); reused for the teacher in Section 4.2]
    "we compute the mean feature, FBicubic[m], and blend it with the original features, yielding a mask-refined feature map at pixels within maskm: FMask-Bicubic[m] = α∗FBicubic[m]+(1−α)∗FBicubic[m], (1) whereα∈ [0, 1] controls the degree of mask refinement."

    As printed, the two terms on the right-hand side are written with the same symbol, so the equation reduces to FMask-Bicubic[m] = FBicubic[m] for every value of α. The claimed mask refinement therefore cancels identically by construction: the Stage-1 pseudo-groundtruth is definitionally equal to the bicubic-upsampled features that Stage 1 was supposed to improve. Any statement that class-agnostic masks add sharp boundary information is unsupported by this equation as written, and the same formal defect is carried into Stage 2 when the paper applies 'the same mask refinement described in Eq. (1)' to the teacher's features.

full rationale

The paper's headline claims are validated on external downstream benchmarks (COCO-Stuff, Cityscapes, NYU/NAVI/DAVIS, etc.) using task heads trained on those datasets, and those measured numbers are not algebraic functions of the training target; so the central empirical claim is not circular. The self-distillation stage is transparently described as an EMA teacher initialized from the Stage-1 model and operating on higher-resolution crops, which is a bootstrapping technique rather than an equation-level reduction to the student's inputs. The concrete circularity is confined to Eq. (1), where the mask-refined pseudo-GT is, as printed, identical to its bicubic input by algebraic cancellation. This is a real formal defect in the derivation of the training objective, but it does not by itself force the downstream results, because the evaluation is external. The score of 4 reflects one partial self-definitional step in the training-signal chain while the main performance claims retain independent empirical content.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The method adds no new physical entities. Its performance rests on hand-set hyperparameters (alpha, EMA decay, block count, crop scale, dataset size, loss choice) and on assumptions about pseudo-GT validity and generalization. None of these parameters are fitted to downstream test sets.

free parameters (6)
  • alpha (mask refinement weight) = 0.8
    Balances boundary sharpness from SAM masks and fine detail from high-resolution features in pseudo-GT construction (Eq. 1). Chosen by hand, not tuned on downstream test sets.
  • EMA decay for teacher update = 0.99
    Stabilizes teacher in self-distillation; teacher updated every 10 steps. Value selected empirically.
  • number of cross-attention blocks L = 2
    Ablation in Table B.3 shows 2 blocks sufficient; deeper models saturate.
  • teacher crop scale t = 2 to 4
    Trade-off between teacher detail and student supervision resolution; chosen as a working range.
  • training dataset size = 1M images from SA1B
    Larger training data improves results with diminishing returns; 1M selected as balance (Table B.3).
  • loss function for self-distillation = affinity matrix loss
    Reported to consistently outperform L2 loss (Sec. 4.2).
assumptions (5)
  • domain assumption Low-resolution VFM features plus image RGB and coordinates contain sufficient information to reconstruct high-resolution features.
    Core premise of feature upsampling; invoked throughout Sec. 3.
  • domain assumption SAM class-agnostic masks provide accurate full-resolution boundary information that improves pseudo-GT.
    Used to construct Mask-Bicubic pseudo-GT in Stage 1, Eq. (1).
  • domain assumption Self-distillation from a teacher processing higher-resolution crops transfers fine detail to the student.
    Stage 2 training relies on this assumption, Sec. 4.2.
  • domain assumption The lightweight downstream decoders (linear layer, DPT head, affinity tracking) are sensitive enough to reflect feature quality differences.
    Evaluation protocol assumes downstream metrics track feature quality, Sec. 5.1.
  • domain assumption Training on a 1M-image SA1B subset generalizes to COCO, Cityscapes, DAVIS, GrabCut, etc.
    All upsamplers trained only on SA1B and evaluated on other datasets; generalization is assumed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LoftUp: Learning a Coordinate-Based Feature Upsampler for Vision Foundation Models." pith.science (2026). https://pith.science/paper/6GCAB457

@misc{pith2026250414032,
  author       = {Pith},
  title        = {Pith review of: LoftUp: Learning a Coordinate-Based Feature Upsampler for Vision Foundation Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6GCAB457}},
  note         = {Machine review of arXiv:2504.14032}
}
read the original abstract

Vision foundation models (VFMs) such as DINOv2 and CLIP have achieved impressive results on various downstream tasks, but their limited feature resolution hampers performance in applications requiring pixel-level understanding. Feature upsampling offers a promising direction to address this challenge. In this work, we identify two critical factors for enhancing feature upsampling: the upsampler architecture and the training objective. For the upsampler architecture, we introduce a coordinate-based cross-attention transformer that integrates the high-resolution images with coordinates and low-resolution VFM features to generate sharp, high-quality features. For the training objective, we propose constructing high-resolution pseudo-groundtruth features by leveraging class-agnostic masks and self-distillation. Our approach effectively captures fine-grained details and adapts flexibly to various input and feature resolutions. Through experiments, we demonstrate that our approach significantly outperforms existing feature upsampling techniques across various downstream tasks. Our code is released at https://github.com/andrehuang/loftup.

Figures

Figures reproduced from arXiv: 2504.14032 by the authors.

Figure 1
Figure 1. LoftUp improves significantly across various tasks over the VFM backbone (DINOv2-S [37]) and current SoTA feature upsampling performance (FeatUp [12] and LiFT [52]). See experiment details in Sec. 5. training or fine-tuning VFMs on high-resolution images de￾mands substantial computational resources and meticulous tuning [37, 43, 45, 55]. An alternative strategy is to train task-specific decoders that leverage multi-… view at source ↗
Figure 2
Figure 2. Comparison of features from upsamplers. Backbone is DINOv2-S/14 [37]. In this work, we systematically explore the design space of feature upsamplers and identify two critical components: the upsampler architecture and the training objective. The architecture determines the capacity of the upsampler to learn effectively, while the training objective defines the up￾per performance limit. By optimizing both elements, o… view at source ↗
Figure 3
Figure 3. Architecture of LoftUp. Our coordinate-based net￾work with cross-attention mechanism effectively integrates the fine-grained details from image RGB values and semantically-rich low-res features to produce high-resolution feature maps. locks the potential of the feature upsampler to capture fine￾grained details in high-resolution images. 3. Coordinate-Based Feature Representation Prior works typically address feature… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Our two-stage LoftUp training approach. Stage 1 trains an upsampler with class-agnostic masks to refine bicubic-upsampled features. Stage 2 employs self-distillation, initializing teacher and student upsamplers from Stage 1’s pre-trained model. All VFM image inputs sha…
Figure 5
Figure 5. Figure 5: Visualization of different pseudo-GT. Both Mask-Bicubic and Self-Distilled are proposed by our work. We set α = 0.8 (in Eq. (1)) to balance sharp boundaries from masks and fine-grained details from high-res features. ture maps at full-resolution, our approach enables t…
Figure 6
Figure 6. Figure 6: Visualization of predictions examples on semantic segmentation on COCO-Stuff [1] and depth estimation on NAVI [18]. We provide additional visualization in the supplements. Resize-conv FeatUp-JBU LoftUp Arch 50 52 54 56 58 60 62 mIoU (%) No Pseudo-GT LoftUp Pseudo-GT […
Figure 7
Figure 7. Figure 7: LoftUp pseudo-GT improves different upsamplers [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Visualization of attended regions(shown as dots) in the low-resolution features corresponding to a high-resolution pixel (marked as a cross). The density of dots represents the values in the attention map. LoftUp leverages relevant information from the global feature m…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. UPLiFT: Efficient Pixel-Dense Feature Upsampling with Local Attenders

    cs.CV 2026-01 conditional novelty 6.0 of 10

    UPLiFT shows that iterative 2× feature upsampling with a locally-defined attention operator beats cross-attention-based upsamplers on dense prediction while scaling linearly with token count.

  2. Maybe you don't need a U-Net: convolutional feature upsampling for materials micrograph segmentation

    cs.CV 2025-08 conditional novelty 5.0 of 10

    A lightweight CNN upsampler, distilled from FeatUp features, makes frozen DINOv2 patch features sharp enough for interactive segmentation of micrographs with sparse labels, and its workflow beats fine-tuning a U-Net i...

Reference graph

Works this paper leans on

64 extracted references · 46 canonical work pages · cited by 2 Pith papers

  1. [1]

    Coco- stuff: Thing and stuff classes in context

    Holger Caesar, Jasper Uijlings, and Vittorio Ferrari. Coco- stuff: Thing and stuff classes in context. In CVPR, pages 1209–1218, 2018. 7, 8, 12, 15

    work page 2018

  2. [2]

    Learning continuous image representation with local implicit image function

    Yinbo Chen, Sifei Liu, and Xiaolong Wang. Learning continuous image representation with local implicit image function. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8628–8638,

  3. [3]

    Schwing, Alexan- der Kirillov, and Rohit Girdhar

    Bowen Cheng, Ishan Misra, Alexander G. Schwing, Alexan- der Kirillov, and Rohit Girdhar. Masked-attention mask transformer for universal image segmentation. 2022. 1

    work page 2022

  4. [4]

    The cityscapes dataset for semantic urban scene understanding

    Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In CVPR,

  5. [5]

    Learning affinity- aware upsampling for deep image matting

    Yutong Dai, Hao Lu, and Chunhua Shen. Learning affinity- aware upsampling for deep image matting. In CVPR, pages 6841–6850, 2021. 3

    work page 2021

  6. [6]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. ICLR, 2021. 12

    work page 2021

  7. [7]

    Lanczos filtering in one and two dimen- sions

    Claude E Duchon. Lanczos filtering in one and two dimen- sions. Journal of Applied Meteorology (1962-1982) , pages 1016–1022, 1979. 2

    work page 1962

  8. [8]

    A guide to con- volution arithmetic for deep learning

    Vincent Dumoulin and Francesco Visin. A guide to con- volution arithmetic for deep learning. arXiv preprint arXiv:1603.07285, 2016. 3

    arXiv 2016

Show all 64 references
  1. [9]

    Depth map prediction from a single image using a multi-scale deep net- work

    David Eigen, Christian Puhrsch, and Rob Fergus. Depth map prediction from a single image using a multi-scale deep net- work. NeurIPS, 27, 2014. 12

    2014

  2. [10]

    Prob- ing the 3d awareness of visual foundation models

    Mohamed El Banani, Amit Raj, Kevis-Kokitsi Maninis, Ab- hishek Kar, Yuanzhen Li, Michael Rubinstein, Deqing Sun, Leonidas Guibas, Justin Johnson, and Varun Jampani. Prob- ing the 3d awareness of visual foundation models. In CVPR, pages 21795–21806, 2024. 1, 7, 12

    2024

  3. [11]

    Single image 3d without a single 3d image

    David F Fouhey, Wajahat Hussain, Abhinav Gupta, and Mar- tial Hebert. Single image 3d without a single 3d image. In ICCV, pages 1053–1061, 2015. 12

    2015

  4. [12]

    Brandt, Axel Feld- mann, Zhoutong Zhang, and William T

    Stephanie Fu, Mark Hamilton, Laura E. Brandt, Axel Feld- mann, Zhoutong Zhang, and William T. Freeman. Featup: A model-agnostic framework for features at any resolution. In ICLR, 2024. 1, 2, 3, 4, 5, 6, 7, 12

    2024

  5. [13]

    Unsupervised semantic segmentation by distilling feature correspondences

    Mark Hamilton, Zhoutong Zhang, Bharath Hariharan, Noah Snavely, and William T Freeman. Unsupervised semantic segmentation by distilling feature correspondences. arXiv preprint arXiv:2203.08414, 2022. 7, 12

    2022 arXiv

  6. [14]

    Semantic contours from inverse detectors

    Bharath Hariharan, Pablo Arbel ´aez, Lubomir Bourdev, Subhransu Maji, and Jitendra Malik. Semantic contours from inverse detectors. In ICCV, pages 991–998. IEEE, 2011. 13

    2011

  7. [15]

    Guided image fil- tering

    Kaiming He, Jian Sun, and Xiaoou Tang. Guided image fil- tering. PAMI, volume=35, number=6, pages=1397–1409, year=2012, publisher=IEEE. 3

    2012

  8. [16]

    Renovating names in open-vocabulary segmenta- tion benchmarks

    Haiwen Huang, Songyou Peng, Dan Zhang, and Andreas Geiger. Renovating names in open-vocabulary segmenta- tion benchmarks. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. 1, 6

    2024

  9. [17]

    Space-time correspondence as a contrastive random walk

    Allan Jabri, Andrew Owens, and Alexei Efros. Space-time correspondence as a contrastive random walk. NeurIPS, 33: 19545–19560, 2020. 6, 7, 12

    2020

  10. [18]

    NA VI: Category- agnostic image collections with high-quality 3d shape and pose annotations

    Varun Jampani, Kevis-Kokitsi Maninis, Andreas Engel- hardt, Arjun Karpur, Karen Truong, Kyle Sargent, Stefan Popov, Andre Araujo, Ricardo Martin-Brualla, Kaushal Pa- tel, Daniel Vlasic, Vittorio Ferrari, Ameesh Makadia, Ce Liu, Yuanzhen Li, and Howard Zhou. NA VI: Category- ag...

    2023

  11. [19]

    Adam: A method for stochastic optimization

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 ,

    arXiv

  12. [20]

    Pointrend: Image segmentation as rendering

    Alexander Kirillov, Yuxin Wu, Kaiming He, and Ross Gir- shick. Pointrend: Image segmentation as rendering. In CVPR, pages 9799–9808, 2020. 3

    2020

  13. [21]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. In ICCV, pages 4015–4026, 2023. 1, 2, 5, 6, 7, 12, 13

    2023

  14. [22]

    Joint bilateral upsampling

    Johannes Kopf, Michael F Cohen, Dani Lischinski, and Matt Uyttendaele. Joint bilateral upsampling. ACM TOG, 26(3): 96–es, 2007. 3, 5

    2007

  15. [23]

    Proxyclip: Proxy at- tention improves clip for open-vocabulary segmentation

    Mengcheng Lan, Chaofeng Chen, Yiping Ke, Xinjiang Wang, Litong Feng, and Wayne Zhang. Proxyclip: Proxy at- tention improves clip for open-vocabulary segmentation. In ECCV, 2024. 6, 7, 12

    2024

  16. [24]

    Exploring plain vision transformer backbones for object de- tection

    Yanghao Li, Hanzi Mao, Ross Girshick, and Kaiming He. Exploring plain vision transformer backbones for object de- tection. In European conference on computer vision , pages 280–296. Springer, 2022. 1

    2022

  17. [25]

    Vision transformer for nerf-based view synthesis from a single input image

    Kai-En Lin, Yen-Chen Lin, Wei-Sheng Lai, Tsung-Yi Lin, Yi-Chang Shih, and Ravi Ramamoorthi. Vision transformer for nerf-based view synthesis from a single input image. pages 806–815, 2023. 3

    2023

  18. [26]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In ECCV, pages 740–755. Springer, 2014. 6, 12

    2014

  19. [27]

    Simpleclick: Interactive image segmentation with sim- ple vision transformers

    Qin Liu, Zhenlin Xu, Gedas Bertasius, and Marc Nietham- mer. Simpleclick: Interactive image segmentation with sim- ple vision transformers. In ICCV, pages 22290–22300, 2023. 1, 6, 7, 12, 13

    2023

  20. [28]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. 12

    2017 arXiv

  21. [29]

    Index networks

    Hao Lu, Yutong Dai, Chunhua Shen, and Songcen Xu. Index networks. IEEE TPAMI, 44(1):242–255, 2020. 3

    2020

  22. [30]

    Fade: Fusing the assets of decoder and encoder for task-agnostic upsampling

    Hao Lu, Wenze Liu, Hongtao Fu, and Zhiguo Cao. Fade: Fusing the assets of decoder and encoder for task-agnostic upsampling. In ECCV, pages 231–247. Springer, 2022. 3

    2022

  23. [31]

    Sapa: Similarity-aware point affiliation for feature upsampling

    Hao Lu, Wenze Liu, Zixuan Ye, Hongtao Fu, Yuliang Liu, and Zhiguo Cao. Sapa: Similarity-aware point affiliation for feature upsampling. NeurIPS, 35:20889–20901, 2022. 3

    2022

  24. [32]

    A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics

    David Martin, Charless Fowlkes, Doron Tal, and Jitendra Malik. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In ICCV, pages 416–423. IEEE, 2001. 6, 13

    2001

  25. [33]

    Cubic spline interpola- tion

    Sky McKinley and Megan Levine. Cubic spline interpola- tion. College of the Redwoods, 45(1):1049–1060, 1998. 2

    1998

  26. [34]

    Nerf: Representing scenes as neural radiance fields for view syn- thesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis. Communications of the ACM, 65(1):99–106, 2021. 3, 13

    2021

  27. [35]

    Learning deconvolution network for semantic segmentation

    Hyeonwoo Noh, Seunghoon Hong, and Bohyung Han. Learning deconvolution network for semantic segmentation. In ICCV, 2015. 3

    2015

  28. [36]

    De- convolution and checkerboard artifacts

    Augustus Odena, Vincent Dumoulin, and Chris Olah. De- convolution and checkerboard artifacts. Distill, 1(10):e3,

  29. [37]

    Maxime Oquab, Timoth ´ee Darcet, Th´eo Moutakanni, Huy V . V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel HAZIZA, Francisco Massa, Alaaeldin El-Nouby, Mido Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael ...

    2024

  30. [38]

    A benchmark dataset and evaluation methodology for video object segmentation

    Federico Perazzi, Jordi Pont-Tuset, Brian McWilliams, Luc Van Gool, Markus Gross, and Alexander Sorkine-Hornung. A benchmark dataset and evaluation methodology for video object segmentation. In CVPR, pages 724–732, 2016. 6, 13

    2016

  31. [39]

    The 2017 davis challenge on video object segmentation

    Jordi Pont-Tuset, Federico Perazzi, Sergi Caelles, Pablo Ar- bel´aez, Alexander Sorkine-Hornung, and Luc Van Gool. The 2017 davis challenge on video object segmentation. arXiv:1704.00675, 2017. 2, 7, 12, 16

    2017 arXiv

  32. [40]

    Three pillars improving vision foundation model distillation for lidar

    Gilles Puy, Spyros Gidaris, Alexandre Boulch, Oriane Sim´eoni, Corentin Sautier, Patrick P´erez, Andrei Bursuc, and Renaud Marlet. Three pillars improving vision foundation model distillation for lidar. In CVPR, pages 21519–21529,

  33. [41]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. pages 8748–8763. PmLR, 2021. 1, 3, 7, 12, 13

    2021

  34. [42]

    Vi- sion transformers for dense prediction

    Ren ´e Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. Vi- sion transformers for dense prediction. In ICCV, pages 12179–12188, 2021. 7

    2021

  35. [43]

    Am-radio: Agglomerative vision foundation model reduce all domains into one

    Mike Ranzinger, Greg Heinrich, Jan Kautz, and Pavlo Molchanov. Am-radio: Agglomerative vision foundation model reduce all domains into one. In CVPR, pages 12490– 12500, 2024. 1, 7, 13

    2024

  36. [44]

    Glamm: Pixel grounding large multimodal model

    Hanoona Rasheed, Muhammad Maaz, Sahal Shaji, Abdel- rahman Shaker, Salman Khan, Hisham Cholakkal, Rao M Anwer, Eric Xing, Ming-Hsuan Yang, and Fahad S Khan. Glamm: Pixel grounding large multimodal model. In CVPR, pages 13009–13018, 2024. 5

    2024

  37. [45]

    Sam 2: Segment anything in images and videos

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman R¨adle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junt- ing Pan, Kalyan Vasudev Alwala, Nicolas Carion, Chao- Yuan Wu, Ross Girshick, Piotr Doll´ar, and Christoph Feic...

    2024 arXiv

  38. [46]

    Grounded sam: Assembling open-world models for diverse visual tasks,

    Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kun- chang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, Zhaoyang Zeng, Hao Zhang, Feng Li, Jie Yang, Hongyang Li, Qing Jiang, and Lei Zhang. Grounded sam: Assembling open-world models for diverse visual tasks,

  39. [47]

    U- net: Convolutional networks for biomedical image segmen- tation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U- net: Convolutional networks for biomedical image segmen- tation. In Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, pa...

    2015

  40. [48]

    ” grabcut” interactive foreground extraction using iterated graph cuts

    Carsten Rother, Vladimir Kolmogorov, and Andrew Blake. ” grabcut” interactive foreground extraction using iterated graph cuts. ACM TOG, 23(3):309–314, 2004. 6, 13

    2004

  41. [49]

    Is the deconvolution layer the same as a convolutional layer?arXiv preprint arXiv:1609.07009, 2016

    Wenzhe Shi, Jose Caballero, Lucas Theis, Ferenc Huszar, Andrew Aitken, Christian Ledig, and Zehan Wang. Is the deconvolution layer the same as a convolutional layer?arXiv preprint arXiv:1609.07009, 2016. 3

    2016 arXiv

  42. [50]

    Adaptis: Adaptive instance selection network

    Konstantin Sofiiuk, Olga Barinova, and Anton Konushin. Adaptis: Adaptive instance selection network. In ICCV, pages 7355–7363, 2019. 13

    2019

  43. [51]

    Re- viving iterative training with mask guidance for interactive segmentation

    Konstantin Sofiiuk, Ilya A Petrov, and Anton Konushin. Re- viving iterative training with mask guidance for interactive segmentation. In ICIP, pages 3141–3145. IEEE, 2022. 13

    2022

  44. [52]

    Lift: A surprisingly simple lightweight feature transform for dense vit descriptors

    Saksham Suri, Matthew Walmer, Kamal Gupta, and Abhinav Shrivastava. Lift: A surprisingly simple lightweight feature transform for dense vit descriptors. In ECCV, pages 110–

  45. [53]

    Splatter image: Ultra-fast single-view 3d recon- struction

    Stanislaw Szymanowicz, Chrisitian Rupprecht, and Andrea Vedaldi. Splatter image: Ultra-fast single-view 3d recon- struction. In CVPR, pages 10208–10217, 2024. 3

    2024

  46. [55]

    Siglip 2: Multilingual vision-language en- coders with improved semantic understanding, localization, and dense features

    Michael Tschannen, Alexey Gritsenko, Xiao Wang, Muham- mad Ferjad Naeem, Ibrahim Alabdulmohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, et al. Siglip 2: Multilingual vision-language en- coders with improved semantic understanding, localization, ...

    2025 arXiv

  47. [56]

    Dino-tracker: Taming dino for self-supervised point tracking in a single video

    Narek Tumanyan, Assaf Singer, Shai Bagon, and Tali Dekel. Dino-tracker: Taming dino for self-supervised point tracking in a single video. In ECCV, pages 367–385. Springer, 2024. 1

    2024

  48. [57]

    Carafe: Content-aware reassembly of fea- tures

    Jiaqi Wang, Kai Chen, Rui Xu, Ziwei Liu, Chen Change Loy, and Dahua Lin. Carafe: Content-aware reassembly of fea- tures. In ICCV, pages 3007–3016, 2019. 3

    2019

  49. [58]

    Self-supervised trans- formers for unsupervised object discovery using normalized cut

    Yangtao Wang, Xi Shen, Shell Xu Hu, Yuan Yuan, James L Crowley, and Dominique Vaufreydaz. Self-supervised trans- formers for unsupervised object discovery using normalized cut. In CVPR, pages 14543–14553, 2022. 6

    2022

  50. [59]

    Clip-dinoiser: Teaching clip a few dino tricks for open- vocabulary semantic segmentation

    Monika Wysocza ´nska, Oriane Sim´eoni, Micha¨el Ramamon- jisoa, Andrei Bursuc, Tomasz Trzci ´nski, and Patrick P ´erez. Clip-dinoiser: Teaching clip a few dino tricks for open- vocabulary semantic segmentation. In ECCV, pages 320–

  51. [60]

    Featuren- erf: Learning generalizable nerfs by distilling foundation models

    Jianglong Ye, Naiyan Wang, and Xiaolong Wang. Featuren- erf: Learning generalizable nerfs by distilling foundation models. In ICCV, pages 8962–8973, 2023. 3, 13

    2023

  52. [61]

    pixelnerf: Neural radiance fields from one or few images

    Alex Yu, Vickie Ye, Matthew Tancik, and Angjoo Kanazawa. pixelnerf: Neural radiance fields from one or few images. 2021 ieee. In CVPR, 2020. 3, 13

    2021

  53. [62]

    Convolutions die hard: Open-vocabulary seg- mentation with single frozen convolutional clip

    Qihang Yu, Ju He, Xueqing Deng, Xiaohui Shen, and Liang- Chieh Chen. Convolutions die hard: Open-vocabulary seg- mentation with single frozen convolutional clip. Advances in Neural Information Processing Systems, 36:32215–32234,

  54. [63]

    Sigmoid loss for language image pre-training

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In ICCV, pages 11975–11986, 2023. 1

    2023

  55. [64]

    LoftUp: earning a Coordinate-Based Feature Upsampler for Vi- sion Foundation Models

    Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Scene parsing through ade20k dataset. In CVPR, pages 633–641, 2017. 6, 12 LoftUp: Learning a Coordinate-Based Feature Upsampler for Vision Foundation Models Supplementary Material This supp...

    2017

  56. [128]

    1, 2, 3, 4, 6, 7, 12

    Springer, 2024. 1, 2, 3, 4, 6, 7, 12

    2024

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.