pith. sign in

arxiv: 2605.31597 · v3 · pith:VEMME5RSnew · submitted 2026-05-29 · 💻 cs.CV

SOCO: Benchmarking Semantic Object Correspondence in Vision Foundation Models

Olaf D\"unkel , Basavaraj Sunagad , Haoran Wang , David T. Hoffmann , Christian Theobalt , Adam Kortylewski This is my paper

Pith reviewed 2026-07-02 22:49 UTC · model grok-4.3

classification 💻 cs.CV
keywords semantic correspondencevision foundation modelsbenchmarkkeypoint matchingpart-level understandingdense prediction tasksobject correspondencemultimodal evaluation
0
0 comments X

The pith

Semantic object correspondence performance predicts dense downstream tasks more strongly than ImageNet classification.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces SOCO, a benchmark for evaluating semantic object correspondence in vision foundation models through consistent keypoint annotations and language descriptions across 100 categories. It tests whether models can match object parts under variations in appearance, viewpoint, and geometry. Experiments reveal that backbones capture semantic structure internally yet transfer it poorly across categories and only partially encode part positions. Large vision-language models localize parts more effectively from text prompts than from visual references. The central result shows that correspondence performance correlates more strongly with outcomes on segmentation, tracking, 3D pose estimation, and 3D detection than ImageNet classification accuracy does.

Core claim

SOCO supplies a taxonomy of correspondence types together with consistent, functionally meaningful keypoint annotations across 100 categories and over 1M pairs, plus language descriptions for evaluating both visual and text-grounded matching. Vision foundation backbones encode strong semantic structure but transfer correspondences poorly across related categories and only partially capture object-part position. LVLMs prove stronger at text-prompted part localization than at visual-reference cross-image matching. Correspondence performance predicts performance on dense downstream tasks, including segmentation, tracking, 3D pose estimation, and 3D detection, more strongly than ImageNet classif

What carries the argument

The SOCO benchmark, which supplies a taxonomy of correspondence types and over 1M consistent keypoint pairs with language descriptions across 100 categories.

If this is right

  • Vision foundation models encode semantic structure that does not transfer reliably across related object categories.
  • Large vision-language models localize parts better from language than from visual reference matching.
  • Semantic correspondence evaluation captures structured understanding that standard classification misses.
  • Models with stronger semantic correspondence are expected to perform better on segmentation, tracking, and 3D tasks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Training objectives that directly optimize semantic correspondence could improve results across multiple dense prediction problems.
  • Extending the benchmark to video or 3D mesh data might expose additional limits in current models' part-level representations.
  • Separate evaluation protocols for visual versus language-grounded correspondence may be needed to close the observed gap in LVLMs.

Load-bearing premise

The provided keypoint annotations are consistent, functionally meaningful, and representative of semantic correspondence across instances and categories.

What would settle it

A model that scores high on SOCO yet performs worse than lower-scoring models on segmentation, tracking, 3D pose estimation, or 3D detection would falsify the claim that correspondence performance is the stronger predictor.

Figures

Figures reproduced from arXiv: 2605.31597 by Adam Kortylewski, Basavaraj Sunagad, Christian Theobalt, David T. Hoffmann, Haoran Wang, Olaf D\"unkel.

Figure 1
Figure 1. Figure 1: SOCO provides the first taxonomy-driven, language-grounded formulation of Semantic Object Correspondence (SOC), enabling structured, semantically co￾herent, and cross-category part annotations across 100 diverse categories, which allows evaluating semantic and structured object understanding in vision foundation models (VFMs) and large vision language models (LVLMs). Abstract. Measuring structured object u… view at source ↗
Figure 2
Figure 2. Figure 2: Illustration of concept correspondence (CC), semantic object cor [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Statistics of labeled keypoints. Keypoints in SOCO are annotated for a diverse set of categories from four super-categories. Each category is labeled with a subset of keypoints that are shared across multiple categories. The animal keypoints are shared across all animal categories. Image collection. All images are samples from ImageNet. We rely on 2D and 3D annotations from ImageNet3D [41] for man-made obj… view at source ↗
Figure 4
Figure 4. Figure 4: Per-task Pearson r across 37 vision models, with 95% bootstrap CIs. Left: SOC correlates with every downstream task more strongly than ImageNet kNN. Right: the SOC advantage ∆r = rSOC −rkNN stays positive on all tasks and is preserved on a 17 subset only including models trained with dense SSL objectives. Overall, recent models show clear improvements in both visual and lan￾guage understanding. For example… view at source ↗
read the original abstract

Measuring structured object understanding in vision foundation models remains challenging due to inconsistent evaluation protocols and limited part-level supervision. Semantic correspondence (SC) evaluates this capability by testing whether object parts can be matched across instances and categories under large variations in appearance, viewpoint, and geometry. To enable a systematic SC evaluation, we introduce SOCO, a new benchmark for Semantic Object Correspondence that introduces a taxonomy of correspondence types and provides consistent, functionally meaningful keypoint annotations across 100 categories and over 1M correspondence pairs. In addition, SOCO includes keypoint language descriptions, enabling the evaluation of large vision-language models (LVLMs) and their fine-grained part-level understanding. Comprehensive experiments reveal that (i) vision foundation backbones encode strong semantic structure but transfer correspondences poorly across related categories and only partially capture object-part position, (ii) LVLMs are stronger at text-prompted part localization than at visual-reference cross-image matching, exposing a gap between language-grounded localization and fine-grained visual correspondence, and (iii) correspondence performance predicts performance on dense downstream tasks, including segmentation, tracking, 3D pose estimation, and 3D detection, more strongly than ImageNet classification. Together, these findings position SOCO as a benchmark for structured, part-level representation quality in vision and multimodal foundation models.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The manuscript introduces SOCO, a benchmark for semantic object correspondence (SC) in vision foundation models and large vision-language models (LVLMs). It defines a taxonomy of correspondence types, supplies consistent keypoint annotations across 100 categories and >1M pairs plus language descriptions, and reports three findings: (i) vision backbones encode semantic structure but transfer correspondences poorly across categories and only partially capture part position; (ii) LVLMs are stronger at text-prompted part localization than visual-reference matching; (iii) SC performance on SOCO predicts dense downstream tasks (segmentation, tracking, 3D pose estimation, 3D detection) more strongly than ImageNet classification accuracy.

Significance. If the keypoint annotations prove reliable, SOCO would supply a large-scale, part-level evaluation resource that directly targets structured object understanding, a capability only indirectly measured by existing protocols. The explicit comparison of SC versus ImageNet as predictors of dense-task performance, together with the inclusion of LVLMs, would be a useful contribution to foundation-model evaluation. The scale (>1M pairs) and the provision of language descriptions are concrete strengths.

major comments (2)
  1. [Abstract and §3] Abstract and §3 (Benchmark Construction): the central claim that SOCO scores are a faithful proxy for structured object understanding (and therefore a stronger predictor than ImageNet) rests on the annotations being “consistent, functionally meaningful” across 100 categories. No inter-annotator agreement statistics, consistency checks across instances, or validation against functional equivalence are reported; without these, any downstream correlation analysis risks capturing annotation artifacts rather than representational structure.
  2. [§4 and §5] §4 (Experiments) and §5 (Downstream Correlation): the claim that correspondence performance predicts segmentation, tracking, 3D pose, and 3D detection more strongly than ImageNet accuracy requires the correlation methodology—data splits, number of models evaluated, statistical significance tests, and controls for model capacity—to be fully specified. These details are absent from the abstract and not verifiable from the provided text, undermining the comparative strength asserted in finding (iii).
minor comments (1)
  1. [Abstract] Abstract: the summary of experimental findings omits any mention of methods, data splits, or statistical details; a one-sentence methods clause would improve readability.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback on annotation reliability and methodological transparency. We address each major comment below and will incorporate clarifications and additions in the revised manuscript.

read point-by-point responses

  1. Referee: [Abstract and §3] Abstract and §3 (Benchmark Construction): the central claim that SOCO scores are a faithful proxy for structured object understanding (and therefore a stronger predictor than ImageNet) rests on the annotations being “consistent, functionally meaningful” across 100 categories. No inter-annotator agreement statistics, consistency checks across instances, or validation against functional equivalence are reported; without these, any downstream correlation analysis risks capturing annotation artifacts rather than representational structure.

    Authors: We agree that explicit validation of annotation consistency is necessary to support the claims. The original submission omitted these statistics. In the revision we will add a dedicated subsection in §3 reporting inter-annotator agreement (Cohen’s kappa and percentage agreement) computed on a 20-category subset annotated by five independent annotators, instance-level consistency checks across the 100 categories, and a functional-equivalence validation performed by domain experts. These additions will be accompanied by the raw agreement numbers and will directly address the risk of annotation artifacts. revision: yes

  2. Referee: [§4 and §5] §4 (Experiments) and §5 (Downstream Correlation): the claim that correspondence performance predicts segmentation, tracking, 3D pose, and 3D detection more strongly than ImageNet accuracy requires the correlation methodology—data splits, number of models evaluated, statistical significance tests, and controls for model capacity—to be fully specified. These details are absent from the abstract and not verifiable from the provided text, undermining the comparative strength asserted in finding (iii).

    Authors: We acknowledge that the correlation analysis protocol must be stated explicitly in the main text. The revised §5 will include: (i) the precise data splits (80/20 per downstream task, with no category overlap between SOCO and downstream sets), (ii) the full list of evaluated models (12 vision backbones + 5 LVLMs), (iii) the statistical procedure (Pearson r with two-tailed p-values and bootstrap confidence intervals), and (iv) capacity controls (partial correlation after regressing out parameter count and ImageNet accuracy). These details existed in the supplementary material; they will now appear in the main paper with a reference to the supplementary tables. revision: yes

Circularity Check

0 steps flagged

Empirical benchmark study with no derivation chain or self-referential fitting

full rationale

The paper presents SOCO as a new benchmark with keypoint annotations across categories and reports experimental correlations between semantic correspondence scores, downstream task performance, and ImageNet accuracy. No equations, fitted parameters, or derivations are described. The central claim (iii) is an empirical statistical observation from model evaluations, not a reduction to inputs by construction. No self-citations are invoked as load-bearing uniqueness theorems or ansatzes. The work is self-contained as an evaluation study; annotation quality affects validity but does not create circularity in any claimed derivation.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central contribution is the creation of the benchmark itself; no free parameters or invented physical entities are introduced. The work rests on the domain assumption that consistent semantic keypoints can be defined and annotated at scale.

axioms (1)
  • domain assumption Keypoint annotations can be defined consistently and meaningfully across object instances and categories
    The benchmark validity depends on this premise for the taxonomy and 1M pairs to represent true semantic correspondence.

pith-pipeline@v0.9.1-grok · 5780 in / 1136 out tokens · 30939 ms · 2026-07-02T22:49:02.940664+00:00 · methodology

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

82 extracted references · 18 canonical work pages · 12 internal anchors

  1. [1]

    NeurIPS35(2022)

    Alayrac, J.B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millicah, K., Reynolds, M., Ring, R., Rutherford, E., Cabi, S., Han, T., Gong, Z., Samangooei, S., Monteiro, M., Menick, J., Borgeaud, S., Brock, A., Nematzadeh, A., Sharifzadeh, S., Binkowski, M., Barreira, R., Vinyals, O., Zisserman, A., Simonyan, K.: Flamingo: A...

    2022

  2. [2]

    In: CVPR

    Andriluka, M., Pishchulin, L., Gehler, P., Schiele, B.: 2D human pose estimation: New benchmark and state of the art analysis. In: CVPR. pp. 3686–3693 (2014)

    2014

  3. [3]

    In: CVPR

    Assran, M., Duval, Q., Misra, I., Bojanowski, P., Vincent, P., Rabbat, M., Le- Cun, Y., Ballas, N.: Self-supervised learning from images with a joint-embedding predictive architecture. In: CVPR. pp. 15619–15629 (2023)

    2023

  4. [4]

    Aydemir, G., Xie, W., Güney, F.: Can visual foundation models achieve long-term point tracking? In: ECCV Workshops (2024)

    2024

  5. [5]

    Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond

    Bai, J., Bai, S., Yang, S., Wang, S., Tan, S., Wang, P., Lin, J., Zhou, C., Zhou, J.: Qwen-VL: A versatile vision-language model for understanding, localization, text reading, and beyond. arXiv preprint arXiv:2308.12966 (2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  6. [6]

    Qwen3-VL Technical Report

    Bai, S., Cai, Y., Chen, R., Chen, K., Chen, X., Cheng, Z., Deng, L., Ding, W., Gao, C., Ge, C., et al.: Qwen3-VL technical report. arXiv preprint arXiv:2511.21631 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  7. [7]

    Qwen2.5-VL Technical Report

    Bai, S., Chen, K., Liu, X., Wang, J., Ge, W., Song, S., Dang, K., Wang, P., Wang, S., Tang, J., Zhong, H., Zhu, Y., Yang, M., Li, Z., Wan, J., Wang, P., Ding, W., Fu, Z., Xu, Y., Ye, J., Zhang, X., Xie, T., Cheng, Z., Zhang, H., Yang, Z., Xu, H., Lin, J.: Qwen2.5-VL technical report. arXiv preprint arXiv:2502.13923 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  8. [8]

    ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Data

    Baruch, G., Chen, Z., Dehghan, A., Dimry, T., Feigin, Y., Fu, P., Gebauer, T., Joffe, B., Kurz, D., Schwartz, A., Shulman, E.: ARKitScenes: A diverse real- world dataset for 3D indoor scene understanding using mobile RGB-D data. arXiv preprint arXiv:2111.08897 (2021)

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  9. [9]

    Perception Encoder: The best visual embeddings are not at the output of the network

    Bolya, D., Huang, P.Y., Sun, P., Cho, J.H., Madotto, A., Wei, C., Ma, T., Zhi, J., Rajasegaran, J., Rasheed, H., et al.: Perception encoder: The best visual em- beddings are not at the output of the network. arXiv preprint arXiv:2504.13181 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  10. [10]

    In: CVPR (2023)

    Brazil, G., Kumar, A., Straub, J., Ravi, N., Johnson, J., Gkioxari, G.: Omni3D: A large benchmark and model for 3D object detection in the wild. In: CVPR (2023)

    2023

  11. [11]

    In: ECCV

    Butler, D.J., Wulff, J., Stanley, G.B., Black, M.J.: A naturalistic open source movie for optical flow evaluation. In: ECCV. pp. 611–625 (2012)

    2012

  12. [12]

    Depthlm: Metric depth from vision language models.arXiv preprint arXiv:2509.25413,

    Cai, Z., Yeh, C.F., Xu, H., Liu, Z., Meyer, G., Lei, X., Zhao, C., Li, S.W., Chandra, V., Shi, Y.: DepthLM: Metric depth from vision language models. arXiv preprint arXiv:2509.25413 (2025)

    work page arXiv 2025

  13. [13]

    In: ICCV (2021)

    Caron, M., Touvron, H., Misra, I., Jégou, H., Mairal, J., Bojanowski, P., Joulin, A.: Emerging properties in self-supervised vision transformers. In: ICCV (2021)

    2021

  14. [14]

    In: 3DV (2026)

    Chi, Y., Sommer, L., Dünkel, O., Muhle, D., Cremers, D., Theobalt, C., Ko- rtylewski, A.: C3PO: Canonicalization of 3D pose from partial views with gen- eralizable correspondence features. In: 3DV (2026)

    2026

  15. [15]

    In: CVPR

    Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., Schiele, B.: The cityscapes dataset for semantic urban scene understanding. In: CVPR. pp. 3213–3223 (2016)

    2016

  16. [16]

    In: CVPR

    Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: A large-scale hierarchical image database. In: CVPR. pp. 248–255 (2009) 20 O. Dünkel et al

    2009

  17. [17]

    NeurIPS35, 13610–13626 (2022)

    Doersch,C.,Gupta,A.,Markeeva,L.,Recasens,A.,Smaira,L.,Aytar,Y.,Carreira, J., Zisserman, A., Yang, Y.: TAP-Vid: A benchmark for tracking any point in a video. NeurIPS35, 13610–13626 (2022)

    2022

  18. [18]

    In: ICCV (2025)

    Dünkel, O., Jesslen, A., Xie, J., Theobalt, C., Rupprecht, C., Kortylewski, A.: CNS-Bench: Benchmarking image classifier robustness under continuous nuisance shifts. In: ICCV (2025)

    2025

  19. [19]

    In: ICCV (2025)

    Dünkel, O., Wimmer, T., Theobalt, C., Rupprecht, C., Kortylewski, A.: Do it yourself: Learning semantic correspondence from pseudo-labels. In: ICCV (2025)

    2025

  20. [20]

    In: CVPR

    El Banani, M., Raj, A., Maninis, K.K., Kar, A., Li, Y., Rubinstein, M., Sun, D., Guibas, L., Johnson, J., Jampani, V.: Probing the 3D awareness of visual foundation models. In: CVPR. pp. 21795–21806 (2024)

    2024

  21. [21]

    IJCV 111(1), 98–136 (2015)

    Everingham, M., Eslami, S.M.A., Van Gool, L., Williams, C.K.I., Winn, J., Zis- serman, A.: The PASCAL visual object classes challenge: A retrospective. IJCV 111(1), 98–136 (2015)

    2015

  22. [22]

    IJCV88(2), 303–338 (2010)

    Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The PASCAL visual object classes (VOC) challenge. IJCV88(2), 303–338 (2010)

    2010

  23. [23]

    Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation

    Florence, P.R., Manuelli, L., Tedrake, R.: Dense object nets: Learning dense visual object descriptors by and for robotic manipulation. arXiv preprint arXiv:1806.08756 (2018)

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  24. [24]

    In: ECCV (2024)

    Fu, X., Hu, Y., Li, B., Feng, Y., Wang, H., Lin, X., Roth, D., Smith, N.A., Ma, W.C., Krishna, R.: BLINK: Multimodal large language models can see but not perceive. In: ECCV (2024)

    2024

  25. [25]

    In: NeurIPS (2025)

    Gan, C., Tu, Y., Chen, X., Chen, T., Li, Y., Harandi, M., Lin, W.: Unleashing diffusiontransformersforvisualcorrespondencebymodulatingmassiveactivations. In: NeurIPS (2025)

    2025

  26. [26]

    Gemini: A Family of Highly Capable Multimodal Models

    Gemini Team: Gemini: A family of highly capable multimodal models. arXiv preprint arXiv:2312.11805 (2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  27. [27]

    In: CVPR

    Ham, B., Cho, M., Schmid, C., Ponce, J.: Proposal flow. In: CVPR. pp. 3475–3484 (2016)

    2016

  28. [28]

    In: CVPR

    He, K., Chen, X., Xie, S., Li, Y., Dollár, P., Girshick, R.: Masked autoencoders are scalable vision learners. In: CVPR. pp. 16000–16009 (2022)

    2022

  29. [29]

    In: CVPR (2025)

    Heinrich, G., Ranzinger, M., Yin, H., Lu, Y., Kautz, J., Tao, A., Catanzaro, B., Molchanov, P.: RADIOv2.5: Improved baselines for agglomerative vision founda- tion models. In: CVPR (2025)

    2025

  30. [30]

    GPT-4o System Card

    Hurst, A., Lerer, A., Goucher, A.P., Perelman, A., Ramesh, A., Clark, A., Ostrow, A.J., Welihinda, A., Hayes, A., Radford, A., et al.: GPT-4o system card. arXiv preprint arXiv:2410.21276 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  31. [31]

    In: NeurIPS (2023)

    Jampani, V., Maninis, K.K., Engelhardt, A., Karpur, A., Truong, K., Sargent, K., Popov, S., Araujo, A., Martin-Brualla, R., Patel, K., Vlasic, D., Ferrari, V., Makadia, A., Liu, C., Li, Y., Zhou, H.: NAVI: Category-agnostic image collections with high-quality 3D shape and pose annotations. In: NeurIPS (2023)

    2023

  32. [32]

    Kornblith, S., Shlens, J., Le, Q.V.: Do better ImageNet models transfer better? In: CVPR. pp. 2661–2671 (2019)

    2019

  33. [33]

    TMLR (2024)

    Li, B., Zhang, Y., Guo, D., Zhang, R., Li, F., Zhang, H., Zhang, K., Zhang, P., Li, Y., Liu, Z., Li, C.: LLaVA-OneVision: Easy visual task transfer. TMLR (2024)

    2024

  34. [34]

    In: ICML (2022)

    Li, J., Li, D., Xiong, C., Hoi, S.: BLIP: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In: ICML (2022)

    2022

  35. [35]

    In: CVPR

    Lin, T.Y., Dollár, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: CVPR. pp. 936–944 (2017) SOCO 21

    2017

  36. [36]

    In: ECCV

    Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft COCO: Common objects in context. In: ECCV. pp. 740–755 (2014)

    2014

  37. [37]

    In: ECCV

    Liu, C., Yuen, J., Torralba, A., Sivic, J., Freeman, W.T.: SIFT flow: Dense corre- spondence across different scenes. In: ECCV. pp. 28–42 (2008)

    2008

  38. [38]

    In: NeurIPS (2023)

    Liu, H., Li, C., Wu, Q., Lee, Y.J.: Visual instruction tuning. In: NeurIPS (2023)

    2023

  39. [39]

    Liu, Y., Duan, H., Zhang, Y., Li, B., Zhang, S., Zhao, W., Yuan, Y., Wang, J., He, C., Liu, Z., et al.: MMBench: Is your multi-modal model an all-around player? In: ECCV. pp. 216–233 (2024)

    2024

  40. [40]

    In: NeurIPS (2023)

    Luo, G., Dunlap, L., Park, D.H., Holynski, A., Darrell, T.: Diffusion hyperfeatures: Searching through time and space for semantic correspondence. In: NeurIPS (2023)

    2023

  41. [41]

    NeurIPS37, 96127–96149 (2024)

    Ma, W., Zhang, G., Liu, Q., Zeng, G., Kortylewski, A., Liu, Y., Yuille, A.: Im- ageNet3D: Towards general-purpose object-level 3D understanding. NeurIPS37, 96127–96149 (2024)

    2024

  42. [42]

    arXiv preprint arXiv:2506.08220 (2025)

    Mariotti, O., Du, Z., Bhalgat, Y., Mac Aodha, O., Bilen, H.: Jamais vu: Expos- ing the generalization gap in supervised semantic correspondence. arXiv preprint arXiv:2506.08220 (2025)

    work page arXiv 2025

  43. [43]

    In: CVPR

    Mariotti, O., Mac Aodha, O., Bilen, H.: Improving semantic correspondence with viewpoint-guided spherical maps. In: CVPR. pp. 19521–19530 (2024)

    2024

  44. [44]

    In: CVPR

    Mayer, N., Ilg, E., Hausser, P., Fischer, P., Cremers, D., Dosovitskiy, A., Brox, T.: A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation. In: CVPR. pp. 4040–4048 (2016)

    2016

  45. [45]

    arXiv prepreint arXiv:1908.10543 , year=

    Min, J., Lee, J., Ponce, J., Cho, M.: SPair-71k: A large-scale benchmark for se- mantic correspondence. arXiv preprint arXiv:1908.10543 (2019)

    work page arXiv 1908

  46. [46]

    OpenAI, Applin, S., Adesso, G., Ashfaq, R., Bai, M., Brammer, M., Fecht, E., Goodman, A., Grossman, S., Groh, M., Kirk, H.R., Gunitsky, S., Huang, Y., Kahn, L., Kumar, S., Madrid-Morales, D., Motoki, F., Ovadya, A., Peters, U., Robinson, M., Röttger, P., Wasserman, H., Wehsener, A., Walker, L., Vidgen, B., Zhu, J.: GPT-4V(ision) system card. Tech. rep., O...

    2023

  47. [47]

    TMLR (2024)

    Oquab, M., Darcet, T., Moutakanni, T., Vo, H.V., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A., Howes, R., Huang, P.Y., Xu, H., Sharma, V., Li, S.W., Galuba, W., Rabbat, M., Assran, M., Ballas, N., Syn- naeve, G., Misra, I., Jégou, H., Mairal, J., Labatut, P., Joulin, A., Bojanowski, P.: DINOv2: Learning robust visual fe...

    2024

  48. [48]

    In: ICML

    Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: ICML. pp. 8748–8763 (2021)

    2021

  49. [49]

    In: ICCV

    Ranftl, R., Bochkovskiy, A., Koltun, V.: Vision transformers for dense prediction. In: ICCV. pp. 12179–12188 (2021)

    2021

  50. [50]

    In: CVPR

    Ranzinger,M.,Heinrich,G.,Kautz,J.,Molchanov,P.:AM-RADIO:Agglomerative vision foundation model reduce all domains into one. In: CVPR. pp. 12490–12500 (2024)

    2024

  51. [51]

    arXiv preprint arXiv:2601.17237 (2026)

    Ranzinger, M., Heinrich, G., McCarthy, C., Kautz, J., Tao, A., Catanzaro, B., Molchanov, P.: C-RADIOv4 technical report. arXiv preprint arXiv:2601.17237 (2026)

    work page arXiv 2026

  52. [52]

    In: CVPR

    Rocco, I., Arandjelovic, R., Sivic, J.: Convolutional neural network architecture for geometric matching. In: CVPR. pp. 6148–6157 (2017)

    2017

  53. [53]

    In: CVPR

    Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: CVPR. pp. 10684–10695 (2022) 22 O. Dünkel et al

    2022

  54. [54]

    In: CVPR

    Sarıyıldız, M.B., Weinzaepfel, P., Lucas, T., De Jorge, P., Larlus, D., Kalantidis, Y.: DUNE: Distilling a universal encoder from heterogeneous 2D and 3D teachers. In: CVPR. pp. 30084–30094 (2025)

    2025

  55. [55]

    IJCV47(1), 7–42 (2002)

    Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. IJCV47(1), 7–42 (2002)

    2002

  56. [56]

    In: ICCV (2015)

    Sedaghat, N., Brox, T.: Unsupervised generation of a viewpoint annotated car dataset from videos. In: ICCV (2015)

    2015

  57. [57]

    Shen, S., Li, L.H., Tan, H., Bansal, M., Rohrbach, A., Chang, K.W., Yao, Z., Keutzer, K.: How much can CLIP benefit vision-and-language tasks? In: ICLR (2022)

    2022

  58. [58]

    In: ICCV (2023)

    Shtedritski, A., Rupprecht, C., Vedaldi, A.: What does CLIP know about a red circle? visual prompt engineering for VLMs. In: ICCV (2023)

    2023

  59. [59]

    In: ECCV (2012)

    Silberman, N., Hoiem, D., Kohli, P., Fergus, R.: Indoor segmentation and support inference from RGB-D images. In: ECCV (2012)

    2012

  60. [60]

    DINOv3

    Siméoni, O., Vo, H.V., Seitzer, M., Baldassarre, F., Oquab, M., Jose, C., Khalidov, V., Szafraniec, M., Yi, S., Ramamonjisoa, M., Massa, F., Haziza, D., Wehrstedt, L., Wang, J., Darcet, T., Moutakanni, T., Sentana, L., Roberts, C., Vedaldi, A., Tolan, J., Brandt, J., Couprie, C., Mairal, J., Jégou, H., Labatut, P., Bojanowski, P.: DINOv3. arXiv preprint a...

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  61. [61]

    In: CVPR

    Sommer, L., Dünkel, O., Theobalt, C., Kortylewski, A.: Common3D: Self- supervised learning of 3D morphable models for common objects in neural feature space. In: CVPR. pp. 6468–6479 (2025)

    2025

  62. [62]

    In: CVPR

    Stracke, N., Baumann, S.A., Bauer, K., Fundel, F., Ommer, B.: CleanDIFT: Dif- fusion features without noise. In: CVPR. pp. 117–127 (2025)

    2025

  63. [63]

    In: CVPR

    Sun, Y., Huang, Y., Guo, H., Zhao, Y., Wu, R., Yu, Y., Ge, W., Zhang, W.: MISC210K: A large-scale dataset for multi-instance semantic correspondence. In: CVPR. pp. 7121–7130 (2023)

    2023

  64. [64]

    NeurIPS36, 1363–1389 (2023)

    Tang, L., Jia, M., Wang, Q., Phoo, C.P., Hariharan, B.: Emergent correspondence from image diffusion. NeurIPS36, 1363–1389 (2023)

    2023

  65. [65]

    In: CVPR

    Taniai, T., Sinha, S.N., Sato, Y.: Joint recovery of dense correspondence and coseg- mentation in two images. In: CVPR. pp. 4246–4255 (2016)

    2016

  66. [66]

    Franca: Nested Matryoshka Clustering for Scalable Visual Representation Learning

    Venkataramanan, S., Pariza, V., Salehi, M., Knobel, L., Gidaris, S., Ramzi, E., Bursuc, A., Asano, Y.M.: Franca: Nested matryoshka clustering for scalable visual representation learning. arXiv preprint arXiv:2507.14137 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  67. [67]

    In: CVPR (2025)

    Wandel, K., Wang, H.: SemAlign3D: Semantic correspondence between RGB im- ages through aligning 3D object-class representations. In: CVPR (2025)

    2025

  68. [68]

    In: CVPR

    Wang, H., Sridhar, S., Huang, J., Valentin, J., Song, S., Guibas, L.J.: Normalized object coordinate space for category-level 6D object pose and size estimation. In: CVPR. pp. 2642–2651 (2019)

    2019

  69. [69]

    InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency

    Wang, W., Gao, Z., Gu, L., Pu, H., Cui, L., Wei, X., Liu, Z., Jing, L., Ye, S., Shao, J., et al.: InternVL3.5: Advancing open-source multimodal models in versatility, reasoning, and efficiency. arXiv preprint arXiv:2508.18265 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  70. [70]

    In: ICCV (2023)

    Weinzaepfel, P., Lucas, T., Leroy, V., Cabon, Y., Arora, V., Brégier, R., Csurka, G., Antsfeld, L., Chidlovskii, B., Revaud, J.: CroCo v2: Improved cross-view com- pletion pre-training for stereo matching and optical flow. In: ICCV (2023)

    2023

  71. [71]

    In: CVPR

    Wu, Y., Lim, J., Yang, M.H.: Online object tracking: A benchmark. In: CVPR. pp. 2411–2418 (2013)

    2013

  72. [72]

    Wu, Y., Kirillov, A., Massa, F., Lo, W.Y., Girshick, R.: Detectron2 (2019), software

    2019

  73. [73]

    In: WACV

    Xiang, Y., Mottaghi, R., Savarese, S.: Beyond PASCAL: A benchmark for 3D object detection in the wild. In: WACV. pp. 75–82 (2014) SOCO 23

    2014

  74. [74]

    In: ICCV

    Xu, J., Zhang, Y., Peng, J., Ma, W., Jesslen, A., Ji, P., Hu, Q., Zhang, J., Liu, Q., Wang, J., et al.: Animal3D: A comprehensive dataset of 3D animal pose and shape. In: ICCV. pp. 9099–9109 (2023)

    2023

  75. [75]

    Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces

    Yang, J., Yang, S., Gupta, A.W., Han, R., Fei-Fei, L., Xie, S.: Thinking in space: How multimodal large language models see, remember and recall spaces. arXiv preprint arXiv:2412.14171 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  76. [76]

    arXiv preprint arXiv:2512.15715 (2025)

    Yang, L., Li, S.W., Li, Y., Lei, X., Wang, D., Mohamed, A., Zhao, H., Xu, H.: In pursuit of pixel supervision for visual pre-training. arXiv preprint arXiv:2512.15715 (2025)

    work page arXiv 2025

  77. [77]

    arXiv preprint arXiv:2108.12617 (2021)

    Yu, H., Xu, Y., Zhang, J., Zhao, W., Guan, Z., Tao, D.: AP-10K: A benchmark for animal pose estimation in the wild. arXiv preprint arXiv:2108.12617 (2021)

    work page arXiv 2021

  78. [78]

    In: CVPR (2024)

    Yue, X., Ni, Y., Zhang, K., Zheng, T., Liu, R., Zhang, G., Stevens, S., Jiang, D., Ren, W., Sun, Y., Wei, C., Yu, B., Yuan, R., Sun, R., Yin, M., Zheng, B., Yang, Z., Liu, Y., Huang, W., Sun, H., Su, Y., Chen, W.: MMMU: A massive multi-discipline multimodal understanding and reasoning benchmark for expert AGI. In: CVPR (2024)

    2024

  79. [79]

    In: CVPR

    Zhang, J., Herrmann, C., Hur, J., Chen, E., Jampani, V., Sun, D., Yang, M.H.: Telling left from right: Identifying geometry-aware semantic correspondence. In: CVPR. pp. 3076–3085 (2024)

    2024

  80. [80]

    NeurIPS36, 45533–45547 (2023)

    Zhang, J., Herrmann, C., Hur, J., Polania Cabrera, L., Jampani, V., Sun, D., Yang, M.H.: A tale of two features: Stable Diffusion complements DINO for zero-shot semantic correspondence. NeurIPS36, 45533–45547 (2023)

    2023

Showing first 80 references.