Pith. sign in

REVIEW 3 major objections 5 minor 105 references

Weakly-Supervised Learning of Dense Functional Correspondences

T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Dense functional correspondence across object categories can be learned with weak supervision.

desk verdict Defines a new task with useful benchmarks and a solid weakly-supervised method, but the synthetic evaluation may have a train/eval asset overlap that needs checking. read the letter →

arxiv 2509.03893 v1 pith:6Z624KOW submitted 2025-09-04 cs.CV

Stefan Stojanov , Linan Zhao , Yunzhi Zhang , Daniel L. K. Yamins , Jiajun Wu This is my paper
classification cs.CV
keywords densecorrespondencefunctionalweaklysupervisedlearningcontrastivevision-languagepseudo-labelscross-categorymatching3Dobjectalignmentaffordancegrounding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that dense pixel-level correspondence between objects of different categories—e.g., matching a kettle's spout to a bottle's neck for pouring—can be defined and learned without manual dense labeling. The authors derive the task from 3D alignment of functionally equivalent parts, then train a model by combining pseudo-labeled functional parts from a vision-language model with multi-view contrastive learning that preserves spatial structure. They build synthetic and real benchmarks for evaluation and report that the trained model outperforms off-the-shelf self-supervised features and vision-language grounding baselines on both. If correct, the result matters for applications like transferring a demonstrated manipulation from one object to another.

What carries the argument

The load-bearing mechanism is a function-conditioned dense feature extractor: frozen self-supervised image features are combined at each pixel with a text embedding of the function, then passed through a small MLP. Two contrastive objectives shape the feature space. The functional-part contrastive loss distills semantic knowledge from the vision-language pseudo-labels by pulling pixels on matching functional parts together and pushing other pixels apart. The spatial multi-view contrastive loss prevents mode collapse by requiring a pixel to match its projection in a second view of the same object and to differ from all other pixels, preserving the internal geometry of a part such as the tip v

What would settle it

Run an inter-annotator study in which several annotators independently align each object pair; if their dense correspondences disagree on a substantial fraction of pairs, then the ground-truth benchmark and reported gains would be tied to one particular alignment convention.

Watch

Extended reading notes

Core claim

The central claim is that object function—the effect an object can cause on another object or substance—provides a reliable guide for dense correspondence across categories, because parts that carry out the same function tend to be similar in shape and appearance even when whole objects are visually different. The paper formalizes dense functional correspondence as a pixel-level mapping induced by aligning the functional parts of two 3D objects in space, and proposes a weakly supervised training scheme: use an off-the-shelf grounded vision-language model to propose functional-part bounding boxes on multi-view renders, aggregate those proposals into 3D pseudo-labels, then train a function-con

Load-bearing premise

The whole evaluation rests on the assumption that one human 3D alignment of functional parts, such as a kettle spout against a bottle neck, gives the unambiguous ground truth for dense pixel correspondence; the paper notes this can be ambiguous for multi-functional parts or radially symmetric objects.

Editorial extensions

If this is right

  • A model can transfer a functional-part label from one object category to another without ever seeing human-annotated correspondence pairs.
  • Because training uses rendered synthetic images, the resulting representation generalizes to real photographs of unseen objects.
  • The pseudo-labeling pipeline converts noisy vision-language bounding-box proposals into spatially consistent pixel-level part masks by aggregating over many views in 3D.
  • Correspondence discovery—finding which pixels correspond without being given either part mask—becomes feasible, a capability needed for aligning a robot's actions to a new object.
  • The method's performance improves with larger backbones and higher spatial resolution, suggesting headroom from scaling.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension would be to condition on finer descriptions than a single function, such as 'pour-with while keeping the container upright,' to resolve the ambiguities the paper acknowledges.
  • The same vision-language pseudo-label plus 3D aggregation recipe could be applied to functions that are not object-part based, such as those involving object-object or object-substance interactions, as long as prompts can be written.
  • The learned feature space could be used as a reward or cost signal for motion planning, not just for matching, because it encodes functionally equivalent alignment.
  • The benchmark construction implies that inter-annotator agreement on functional alignment should be measured; if agreement is low, a probabilistic correspondence model would be needed.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces the task of dense functional correspondence: establishing dense pixel-level matches between images of objects from different categories whose parts support the same function (e.g., kettle spout to bottle neck). The authors propose a weakly-supervised training pipeline that (i) uses GPT-4 to build a function/object taxonomy, (ii) pseudo-labels functional parts on multi-view renders of Objaverse assets with CogVLM bounding boxes aggregated on 3D point clouds, and (iii) trains a function-conditioned MLP on top of frozen DINOv2/CLIP features with two contrastive losses—a functional part loss and a multi-view spatial loss—plus an optional mask loss. For evaluation, they define ground truth by aligning 3D object pairs in functionally equivalent poses and derive dense 2D correspondences; they curate 950 synthetic Objaverse pairs and 190 real HANDAL pairs. Experiments compare against DINOv2, Stable Diffusion, and VLM-grounded baselines (CogVLM+DINO, ManipVQA+DINO), reporting consistent improvements on label transfer and correspondence discovery metrics.

Significance. If the results hold, the paper makes a useful contribution: it gives a clean, actionable definition of dense functional correspondence, a scalable pseudo-labeling recipe that combines VLM semantic grounding with self-supervised spatial features, and two new benchmarks. The method is clearly specified, the grounding of pseudo-labels in an external VLM avoids the most obvious circularity in training, and the ablations (functional-only, spatial-only, mask loss, backbone scaling, loss-weight sensitivity) are extensive. The main uncertainty is not the method itself but the validity of the synthetic benchmark and the benchmark curation process, which the authors partly acknowledge. The real HANDAL evaluation provides some external anchor, but the headline synthetic numbers are the strongest quantitative claim.

major comments (3)
  1. [§3.2 and §4.1] The synthetic evaluation set may overlap with the training set, and this is load-bearing for the main claim. Section 4.1 states that 8,285 curated assets are used, with 80% for training, but Section 3.2 says the 950 evaluation pairs are 'assets from Objaverse selected for quality and diversity' and refers to Section 4.1 for selection without stating that the evaluation assets are held out from the training split. Since the spatial contrastive loss (Eq. 2) is applied across rendered views of the same training assets, the model could memorize object-specific geometry, inflating both label-transfer and discovery metrics on the synthetic set. Please state explicitly whether the 950 pairs are drawn exclusively from the held-out 20% (or otherwise exclude all training assets), and if not, report the evaluation on a disjoint held-out split. This is a concrete, checkable methodological requiremen
  2. [§9.3, Table 4] Hyperparameters λspatial and λmask appear to be selected on the same benchmark used for final reporting. Table 4 sweeps λspatial ∈ {1,5,10} and λmask ∈ {1,5,10} and reports test metrics on the synthetic and real evaluation sets, with λspatial=10 chosen as 'best result.' No validation split is described. If these choices were made by looking at the evaluation set, the reported numbers are optimistic. Please describe the model-selection protocol (e.g., a separate validation subset, or a fixed choice made before evaluation).
  3. [§8.1 and §11] The ground-truth annotation procedure involves manual disambiguation and filtering. The appendix says ambiguity is resolved 'based on the objects' orientation' and that annotations are 'manually filter[ed] ... to disambiguate and ensure high quality,' and the Limitations section concedes that radial symmetry and multiple functional regions make the task ambiguous. This is reasonable for a first benchmark, but the evaluation is partly defined by the annotators' judgment. Please report inter-annotator agreement or at least quantify how many pairs were filtered and whether the reported metrics are stable under alternative disambiguation choices. Without this, it is hard to know how much of the measured performance reflects the model versus the annotation protocol.
minor comments (5)
  1. [Table 1] The row labels 'DINO' and 'DINOv2' are used inconsistently; the text cites DINOv2 but the table says DINO. Please unify.
  2. [§5.1] The definition of PCK@k is missing an indicator function: it should read 1[||p2^{j(i)} - p2^i|| < k], not the expression as written.
  3. [Eq. (2)] The negative set for the spatial contrastive loss is described only as 'any other pixel on I'. Please clarify whether negatives are sampled uniformly, how many are used, and whether hard-negative mining is applied; this affects reproducibility.
  4. [§4.2 and Table 1] The ablation labeled 'Ours (functional only)' underperforms DINOv2 on several metrics. The text says the full model 'consistently outperforms,' but the functional-only variant does not; please qualify the claim to refer to the full model.
  5. [§5.2] ManipVQA is described as a 7B model, while CogVLM is described as 17B. It would be useful to note the parameter-count gap when interpreting the comparison, since the baselines have very different capacities.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: human-annotated evaluation is independent of the VLM pseudo-label training loop, with only a minor non-load-bearing self-citation.

full rationale

The paper's derivation chain is not circular. The task definition (Sec. 3.1) is a formal definition based on 3D functional alignment, and the evaluation benchmarks (Sec. 3.2) apply that definition through human 3D alignments to obtain ground truth. Training uses external VLM pseudo-labels (GPT-4, CogVLM) and multi-view geometry, while evaluation uses human-annotated correspondences, so the reported predictions are not equal to the training inputs by construction. The only self-citation is [68] for general multi-view correspondence, and it is not load-bearing: the spatial contrastive loss (Eq. 2) relies on known render geometry to identify corresponding pixels, not on learned descriptors from [68]. Two experimental-validity concerns should be flagged but do not constitute circularity: (i) the synthetic evaluation assets are selected via the same curation pipeline as training assets (Sec. 3.2 vs. Sec. 4.1), and the paper never states that the evaluation pairs are disjoint from the 80% training split, so synthetic generalization may be optimistically biased; (ii) loss weights are selected using the same evaluation sets reported in Table 1 (Sec. 9.3), introducing test-set tuning bias. Neither concern makes a prediction reduce to an input by construction, and the real HANDAL benchmark is externally sourced and partially mitigates these issues.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The method is empirical; its success depends on the quality of external VLM pseudo-labels, the suitability of DINOv2 features, and the validity of the human-annotated 3D alignments. These are domain assumptions. The loss weights and sampling choices are free parameters. No invented entities are proposed.

free parameters (5)
  • lambda_spatial = 10
    Weight for the multi-view contrastive loss; chosen via sensitivity analysis and reported as best on the evaluation sets (Section 9.3).
  • lambda_mask = 1
    Weight for the binary functional part mask loss; stable across values (Section 9.3).
  • temperature_tau = not reported
    Temperature in the contrastive losses (Equations 1 and 2) is not specified; affects feature sharpness.
  • num_sampled_points = 128
    Number of positive and negative points sampled per image for contrastive learning (Section 4.3).
  • num_views_and_trials = 19 views x 4 trials
    Heuristics for pseudo-labeling with CogVLM; more views/trials improve consistency but cost compute (Section 7.4).
assumptions (5)
  • domain assumption Form follows function: parts enabling the same function share shape/appearance consistency across objects.
    Central premise behind the definition of dense functional correspondence (Section 3).
  • domain assumption CogVLM can produce sufficiently accurate functional part bounding boxes for pseudo-labeling.
    The training pipeline relies on these labels; some validation via segmentation IoU in Table 5 (Section 4.1).
  • domain assumption Human 3D alignment of functional parts yields valid ground-truth correspondences despite acknowledged ambiguity.
    Evaluation benchmarks depend on this; limitations are discussed in Section 11.
  • domain assumption DINOv2 features provide a spatial prior sufficient for cross-category structural generalization after training.
    The method builds on frozen DINOv2 features and a small MLP (Section 4.2).
  • domain assumption Multi-view contrastive learning prevents modal collapse and preserves structural detail within functional parts.
    Motivates Lspatial in Section 4.2; supported by ablations.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Weakly-Supervised Learning of Dense Functional Correspondences." pith.science (2026). https://pith.science/paper/6Z624KOW

@misc{pith2026250903893,
  author       = {Pith},
  title        = {Pith review of: Weakly-Supervised Learning of Dense Functional Correspondences},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6Z624KOW}},
  note         = {Machine review of arXiv:2509.03893}
}
read the original abstract

Establishing dense correspondences across image pairs is essential for tasks such as shape reconstruction and robot manipulation. In the challenging setting of matching across different categories, the function of an object, i.e., the effect that an object can cause on other objects, can guide how correspondences should be established. This is because object parts that enable specific functions often share similarities in shape and appearance. We derive the definition of dense functional correspondence based on this observation and propose a weakly-supervised learning paradigm to tackle the prediction task. The main insight behind our approach is that we can leverage vision-language models to pseudo-label multi-view images to obtain functional parts. We then integrate this with dense contrastive learning from pixel correspondences to distill both functional and spatial knowledge into a new model that can establish dense functional correspondence. Further, we curate synthetic and real evaluation datasets as task benchmarks. Our results demonstrate the advantages of our approach over baseline solutions consisting of off-the-shelf self-supervised image representations and grounded vision language models.

Figures

Figures reproduced from arXiv: 2509.03893 by the authors.

Figure 1
Figure 1. Dense Functional Correspondence refers to establishing dense correspondences across object instances based on function sim￾ilarity (e.g., “pour-with”). This task is especially challenging when objects have visually different but functionally similar parts, requiring both semantic understanding, i.e., identifying which parts can perform the same function, and structural understanding, i.e., establishing dense corresp… view at source ↗
Figure 2
Figure 2. Annotation Pipeline (Evaluation Only). Given a 3D object pair (left) and a function (“pour-with”), we annotate the functional alignment of two objects by aligning the functional parts in 3D (middle). Afterward, we derive dense 2D correspon￾dences (right) based on 3D distances of corresponding object sur￾face points, with matching pixels shown in the same color. ally equivalent parts, which is central to robotic appl… view at source ↗
Figure 3
Figure 3. Training Data Curation via VLM Pseudo Labeling. Given a large unstructured dataset like Objaverse [7], we leverage off￾the-shelf VLMs to curate and label the functional parts. Specifically, GPT-4 [55] generates category-specific functional part prompts, and CogVLM [82] produces bounding box proposals for multi-view image renderings, which are aggregated onto a 3D point cloud. The point cloud is post-processed to pro… view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Local Functional Feature Extraction. To obtain dense functionally conditioned features, we apply an MLP on top of a function text embedding and the spatial DINO features. The MLP is trained with both functional and spatial contrastive losses. tion “pour-with,” the feat…
Figure 5
Figure 5. Figure 5: Training Objectives. To ensure functional part simi￾larity in the learned feature space, we use a part-level contrastive objective to distill functional part semantics from VLMs (left). The spatial contrastive loss (right) serves a complementary role and prevents the m…
Figure 6
Figure 6. Figure 6: Correspondence Discovery Comparisons. We observe that our approach more reliably retrieves the functionally relevant correspondences than off-the-shelf DINOv2. The top 10 highest-ranked matches are shown. • ManipVQA [25], an affordance-grounding model that outputs boun…
Figure 7
Figure 7. Figure 7: Examples of pseudo-labeled functional parts in point clouds and images using CogVLM [82]. Using the procedure outlined in Section 7.4, we pseudo-label images with masks for the object functional parts. Notably, this pipeline has the ability to generate part labels for …
Figure 8
Figure 8. Figure 8: Additional Label Transfer Dense Visualizations. For each target image (right), our model predicts the functional part mask. To generate the transferred color map, each pixel in the predicted mask is matched to its best corresponding pixel within the ground-truth mask o…
Figure 9
Figure 9. Figure 9: Qualitative examples for the impact of mask loss. Functional part predictions can help avoid incorrect matches out￾side the functional parts in correspondence discovery [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]
Figure 10
Figure 10. Figure 10: Comparison with [38] A visual comparison of dense correspondence between [38] (left) and our method (right). for dense correspondence, the method is not designed for this and it qualitatively appears to be relatively coarse. A visual comparison is provided in [PITH_F…
Figure 11
Figure 11. Figure 11: Additional Correspondence Discovery Results on Objaverse Evaluation Dataset. We show more qualitative examples of correspondence discovery on the synthetic Objaverse evaluation dataset, comparing our model against baselines [PITH_FULL_IMAGE:figures/full_fig_p020_11.png]
Figure 12
Figure 12. Figure 12: Additional Correspondence Discovery Results on HANDAL Evaluation Dataset [PITH_FULL_IMAGE:figures/full_fig_p020_12.png]
Figure 14
Figure 14. Figure 14: Overview of our strategy for prompting GPT-4 [ [PITH_FULL_IMAGE:figures/full_fig_p021_14.png]
Figure 13
Figure 13. Figure 13: Our GPT-4 [55] prompting procedure for creating ob￾ject categories given a list of functions. System Prompt: Here are your instructions for the rest of the chat: Respond as if you are a human expert giving simplifying instructions to a robot learning to interact with …

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

105 extracted references · 63 canonical work pages

  1. [1]

    Deep vit features as dense visual descriptors

    Shir Amir, Yossi Gandelsman, Shai Bagon, and Tali Dekel. Deep vit features as dense visual descriptors. arXiv preprint arXiv:2112.05814, 2021. 3

    arXiv 2021

  2. [2]

    Contactdb: Analyzing and predicting grasp contact via thermal imaging

    Samarth Brahmbhatt, Cusuh Ham, Charles C Kemp, and James Hays. Contactdb: Analyzing and predicting grasp contact via thermal imaging. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8709–8719, 2019. 2

    2019

  3. [3]

    Emerg- ing properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. In Pro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 9650–9660, 2021. 3

    2021

  4. [4]

    A simple framework for contrastive learning of visual representations

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Ge- offrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on ma- chine learning, pages 1597–1607. PMLR, 2020. 5

    2020

  5. [5]

    Learning to act properly: Predicting and explaining affordances from images

    Ching-Yao Chuang, Jiaman Li, Antonio Torralba, and Sanja Fidler. Learning to act properly: Predicting and explaining affordances from images. In Proceedings of the IEEE Con- ference on Computer Vision and Pattern Recognition, pages 975–983, 2018. 2

    2018

  6. [6]

    Blender - a 3D modelling and rendering package

    Blender Online Community. Blender - a 3D modelling and rendering package. Blender Foundation, Stichting Blender Foundation, Amsterdam, 2018. 4

    2018

  7. [7]

    Objaverse: A universe of annotated 3d objects

    Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13142–13153, 2023. 2, 4, 13, 16

    2023

  8. [8]

    Molmo and pixmo: Open weights and open data for state-of-the-art multimodal models

    Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tri- pathi, Yue Yang, Jae Sung Park, Mohammadreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, et al. Molmo and pixmo: Open weights and open data for state-of-the-art multimodal models. arXiv preprint arXiv:2409.17146, 2024. 5

    arXiv 2024

Show all 105 references
  1. [9]

    Objaverse-xl: A universe of 10m+ 3d objects

    Matt Deitke, Ruoshi Liu, Matthew Wallingford, Huong Ngo, Oscar Michel, Aditya Kusupati, Alan Fan, Christian Laforte, Vikram V oleti, Samir Yitzhak Gadre, et al. Objaverse-xl: A universe of 10m+ 3d objects. Advances in Neural Informa- tion Processing Systems, 36, 2024. 5

    2024

  2. [10]

    3d affordancenet: A benchmark for visual object af- fordance understanding

    Shengheng Deng, Xun Xu, Chaozheng Wu, Ke Chen, and Kui Jia. 3d affordancenet: A benchmark for visual object af- fordance understanding. In proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 1778–1787, 2021. 2

    2021

  3. [11]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018. 3

    2018 arXiv

  4. [12]

    On the effectiveness of retrieval, alignment, and replay in manipulation

    Norman Di Palo and Edward Johns. On the effectiveness of retrieval, alignment, and replay in manipulation. IEEE Robotics and Automation Letters, 2024. 3

    2024

  5. [13]

    Preafford: Universal affordance- based pre-grasping for diverse objects and environments

    Kairui Ding, Boyuan Chen, Ruihai Wu, Yuyang Li, Zongzheng Zhang, Huan-ang Gao, Siqi Li, Guyue Zhou, Yixin Zhu, Hao Dong, et al. Preafford: Universal affordance- based pre-grasping for diverse objects and environments. arXiv preprint arXiv:2404.03634, 2024. 3

    2024 arXiv

  6. [14]

    Affordancenet: An end-to-end deep learning approach for object affordance detection

    Thanh-Toan Do, Anh Nguyen, and Ian Reid. Affordancenet: An end-to-end deep learning approach for object affordance detection. In 2018 IEEE international conference on robotics and automation (ICRA), pages 5882–5889. IEEE, 2018. 2

    2018

  7. [15]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Ab- hishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 ,

    arXiv

  8. [16]

    Graspnet-1billion: A large-scale benchmark for general ob- ject grasping

    Hao-Shu Fang, Chenxi Wang, Minghao Gou, and Cewu Lu. Graspnet-1billion: A large-scale benchmark for general ob- ject grasping. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 11444– 11453, 2020. 2

    2020

  9. [17]

    Dense object nets: Learning dense visual object descriptors by and for robotic manipulation

    Peter R Florence, Lucas Manuelli, and Russ Tedrake. Dense object nets: Learning dense visual object descriptors by and for robotic manipulation. In Conference on Robot Learning, pages 373–385. PMLR, 2018. 1, 2, 6

    2018

  10. [18]

    The ecological approach to visual percep- tion: Classic edition

    James J Gibson. The ecological approach to visual percep- tion: Classic edition. Taylor & Francis, 1979. 2

    1979

  11. [19]

    Videoswap: Customized video subject swapping with interactive semantic point cor- respondence

    Yuchao Gu, Yipin Zhou, Bichen Wu, Licheng Yu, Jia-Wei Liu, Rui Zhao, Jay Zhangjie Wu, David Junhao Zhang, Mike Zheng Shou, and Kevin Tang. Videoswap: Customized video subject swapping with interactive semantic point cor- respondence. In Proceedings of the IEEE/CVF Conference o...

    2024

  12. [20]

    Handal: A dataset of real-world manipulable object categories with pose annotations, affordances, and reconstructions

    Andrew Guo, Bowen Wen, Jianhe Yuan, Jonathan Tremblay, Stephen Tyree, Jeffrey Smith, and Stan Birchfield. Handal: A dataset of real-world manipulable object categories with pose annotations, affordances, and reconstructions. In 2023 IEEE/RSJ International Conference on Intelli...

    2023

  13. [21]

    Hdri haven

    HDRI Haven. Hdri haven. https://hdri-haven.com,

  14. [22]

    Momentum contrast for unsupervised visual rep- resentation learning

    Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual rep- resentation learning. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 9729–9738, 2020. 5

    2020

  15. [23]

    Ditto: Demonstration imitation by trajectory transformation

    Nick Heppert, Max Argus, Tim Welschehold, Thomas Brox, and Abhinav Valada. Ditto: Demonstration imitation by trajectory transformation. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE,

    2024

  16. [24]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models, 2021. 14

    2021

  17. [25]

    Manipvqa: Injecting robotic affordance and physi- cally grounded information into multi-modal large language models

    Siyuan Huang, Iaroslav Ponomarenko, Zhengkai Jiang, Xi- aoqi Li, Xiaobin Hu, Peng Gao, Hongsheng Li, and Hao Dong. Manipvqa: Injecting robotic affordance and physi- cally grounded information into multi-modal large language models. arXiv preprint arXiv:2403.11289, 2024. 1, 3, ...

    2024 arXiv

  18. [26]

    Flowformer: A transformer architecture for optical flow

    Zhaoyang Huang, Xiaoyu Shi, Chao Zhang, Qiang Wang, Ka Chun Cheung, Hongwei Qin, Jifeng Dai, and Hongsheng Li. Flowformer: A transformer architecture for optical flow. In European conference on computer vision, pages 668–685. Springer, 2022. 2

    2022

  19. [27]

    Flownet 2.0: Evolu- tion of optical flow estimation with deep networks

    Eddy Ilg, Nikolaus Mayer, Tonmoy Saikia, Margret Keuper, Alexey Dosovitskiy, and Thomas Brox. Flownet 2.0: Evolu- tion of optical flow estimation with deep networks. In Pro- ceedings of the IEEE conference on computer vision and pat- tern recognition, pages 2462–2470, 2017. 2

    2017

  20. [28]

    Cotr: Correspondence transformer for matching across images

    Wei Jiang, Eduard Trulls, Jan Hosang, Andrea Tagliasacchi, and Kwang Moo Yi. Cotr: Correspondence transformer for matching across images. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6207– 6217, 2021. 2

    2021

  21. [29]

    Doduo: Learning dense visual correspondence from unsupervised semantic-aware flow

    Zhenyu Jiang, Hanwen Jiang, and Yuke Zhu. Doduo: Learning dense visual correspondence from unsupervised semantic-aware flow. In 2024 IEEE International Confer- ence on Robotics and Automation (ICRA) , pages 12420– 12427. IEEE, 2024. 1

    2024

  22. [30]

    Robo-abc: Affordance gener- alization beyond categories via semantic correspondence for robot manipulation

    Yuanchen Ju, Kaizhe Hu, Guowei Zhang, Gu Zhang, Min- grun Jiang, and Huazhe Xu. Robo-abc: Affordance gener- alization beyond categories via semantic correspondence for robot manipulation. arXiv preprint arXiv:2401.07487, 2024. 1, 3

    2024 arXiv

  23. [31]

    Segment anything in high qual- ity

    Lei Ke, Mingqiao Ye, Martin Danelljan, Yu-Wing Tai, Chi- Keung Tang, Fisher Yu, et al. Segment anything in high qual- ity. Advances in Neural Information Processing Systems, 36,

  24. [32]

    Two-year-olds will name artifacts by their func- tions

    Deborah G Kemler Nelson, Rachel Russell, Nell Duke, and Kate Jones. Two-year-olds will name artifacts by their func- tions. Child development, 71(5):1271–1288, 2000. 3

    2000

  25. [33]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2017. 6

    2017

  26. [34]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 4015–4026, 2023. 5

    2023

  27. [35]

    Omninocs: A unified nocs dataset and model for 3d lifting of 2d objects

    Akshay Krishnan, Abhijit Kundu, Kevis-Kokitsi Maninis, James Hays, and Matthew Brown. Omninocs: A unified nocs dataset and model for 3d lifting of 2d objects. arXiv preprint arXiv:2407.08711, 2024. 1, 2

    2024 arXiv

  28. [36]

    Ram: Retrieval-based affordance transfer for gen- eralizable zero-shot robotic manipulation

    Yuxuan Kuang, Junjie Ye, Haoran Geng, Jiageng Mao, Congyue Deng, Leonidas Guibas, He Wang, and Yue Wang. Ram: Retrieval-based affordance transfer for gen- eralizable zero-shot robotic manipulation. arXiv preprint arXiv:2407.04689, 2024. 3

    2024 arXiv

  29. [37]

    Ge- ometry and context for semantic correspondences and func- tionality recognition in man-made 3d shapes

    Hamid Laga, Michela Mortara, and Michela Spagnuolo. Ge- ometry and context for semantic correspondences and func- tionality recognition in man-made 3d shapes. ACM Transac- tions on Graphics (TOG), 32(5):1–16, 2013. 3

    2013

  30. [38]

    The functional correspondence problem

    Zihang Lai, Senthil Purushwalkam, and Abhinav Gupta. The functional correspondence problem. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 15772–15781, 2021. 1, 2, 3, 4, 13, 18, 19

    2021

  31. [39]

    Object shape, object function, and object name

    Barbara Landau, Linda Smith, and Susan Jones. Object shape, object function, and object name. Journal of mem- ory and language, 38(1):1–27, 1998. 3

    1998

  32. [40]

    Corresnerf: Image correspondence priors for neural radiance fields

    Yixing Lao, Xiaogang Xu, Xihui Liu, Hengshuang Zhao, et al. Corresnerf: Image correspondence priors for neural radiance fields. Advances in Neural Information Processing Systems, 36:40504–40520, 2023. 1

    2023

  33. [41]

    Locate: Localize and transfer object parts for weakly supervised affordance grounding

    Gen Li, Varun Jampani, Deqing Sun, and Laura Sevilla- Lara. Locate: Localize and transfer object parts for weakly supervised affordance grounding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10922–10931, 2023. 2

    2023

  34. [42]

    Learning precise affordances from egocentric videos for robotic manipulation

    Gen Li, Nikolaos Tsagkas, Jifei Song, Ruaridh Mon- Williams, Sethu Vijayakumar, Kun Shao, and Laura Sevilla- Lara. Learning precise affordances from egocentric videos for robotic manipulation. arXiv preprint arXiv:2408.10123,

  35. [43]

    Surface and edge detection for primitive fitting of point clouds

    Yuanqi Li, Shun Liu, Xinran Yang, Jianwei Guo, Jie Guo, and Yanwen Guo. Surface and edge detection for primitive fitting of point clouds. In ACM SIGGRAPH 2023 conference proceedings, pages 1–10, 2023. 14

    2023

  36. [44]

    Sphinx: The joint mixing of weights, tasks, and visual embeddings for multi-modal large language models

    Ziyi Lin, Chris Liu, Renrui Zhang, Peng Gao, Longtian Qiu, Han Xiao, Han Qiu, Chen Lin, Wenqi Shao, Keqin Chen, et al. Sphinx: The joint mixing of weights, tasks, and visual embeddings for multi-modal large language models. arXiv preprint arXiv:2311.07575, 2023. 3, 5

    2023 arXiv

  37. [45]

    Pixel-perfect structure-from- motion with featuremetric refinement

    Philipp Lindenberger, Paul-Edouard Sarlin, Viktor Lars- son, and Marc Pollefeys. Pixel-perfect structure-from- motion with featuremetric refinement. In Proceedings of the IEEE/CVF international conference on computer vision, pages 5987–5997, 2021. 1

    2021

  38. [46]

    Learning affordance grounding from exocen- tric images

    Hongchen Luo, Wei Zhai, Jing Zhang, Yang Cao, and Dacheng Tao. Learning affordance grounding from exocen- tric images. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2252–2261,

  39. [47]

    Scalable 3d captioning with pretrained models

    Tiange Luo, Chris Rockwell, Honglak Lee, and Justin John- son. Scalable 3d captioning with pretrained models. Ad- vances in Neural Information Processing Systems, 36, 2024. 5, 13, 21

    2024

  40. [48]

    Tracknerf: Bundle adjusting nerf from sparse and noisy views via feature tracks

    Jinjie Mai, Wenxuan Zhu, Sara Rojas, Jesus Zarzar, Abdul- lah Hamdi, Guocheng Qian, Bing Li, Silvio Giancola, and Bernard Ghanem. Tracknerf: Bundle adjusting nerf from sparse and noisy views via feature tracks. arXiv preprint arXiv:2408.10739, 2024. 1

    2024 arXiv

  41. [49]

    Spair-71k: A large-scale benchmark for semantic correspon- dence

    Juhong Min, Jongmin Lee, Jean Ponce, and Minsu Cho. Spair-71k: A large-scale benchmark for semantic correspon- dence. arXiv preprint arXiv:1908.10543, 2019. 2, 6

    1908 arXiv

  42. [50]

    6-dof graspnet: Variational grasp generation for object manipula- tion

    Arsalan Mousavian, Clemens Eppner, and Dieter Fox. 6-dof graspnet: Variational grasp generation for object manipula- tion. In Proceedings of the IEEE/CVF international confer- ence on computer vision, pages 2901–2910, 2019. 2

    2019

  43. [51]

    Same object, differ- ent grasps: Data and semantic knowledge for task-oriented grasping

    Adithyavairavan Murali, Weiyu Liu, Kenneth Marino, So- nia Chernova, and Abhinav Gupta. Same object, differ- ent grasps: Data and semantic knowledge for task-oriented grasping. In Conference on robot learning , pages 1540–

  44. [52]

    Affordance detection of tool parts from ge- ometric features

    Austin Myers, Ching L Teo, Cornelia Ferm ¨uller, and Yian- nis Aloimonos. Affordance detection of tool parts from ge- ometric features. In 2015 IEEE International Conference on Robotics and Automation (ICRA) , pages 1374–1381. IEEE,

    2015

  45. [53]

    Grounded human-object interaction hotspots from video

    Tushar Nagarajan, Christoph Feichtenhofer, and Kristen Grauman. Grounded human-object interaction hotspots from video. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 8688–8697, 2019. 2

    2019

  46. [54]

    Caldwell, and Nikos G

    Anh Nguyen, Dimitrios Kanoulas, Darwin G. Caldwell, and Nikos G. Tsagarakis. Object-based affordances detection with convolutional neural networks and dense conditional random fields. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5908–5915,

    2017

  47. [55]

    OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anad- kat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Bal- com, Paul Baltescu, Haiming Bao, Mohammad Bavarian, J...

    2024

  48. [56]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 1, 2, ...

    2023 arXiv

  49. [57]

    A threshold selection method from gray-level histograms

    Nobuyuki Otsu et al. A threshold selection method from gray-level histograms. Automatica, 11(285-296):23–27,

  50. [58]

    Dinobot: Robot ma- nipulation via retrieval and alignment with vision foundation models

    Norman Di Palo and Edward Johns. Dinobot: Robot ma- nipulation via retrieval and alignment with vision foundation models. In IEEE International Conference on Robotics and Automation (ICRA), 2024. 1, 3

    2024

  51. [59]

    Miles: Making imitation learning easy with self-supervision

    Georgios Papagiannis and Edward Johns. Miles: Making imitation learning easy with self-supervision. InProceedings of the Conference on Robot Learning (CoRL), 2024. 3

    2024

  52. [60]

    Film: Visual reasoning with a general conditioning layer, 2017

    Ethan Perez, Florian Strub, Harm de Vries, Vincent Du- moulin, and Aaron Courville. Film: Visual reasoning with a general conditioning layer, 2017. 14

    2017

  53. [61]

    Affordancellm: Grounding affordance from vision language models

    Shengyi Qian, Weifeng Chen, Min Bai, Xiong Zhou, Zhuowen Tu, and Li Erran Li. Affordancellm: Grounding affordance from vision language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7587–7597, 2024. 3

    2024

  54. [62]

    Keto: Learning keypoint representations for tool manipulation

    Zengyi Qin, Kuan Fang, Yuke Zhu, Li Fei-Fei, and Silvio Savarese. Keto: Learning keypoint representations for tool manipulation. In 2020 IEEE International Conference on Robotics and Automation (ICRA) , pages 7278–7285. IEEE,

    2020

  55. [63]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  56. [64]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learn- ing Research, 21(140):1–67, 2020. 3

    2020

  57. [65]

    Com- mon objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction

    Jeremy Reizenstein, Roman Shapovalov, Philipp Henzler, Luca Sbordone, Patrick Labatut, and David Novotny. Com- mon objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction. In Proceedings of the IEEE/CVF international conference on computer vi...

    2021

  58. [66]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 1, 3, 7

    2022

  59. [67]

    Superglue: Learning feature matching with graph neural networks

    Paul-Edouard Sarlin, Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich. Superglue: Learning feature matching with graph neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4938–4947, 2020. 2

    2020

  60. [68]

    Learning dense object descriptors from multiple views for low-shot category generalization

    Stefan Stojanov, Anh Thai, Zixuan Huang, and James M Rehg. Learning dense object descriptors from multiple views for low-shot category generalization. Advances in Neural In- formation Processing Systems, 35:12566–12580, 2022. 2

    2022

  61. [69]

    Loftr: Detector-free local feature matching with transformers

    Jiaming Sun, Zehong Shen, Yuang Wang, Hujun Bao, and Xiaowei Zhou. Loftr: Detector-free local feature matching with transformers. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 8922–8931, 2021. 2

    2021

  62. [70]

    Misc210k: A large-scale dataset for multi-instance seman- tic correspondence

    Yixuan Sun, Yiwen Huang, Haijing Guo, Yuzhou Zhao, Run- min Wu, Yizhou Yu, Weifeng Ge, and Wenqiang Zhang. Misc210k: A large-scale dataset for multi-instance seman- tic correspondence. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, page...

    2023

  63. [71]

    Emergent correspondence from image diffusion

    Luming Tang, Menglin Jia, Qianqian Wang, Cheng Perng Phoo, and Bharath Hariharan. Emergent correspondence from image diffusion. Advances in Neural Information Pro- cessing Systems, 36:1363–1389, 2023. 3

    2023

  64. [72]

    Joint re- covery of dense correspondence and cosegmentation in two images

    Tatsunori Taniai, Sudipta N Sinha, and Yoichi Sato. Joint re- covery of dense correspondence and cosegmentation in two images. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4246–4255, 2016. 1

    2016

  65. [73]

    Raft: Recurrent all-pairs field transforms for optical flow

    Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23– 28, 2020, Proceedings, Part II 16, pages 402–419. Springer,

    2020

  66. [74]

    Shape-based transfer of generic skills

    Skye Thompson, Leslie Pack Kaelbling, and Tomas Lozano- Perez. Shape-based transfer of generic skills. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 5996–6002. IEEE, 2021. 3

    2021

  67. [75]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth´ee Lacroix, Baptiste Rozi`ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. 3

    2023 arXiv

  68. [76]

    Sparf: Neural radiance fields from sparse and noisy poses

    Prune Truong, Marie-Julie Rakotosaona, Fabian Manhardt, and Federico Tombari. Sparf: Neural radiance fields from sparse and noisy poses. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4190–4200, 2023. 1

    2023

  69. [77]

    Gift: Generalizable interaction- aware functional tool affordances without labels

    Dylan Turpin, Liquan Wang, Stavros Tsogkas, Sven Dick- inson, and Animesh Garg. Gift: Generalizable interaction- aware functional tool affordances without labels. arXiv preprint arXiv:2106.14973, 2021. 2

    2021 arXiv

  70. [78]

    Repre- sentation learning with contrastive predictive coding, 2019

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Repre- sentation learning with contrastive predictive coding, 2019. 6

    2019

  71. [79]

    Knowledge transfer from vision foundation models for efficient training of small task-specific models

    Raviteja Vemulapalli, Hadi Pouransari, Fartash Faghri, Sachin Mehta, Mehrdad Farajtabar, Mohammad Rastegari, and Oncel Tuzel. Knowledge transfer from vision foundation models for efficient training of small task-specific models. In Forty-first International Conference on Machi...

  72. [80]

    Socs: Semantically- aware object coordinate space for category-level 6d object pose estimation under large shape variations

    Boyan Wan, Yifei Shi, and Kai Xu. Socs: Semantically- aware object coordinate space for category-level 6d object pose estimation under large shape variations. InProceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 14065–14074, 2023. 2

    2023

  73. [81]

    Normalized object coordinate space for category-level 6d object pose and size estimation

    He Wang, Srinath Sridhar, Jingwei Huang, Julien Valentin, Shuran Song, and Leonidas J Guibas. Normalized object coordinate space for category-level 6d object pose and size estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages ...

    2019

  74. [82]

    Cogvlm: Visual expert for pretrained language models

    Weihan Wang, Qingsong Lv, Wenmeng Yu, Wenyi Hong, Ji Qi, Yan Wang, Junhui Ji, Zhuoyi Yang, Lei Zhao, Xixuan Song, et al. Cogvlm: Visual expert for pretrained language models. arXiv preprint arXiv:2311.03079, 2023. 1, 2, 3, 4, 5, 7, 14, 15, 16, 17, 18

    2023 arXiv

  75. [83]

    Form follows function: Learning about function helps children learn about shape

    Elizabeth A Ware and Amy E Booth. Form follows function: Learning about function helps children learn about shape. Cognitive Development, 25(2):124–137, 2010. 3

    2010

  76. [84]

    Omniobject3d: Large-vocabulary 3d object dataset for realistic perception, reconstruction and generation

    Tong Wu, Jiarui Zhang, Xiao Fu, Yuxin Wang, Liang Pan Jiawei Ren, Wayne Wu, Lei Yang, Jiaqi Wang, Chen Qian, Dahua Lin, and Ziwei Liu. Omniobject3d: Large-vocabulary 3d object dataset for realistic perception, reconstruction and generation. In IEEE/CVF Conference on Computer V...

    2023

  77. [85]

    An affordance keypoint detection network for robot manipulation

    Ruinian Xu, Fu-Jen Chu, Chao Tang, Weiyu Liu, and Patri- cio A Vela. An affordance keypoint detection network for robot manipulation. IEEE Robotics and Automation Letters, 6(2):2870–2877, 2021. 2

    2021

  78. [86]

    Depth anything: Unleashing the power of large-scale unlabeled data

    Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10371–10381, 2024. 5

    2024

  79. [87]

    Grounding 3d object affordance from 2d interactions in images

    Yuhang Yang, Wei Zhai, Hongchen Luo, Yang Cao, Jiebo Luo, and Zheng-Jun Zha. Grounding 3d object affordance from 2d interactions in images. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 10905–10915, 2023. 2

    2023

  80. [88]

    Mvimgnet: A large-scale dataset of multi-view images

    Xianggang Yu, Mutian Xu, Yidan Zhang, Haolin Liu, Chongjie Ye, Yushuang Wu, Zizheng Yan, Chenming Zhu, Zhangyang Xiong, Tianyou Liang, et al. Mvimgnet: A large-scale dataset of multi-view images. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognit...

    2023

  81. [89]

    Robopoint: A vision-language model for spatial affordance prediction for robotics

    Wentao Yuan, Jiafei Duan, Valts Blukis, Wilbert Pumacay, Ranjay Krishna, Adithyavairavan Murali, Arsalan Mousa- vian, and Dieter Fox. Robopoint: A vision-language model for spatial affordance prediction for robotics. arXiv preprint arXiv:2406.10721, 2024. 3

    2024 arXiv

  82. [90]

    Sigmoid loss for language image pre-training

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 11975–11986, 2023. 3

    2023

  83. [91]

    A tale of two features: Stable diffusion complements dino for zero-shot semantic correspondence, 2023

    Junyi Zhang, Charles Herrmann, Junhwa Hur, Luisa Pola- nia Cabrera, Varun Jampani, Deqing Sun, and Ming-Hsuan Yang. A tale of two features: Stable diffusion complements dino for zero-shot semantic correspondence, 2023. 1, 7, 15

    2023

  84. [92]

    Egoobjects: A large-scale egocentric dataset for fine-grained object understanding

    Chenchen Zhu, Fanyi Xiao, Andr ´es Alvarado, Yasmine Babaei, Jiabo Hu, Hichem El-Mohri, Sean Culatana, Roshan Sumbaly, and Zhicheng Yan. Egoobjects: A large-scale egocentric dataset for fine-grained object understanding. In Proceedings of the IEEE/CVF International Conference ...

    2023

  85. [93]

    Understanding tools: Task-oriented object modeling, learning and recogni- tion

    Yixin Zhu, Yibiao Zhao, and Song Chun Zhu. Understanding tools: Task-oriented object modeling, learning and recogni- tion. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2855–2864, 2015. 3

    2015

  86. [94]

    Vision- based manipulation from single human video with open- world object graphs

    Yifeng Zhu, Arisrei Lim, Peter Stone, and Yuke Zhu. Vision- based manipulation from single human video with open- world object graphs. arXiv preprint arXiv:2405.20321 ,

    arXiv

  87. [96]

    Given this list of functions, generate more options for object func- tions

    Training Data Generation 7.1. Function and Object Taxonomy List of functions. To obtain our taxonomy of functions, we first take the function lists defined by [38, 51] and ask GPT-4 [55] to expand them. Our prompt is a simple “Given this list of functions, generate more option...

  88. [97]

    Render 19 views per object that shows it from various angles, including from above and below

  89. [98]

    For small parts like points or tips, we do a second iteration that zooms into the initial bounding boxes to improve precision

    For each functional part description and each view, query CogVLM for four trials to obtain the bounding box pseudo-labels. For small parts like points or tips, we do a second iteration that zooms into the initial bounding boxes to improve precision

  90. [99]

    Ev- ery time a given point in the point cloud gets labeled by a bounding box in a different view, we increment its score

    Aggregate all trials and views onto a point cloud of 100K randomly sampled points on the object’s surface. Ev- ery time a given point in the point cloud gets labeled by a bounding box in a different view, we increment its score. The final numbers are normalized to be in 0-

  91. [100]

    For prompts that specifically ask for the labeling of edges, we multiply the point cloud with the per-point edge probabilities from SED-net [43], a method for de- composing point clouds into primitives

  92. [101]

    Example outputs of this procedure are shown in Figure 7

    Given this point cloud, for any rendered image of the ob- ject, we can project the point cloud into 2D and produce a binary mask with Otsu’s method [57] and a series of binary dilation/erosion steps to close holes in the mask. Example outputs of this procedure are shown in Figure 7

  93. [102]

    Ground-Truth Generation In this section, we provide additional details for deriv- ing ground-truth 2D dense correspondences from 3D align- ment

    Additional Training and Evaluation Details 8.1. Ground-Truth Generation In this section, we provide additional details for deriv- ing ground-truth 2D dense correspondences from 3D align- ment. Given two object meshes that can perform the same function, we obtain their 3D funct...

  94. [103]

    Additional Quantitative Results 9.1. Within- and Cross-Category Comparison Since the evaluation dataset contains both within-category pairs and across-category pairs, we further separate the metrics in Table 1 into within-category results and across- category results in Table ...

  95. [104]

    These results high- light the strong performance of our model in transferring functional part labels across diverse object categories

    Additional Qualitative Results Additional dense label transfer results on the synthetic Ob- javerse dataset, which further validate the effectiveness of our approach, are presented in Figure 8. These results high- light the strong performance of our model in transferring funct...

  96. [105]

    bottle” and a “kettle,

    Discussion Differences with FunKPoint [38]. The concept of func- tional correspondence was previously introduced by [38]. However, our formulation is different in three key aspects. First, our problem requires dense functional correspon- dences to be established, whereas [38] ...

  97. [1557]

    2, 4, 13

    PMLR, 2021. 2, 4, 13

    2021

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.