Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

O$^3$Afford: One-Shot 3D Object-to-Object Affordance Grounding for Generalizable Robotic Manipulation

T0 review · 4 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper argues that a robot can learn where and how two objects should interact—per-point on both 3D point clouds—from a single annotated example of each interaction type, and that these affordance maps improve LLM-generated manipulation

desk verdict A solid one-shot object-to-object affordance system with real robot demos, but the quantitative grounding results rest on self-annotated labels that need independent validation. read the letter →

arxiv 2509.06233 v1 pith:TY2N3TY5 submitted 2025-09-07 cs.RO cs.CV

Tongxuan Tian , Xuhui Kang , Yen-Ling Kuo This is my paper
classification cs.ROcs.CV
keywords affordancegroundingone-shotlearning3Dpointcloudsobject-to-objectinteractionvisionfoundationmodelsroboticmanipulationLLMconstraintgenerationsemanticfeatures
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a robot can learn to predict where two objects should interact—which point on a teapot should meet which region of a bowl, which edge of a knife should contact which part of an apple—from a single annotated example of each interaction type. This matters because daily manipulation tasks are almost always relational, yet existing affordance systems mostly label one object in isolation or only handle simple contact relations learned from large simulation data. The proposed system fuses semantic features from a visual foundation model with 3D point-cloud geometry for both objects, then uses a bidirectional joint-attention decoder to emit per-point interaction scores on both objects at once. In the paper's experiments, these maps generalize to unseen object instances and categories, survive occlusion, and, when turned into constraints for an LLM-generated pose optimizer, improve success rates on five two-object manipulation tasks in simulation and on a real robot. If the claim holds, object-to-object affordance becomes a data-cheap intermediate representation for robotic manipulation.

What carries the argument

The load-bearing object is the 3D semantic point cloud: DINOv2 features extracted from multiple RGB-D views are projected and fused onto each object's point cloud, giving every point a rich semantic descriptor alongside its coordinates. On top of this, a joint-attention transformer decoder tokenizes each point cloud into local patches with a PointNet encoder, then applies cross-attention in both directions—source attending to target and target attending to source—so that the predicted affordance on each object depends on the other object's geometry and semantics. One-hot source/target embeddings keep the roles distinct. The final per-point maps are trained with binary cross-entropy against d

What would settle it

Retrain the model on the same object pairs using independently annotated contact points from a second annotator, or contact labels recovered from physical interaction, instead of distance-propagated seed labels, then compare held-out IOU and real-robot success; a large drop would show the model learned label propagation artifacts rather than object-interaction semantics.

Watch

Extended reading notes

Core claim

The paper's central claim is that object-to-object affordance can be learned in 3D from one annotated example per interaction verb. The system takes two point clouds—a source object and a target object—with per-point semantic features distilled from a self-supervised visual foundation model, and a joint-attention transformer decoder predicts a per-point interaction-probability map on both objects simultaneously. The affordance category is defined by the verb (pour, cut, press, hang, insert), not by the object pair, so at test time the same decoder is applied zero-shot to entirely unseen object pairs and even unseen categories with similar functional parts. The paper further claims that feedi

Load-bearing premise

The results assume that a few user-picked contact points, spread to nearby points by distance, correctly mark the functional interaction region on each object, and that image-derived semantic features projected onto the point cloud carry enough signal for one example per interaction verb to generalize to unseen objects and categories.

Editorial extensions

If this is right

  • A robot can be taught a new interaction verb from a single example pair and generalize it to unseen object instances and categories.
  • Object-to-object affordance in 3D is more robust to viewpoint and occlusion changes than 2D affordance grounding, because semantic and geometric cues are fused in point-cloud space.
  • Affordance maps can act as effective intermediate constraints for LLM-based optimization planners, raising manipulation success in two-object tasks.
  • The one-shot formulation avoids large annotation efforts for pairwise affordance data, making the approach scalable to many everyday tasks.
  • The model degrades gracefully under partial observations, such as two camera views instead of four, supporting deployment with limited sensing.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct implication the paper leaves implicit: because the affordance category is a verb rather than a fixed object pair, the same decoder could be trained on one example per verb from human demonstrations, replacing seed-point annotations with contact traces recovered from manipulation videos.
  • The failure analysis suggests that once affordance grounding is adequate, sensor depth quality and inverse-kinematics feasibility become the dominant bottlenecks; combining this method with sparse-view 3D reconstruction could raise success more than further affordance-model tuning.
  • The one-shot formulation could be tested as an open-vocabulary system: substitute a language embedding for the one-hot affordance channel and ask the model to ground any verb, letting a user specify novel interactions without retraining.
  • Cross-category generalization to objects that share no part geometry (e.g., pouring from a bag versus a teapot) would stress whether the method relies on semantic features or on local geometry clusters; the paper does not test that limit.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes O3Afford, a one-shot 3D object-to-object affordance grounding method. It constructs semantic point clouds by projecting DINOv2 features from multi-view RGB-D frames, encodes source and target point clouds with a PointNet patch encoder, and uses a joint-attention transformer decoder to predict per-point affordance maps for both objects. The affordance maps are then integrated with LLM-generated geometric constraint functions for optimization-based robotic manipulation. Training uses a single annotated object pair per affordance category. Experiments report affordance grounding metrics (Table 1), ablations (Table 4), and success rates on five simulated and real-robot manipulation tasks (Tables 2-3), with qualitative generalization to unseen categories and occlusion conditions.

Significance. If the results hold, the paper makes a useful contribution: it formulates object-to-object affordance grounding as a one-shot 3D problem, combines DINOv2 feature distillation with a joint-attention interaction module, and demonstrates a plausible integration of affordance maps with LLM-generated constraints for manipulation. The simulation and real-robot evaluation scope is broader than in many affordance papers, and the method is relatively simple to reproduce from the description. The strongest parts are the clean problem formulation, the use of pretrained semantic features to reduce supervision, and the concrete downstream manipulation formulation. However, the central claims rest on a self-annotated label-generation procedure and on success counts without statistical support; these need independent validation before the reported margins can be taken as evidence of genuine object-to-object affordance understanding.

major comments (4)
  1. [Sec. 5.1, Table 1, Sec. 4.3] The ground-truth affordance labels are created by the authors via user-assigned contact points followed by distance-based propagation (Sec. 5.1, following [26]). The same annotation procedure is used for both training (BCE loss in Sec. 4.3) and evaluation (Table 1). There is no independent human annotation, no inter-annotator agreement, and no sensitivity analysis of the propagation radius/kernel. Consequently, the large margins in Table 1 (aIOU 26.19 vs 16.89 for the best baseline) may reflect learning the distance-propagation label bias rather than functional object-to-object semantics. The concern is reinforced by Table 4: removing DINOv2 features drops aIOU from 26.19 to 6.59, suggesting that the pretrained semantic features dominate the result. I recommend adding an independent human-annotated evaluation set (or a leave-one-out cross-check against human labels), reporting per-catego
  2. [Tables 2 and 3, Sec. 5.3] Success rates are reported as success counts over 10 trials per task with no confidence intervals, no significance tests, and no description of trial randomization or initial-state variation. The text repeatedly states that the method 'significantly outperforms' baselines (e.g., Sec. 5.3). With 10 trials, a 3/10 vs 8/10 difference is suggestive but not statistically secured, and smaller differences (e.g., Hang 3/10 vs 1/10 under 2-view) are within binomial noise. Please provide binomial confidence intervals, per-trial variability, and, where feasible, repeated seeds or independent trials. Also specify how ReKep is configured (keypoint proposal, optimizer, and whether the same LLM is used) so the comparison is reproducible.
  3. [Sec. 5.2.2 and Fig. 4] Category-level generalization is shown only qualitatively on unseen categories (scissor, coat rack, spray bottle). Since the title and abstract claim 'generalizable' and 'strong semantic and geometric generalization,' a quantitative evaluation on held-out categories is needed. As written, the claim rests on three qualitative examples. An extension would be to add a held-out category split to Tables 1/4 or report per-category aIOU/AUC for unseen categories. This is load-bearing for the generalization claim.
  4. [Table 1 and Appendix B.1] Baseline comparability is questionable: O2O-Afford is designed for annotation-free contact-based affordances (place/fit), and IAGNet is adapted from a single-image 2D setting by providing both source and target bounding boxes. The paper should justify that these are meaningful comparators for the one-shot 3D O2O setting, or add a baseline that uses the same input representation and supervision level. Otherwise the 'outperforms baselines' claim is difficult to interpret.
minor comments (5)
  1. [Abstract and Sec. 5.1] Typo: 'contraints' in the abstract. In Sec. 5.1, the phrase 'several user-assigned contact points' is vague; please specify how many contact points are used per object and the exact distance-propagation rule.
  2. [Table 1] The header arrows are inconsistent: IOU and SIM are marked with up arrows while MAE is marked with a down arrow, but the text later refers to '↑ IOU ↑ SIM ↓ MAE ↑ AUC' without explaining the direction. Clarify which direction is better for each metric.
  3. [Sec. 4.4] Equation (1) is referenced as 'Eq. 4.4' in the text, which is a formatting artifact. Also, the objective defines a weighted sum of score terms but does not explicitly list the constraints mentioned in the introductory sentence.
  4. [References] References [11] and [13] are duplicates (CLIP), and [12] and [34] are duplicates (latent diffusion). Please consolidate.
  5. [Sec. 3 and Sec. 4.3] The symbol K is used both for the number of affordance categories in the problem formulation and for the output-channel dimension in A ∈ [0,1]^{2B×K×N} in Sec. 4.3. Use distinct symbols to avoid confusion.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the core training/evaluation loop is supervised against human-assigned labels and external benchmarks, with no fitted input renamed as a prediction.

full rationale

No load-bearing circular step is present in the paper. The affordance model is trained with a standard binary cross-entropy loss (Sec. 4.3) against ground-truth affordance maps produced by user-assigned contact points propagated by distance (Sec. 5.1). This label-generation procedure is borrowed from prior work (3D AffordanceNet [26]) and is external to the authors' own method; the test-time affordance metrics (aIOU, SIM, MAE, AUC) are computed against held-out labels of the same type. The model does not fit a parameter to the test set and then rename it a prediction; it is trained on one example per category and evaluated on unseen object pairs, which is a standard supervised generalization setup. The manipulation experiments (Tables 2-3) measure physical task success in simulation and on a real Franka robot, which is an external, task-level outcome rather than a quantity derived from the model's own outputs. The only pretrained component, DINOv2, is used as a frozen external feature extractor, and the multi-view feature projection method is adapted from D3Field [14], which is not authored by the present authors. There are no self-citations in the reference list that carry the argument, and the paper does not invoke any uniqueness theorem or ansatz that is smuggled in via the authors' prior work. Concerns about distance-propagated label quality, the small number of trials, or the choice of LLM constraint weights in Table 7 are validity and robustness issues, not circularity: they do not make any stated prediction equivalent to its own input by construction.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The model itself has no exotic free constants, but the manipulation pipeline uses hand-set LLM constraint weights per task, and the entire evaluation rests on self-annotated affordance labels and the transferability of DINOv2 semantic features.

free parameters (2)
  • LLM constraint weights lambda_i per task = e.g., Pour: position 0.3, orientation 0.2, clearance 0.3, affordance alignment 0.1 (Table 7)
    These weights in Eq. 4.4 are selected by the authors/LLM for each task and directly affect the optimized pose; they are not learned from data.
  • Number of training examples per category = 1 (one-shot)
    The one-shot assumption is the core experimental setting; results may not transfer to other values of k.
assumptions (4)
  • domain assumption DINOv2 features projected onto 3D point clouds preserve semantic part correspondence useful for affordance grounding.
    The entire one-shot generalization depends on the transferability of these 2D features to novel 3D shapes (Sec. 4.2, Sec. 5.2.2).
  • domain assumption PointNet patch encoder captures local geometric primitives that transfer across object categories.
    The model relies on PointNet embeddings to generalize under occlusion and to new categories (Sec. 4.3, Appendix A.5).
  • domain assumption LLM-generated constraint functions are physically meaningful and correctly weighted for the specified tasks.
    Manipulation success depends on the LLM's commonsense translation of task names into geometric costs and on the hand-set weights in Table 7 (Sec. 4.4).
  • domain assumption The affordance labels produced by user-assigned contact points and distance propagation are a valid ground truth for object-to-object interaction.
    Both training and evaluation use these labels, so the metric values inherit their quality (Sec. 5.1).

how reviews work

0 comments
Cite this review

Pith. "Pith review of O$^3$Afford: One-Shot 3D Object-to-Object Affordance Grounding for Generalizable Robotic Manipulation." pith.science (2026). https://pith.science/paper/TY2N3TY5

@misc{pith2026250906233,
  author       = {Pith},
  title        = {Pith review of: O$^3$Afford: One-Shot 3D Object-to-Object Affordance Grounding for Generalizable Robotic Manipulation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TY2N3TY5}},
  note         = {Machine review of arXiv:2509.06233}
}
abstract

Grounding object affordance is fundamental to robotic manipulation as it establishes the critical link between perception and action among interacting objects. However, prior works predominantly focus on predicting single-object affordance, overlooking the fact that most real-world interactions involve relationships between pairs of objects. In this work, we address the challenge of object-to-object affordance grounding under limited data contraints. Inspired by recent advances in few-shot learning with 2D vision foundation models, we propose a novel one-shot 3D object-to-object affordance learning approach for robotic manipulation. Semantic features from vision foundation models combined with point cloud representation for geometric understanding enable our one-shot learning pipeline to generalize effectively to novel objects and categories. We further integrate our 3D affordance representation with large language models (LLMs) for robotics manipulation, significantly enhancing LLMs' capability to comprehend and reason about object interactions when generating task-specific constraint functions. Our experiments on 3D object-to-object affordance grounding and robotic manipulation demonstrate that our O$^3$Afford significantly outperforms existing baselines in terms of both accuracy and generalization capability.

Figures

Figures reproduced from arXiv: 2509.06233 by the authors.

Figure 1
Figure 1. O3Afford framework. O 3Afford first (a) constructs semantic point clouds; (b) employs joint cross-attention between the source and target objects for decoding; lastly, (c) leverages LLMs for task-relevant constraint function generation and optimizes the target pose for robot execution. fordance is commonly used either as an additional conditional input [45, 46] or as an auxiliary learn￾ing objective [47, 48] to enha… view at source ↗
Figure 2
Figure 2. Qualitative examples of the predicted object-to-object affordance in O3Afford. Bolt cameras to get RGB-D observations. We employ the LLM GPT-4o [54] from OpenAI for constraint function generation in planning. We design five tasks that require two objects to interact meaningfully with each other: pouring from teapot into bowl, inserting toast into toaster, pressing the button with hammer, hanging mug onto mug tree an… view at source ↗
Figure 4
Figure 4. Qualitative Results of Unseen Object Category. 5.2.2 Generalization Analysis Beyond the robust intra-class generalization under extreme data constraints demonstrated in Sec. 5.2.1, we also analyze the generalization capability of O3Afford in this section. We study two forms of generalization: generalization to different levels of occlusion and cross-category gen￾eralization, both of which are prevalent and essential… view at source ↗
Figures from the paper (11 more)
Figure 3
Figure 3. Figure 3: Quantitative results under different occlusion levels. Occlusion level (x-axis) ranges from 10% to 50% occluded point clouds. Occlusion Generalization. Rather than sim￾ulating occlusion by directly dropping points from the object point cloud, we evaluate gen￾eralizatio…
Figure 5
Figure 5. Figure 5: Example execution of real-world manipulation tasks. (Top) inserting, hanging, and cutting; (Bottom) pouring and pressing. The colors on the point clouds indicate the predicted affor￾dance values as mapped to the colorbar on the right. Results in simulation. Tab. 2 show…
Figure 6
Figure 6. Figure 6: presents the qualitative examples of our occlusion experiment in the simulated environment. We show that our method, leveraging the generalizable semantic features of DINOv2, demonstrates robustness under varying levels of occlusion. This includes extreme case with up …
Figure 7
Figure 7. Figure 7: Qualitative examples of RoboPoint predictions. in [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Qualitative examples of OOAL predictions for four different views. Each row renders the affordance heatmaps of the same source/target object across views. A.5 Visualization of Latent Patch Embedding For the PointNet-encoded patch embeddings, we observe that they encode…
Figure 9
Figure 9. Figure 9: t-SNE visualization of latent patch embeddings. Pour Insert Cut Hang Press [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: Qualitative results in simulation environment. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]
Figure 11
Figure 11. Figure 11: Example of the generated function for object alignment. [PITH_FULL_IMAGE:figures/full_fig_p018_11.png]
Figure 12
Figure 12. Figure 12: Example of incorrect point cloud. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_12.png]
Figure 13
Figure 13. Figure 13: Example of IK failure. D Environment Setup Franka Research 3 Femto Bolt [PITH_FULL_IMAGE:figures/full_fig_p020_13.png]
Figure 14
Figure 14. Figure 14: Real robot experiment setup. Our task uses a Franka Research 3 robot, a 7-DOF manipulator. To capture comprehensive visual information and mitigate occlusions, we employ two Femto Bolt RGB-D cameras. These Time￾20 [PITH_FULL_IMAGE:figures/full_fig_p020_14.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. SemAnCorr: Semantic Anchored Correspondence for Zero-Shot Manipulation Skill Transfer

    cs.RO 2026-07 conditional novelty 5.0 of 10

    Training-free semantic anchors plus functional maps yield dense correspondences that are both part-accurate and geometrically coherent, improving zero-shot robot skill transfer over nearest-neighbor 3D feature matching.

  2. Beyond Point-Attached Semantics: Object-Centric Semantic Fields for Generalizable Manipulation

    cs.RO 2026-07 conditional novelty 5.0 of 10

    An object-conditioned continuous semantic field queried at explicit 3D locations yields more stable part cues and higher manipulation success than point-attached 2D/3D features.

Reference graph

Works this paper leans on

61 extracted references · 45 canonical work pages · cited by 2 Pith papers

  1. [26]

    S. Deng, X. Xu, C. Wu, K. Chen, and K. Jia. 3d affordancenet: A benchmark for visual object affordance understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1778–1787, June 2021

    work page 2021

  2. [1]

    Kaptelinin and B

    V . Kaptelinin and B. Nardi. Affordances in hci: toward a mediated action perspective. In Proceedings of the SIGCHI conference on human factors in computing systems , pages 967– 976, 2012

    work page 2012

  3. [2]

    Delitzas, A

    A. Delitzas, A. Takmaz, F. Tombari, R. Sumner, M. Pollefeys, and F. Engelmann. Scene- fun3d: fine-grained functionality and affordance understanding in 3d scenes. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14531–14542, 2024

    work page 2024

  4. [3]

    Kuang, J

    Y . Kuang, J. Ye, H. Geng, J. Mao, C. Deng, L. Guibas, H. Wang, and Y . Wang. Ram: Retrieval- based affordance transfer for generalizable zero-shot robotic manipulation. arXiv preprint arXiv:2407.04689, 2024

    arXiv 2024

  5. [4]

    G. Li, D. Sun, L. Sevilla-Lara, and V . Jampani. One-shot open affordance learning with foun- dation models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3086–3096, 2024

    work page 2024

  6. [5]

    H. Luo, W. Zhai, J. Zhang, Y . Cao, and D. Tao. Learning affordance grounding from exo- centric images. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2252–2261, 2022

    work page 2022

  7. [6]

    H. Luo, W. Zhai, J. Zhang, Y . Cao, and D. Tao. Leverage interactive affinity for affordance learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, pages 6809–6819, 2023

    work page 2023

  8. [7]

    W. Zhai, H. Luo, J. Zhang, Y . Cao, and D. Tao. One-shot object affordance detection in the wild. International Journal of Computer Vision, 130(10):2472–2500, 2022

    work page 2022

Show all 61 references
  1. [8]

    X. Zhao, Y . Cao, and Y . Kang. Object affordance detection with relationship-aware network. Neural Computing and Applications, 32(18):14321–14333, 2020. 9

    2020

  2. [9]

    K. Mo, Y . Qin, F. Xiang, H. Su, and L. Guibas. O2o-afford: Annotation-free large-scale object-object affordance learning. In Conference on robot learning, pages 1666–1677. PMLR, 2022

    2022

  3. [10]

    Oquab, T

    M. Oquab, T. Darcet, T. Moutakanni, H. V o, M. Szafraniec, V . Khalidov, P. Fernandez, D. Haz- iza, F. Massa, A. El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023

    2023 arXiv

  4. [11]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever. Learning transferable visual models from natural language supervision. In M. Meila and T. Zhang, editors, Proceedings of the 38th In- ter...

    2021

  5. [12]

    Rombach, A

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer. High-resolution image syn- thesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022

    2022

  6. [13]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, pages 8748–8763. PmLR, 2021

    2021

  7. [14]

    Y . Wang, M. Zhang, Z. Li, T. Kelestemur, K. Driggs-Campbell, J. Wu, L. Fei-Fei, and Y . Li. D3fields: Dynamic 3d descriptor fields for zero-shot generalizable rearrangement. In 8th An- nual Conference on Robot Learning, 2024

    2024

  8. [15]

    Z. Zhou, Y . Lei, B. Zhang, L. Liu, and Y . Liu. Zegclip: Towards adapting clip for zero-shot semantic segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023

    2023

  9. [16]

    Han and S.-N

    G. Han and S.-N. Lim. Few-shot object detection with foundation models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 28608–28618, June 2024

    2024

  10. [17]

    Zhang, C

    J. Zhang, C. Herrmann, J. Hur, L. Polania Cabrera, V . Jampani, D. Sun, and M.-H. Yang. A tale of two features: Stable diffusion complements dino for zero-shot semantic correspondence. Advances in Neural Information Processing Systems, 36:45533–45547, 2023

    2023

  11. [18]

    Myers, C

    A. Myers, C. L. Teo, C. Fermüller, and Y . Aloimonos. Affordance detection of tool parts from geometric features. In 2015 IEEE international conference on robotics and automation (ICRA), pages 1374–1381. IEEE, 2015

    2015

  12. [19]

    Sawatzky, A

    J. Sawatzky, A. Srikantha, and J. Gall. Weakly supervised affordance detection. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2795–2804, 2017

    2017

  13. [20]

    J. H. Jang, H. Seo, and S. Y . Chun. Intra: Interaction relationship-aware weakly supervised affordance grounding. In European Conference on Computer Vision, pages 18–34. Springer, 2024

    2024

  14. [21]

    G. Li, V . Jampani, D. Sun, and L. Sevilla-Lara. Locate: Localize and transfer object parts for weakly supervised affordance grounding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10922–10931, 2023

    2023

  15. [22]

    S. Qian, W. Chen, M. Bai, X. Zhou, Z. Tu, and L. E. Li. Affordancellm: Grounding affordance from vision language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7587–7597, 2024. 10

    2024

  16. [23]

    C. Chen, Y . Cong, and Z. Kan. Worldafford: Affordance grounding based on natural language instructions. In 2024 IEEE 36th International Conference on Tools with Artificial Intelligence (ICTAI), pages 822–828. IEEE, 2024

    2024

  17. [24]

    Y . Ju, K. Hu, G. Zhang, G. Zhang, M. Jiang, and H. Xu. Robo-abc: Affordance general- ization beyond categories via semantic correspondence for robot manipulation. In European Conference on Computer Vision, pages 222–239. Springer, 2024

    2024

  18. [25]

    E. Tong, A. Opipari, S. Lewis, Z. Zeng, and O. C. Jenkins. Oval-prompt: Open-vocabulary af- fordance localization for robot manipulation through llm affordance-grounding.arXiv preprint arXiv:2404.11000, 2024

    2024 arXiv

  19. [27]

    Y . Yang, W. Zhai, H. Luo, Y . Cao, J. Luo, and Z.-J. Zha. Grounding 3d object affordance from 2d interactions in images. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 10905–10915, October 2023

    2023

  20. [28]

    Y . Yang, W. Zhai, H. Luo, Y . Cao, and Z.-J. Zha. Lemon: Learning 3d human-object interaction relation from 2d images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16284–16295, 2024

    2024

  21. [29]

    H. Chu, X. Deng, Q. Lv, X. Chen, Y . Li, J. Hao, and L. Nie. 3d-affordancellm: Harnessing large language models for open-vocabulary affordance detection in 3d worlds. arXiv preprint arXiv:2502.20041, 2025

    2025 arXiv

  22. [30]

    Y . Li, N. Zhao, J. Xiao, C. Feng, X. Wang, and T.-s. Chua. Laso: Language-guided affordance segmentation on 3d object. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 14251–14260, June 2024

    2024

  23. [31]

    Brown, B

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Nee- lakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-V oss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. ...

    1901

  24. [32]

    H. Song, L. Dong, W.-N. Zhang, T. Liu, and F. Wei. Clip models are few-shot learners: Empirical studies on vqa and visual entailment. arXiv preprint arXiv:2203.07190, 2022

    2022 arXiv

  25. [33]

    Touvron, L

    H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y . Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  26. [34]

    Rombach, A

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer. High-resolution image syn- thesis with latent diffusion models, 2021

    2021

  27. [35]

    Qiu, Y .-X

    R.-Z. Qiu, Y .-X. Wang, and K. Hauser. Aligndiff: aligning diffusion models for general few- shot segmentation. In European Conference on Computer Vision , pages 384–400. Springer, 2024

    2024

  28. [36]

    M. Zhu, Y . Liu, Z. Luo, C. Jing, H. Chen, G. Xu, X. Wang, and C. Shen. Unleash- ing the potential of the diffusion model in few-shot semantic segmentation. arXiv preprint arXiv:2410.02369, 2024. 11

    2024 arXiv

  29. [37]

    W. Tan, S. Chen, and B. Yan. Diffss: Diffusion model for few-shot semantic segmentation. arXiv preprint arXiv:2307.00773, 2023

    2023 arXiv

  30. [38]

    J. Zhou, J. Wang, B. Ma, Y .-S. Liu, T. Huang, and X. Wang. Uni3d: Exploring unified 3d rep- resentation at scale. In International Conference on Learning Representations (ICLR), 2024

    2024

  31. [39]

    Zhang, Z

    R. Zhang, Z. Guo, W. Zhang, K. Li, X. Miao, B. Cui, Y . Qiao, P. Gao, and H. Li. Pointclip: Point cloud understanding by clip. arXiv preprint arXiv:2112.02413, 2021

    2021 arXiv

  32. [40]

    Borja-Diaz, O

    J. Borja-Diaz, O. Mees, G. Kalweit, L. Hermann, J. Boedecker, and W. Burgard. Affordance learning from play for sample-efficient policy learning. In 2022 International Conference on Robotics and Automation (ICRA), pages 6372–6378. IEEE, 2022

    2022

  33. [41]

    O. Y . Lee, A. Xie, K. Fang, K. Pertsch, and C. Finn. Affordance-guided reinforcement learning via visual prompting. arXiv preprint arXiv:2407.10341, 2024

    2024 arXiv

  34. [42]

    S. Bahl, R. Mendonca, L. Chen, U. Jain, and D. Pathak. Affordances from human videos as a versatile representation for robotics. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13778–13790, 2023

    2023

  35. [43]

    Y . Geng, B. An, H. Geng, Y . Chen, Y . Yang, and H. Dong. Rlafford: End-to-end affordance learning for robotic manipulation. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 5880–5886. IEEE, 2023

    2023

  36. [44]

    K. Fang, P. Yin, A. Nair, H. R. Walke, G. Yan, and S. Levine. Generalization with lossy affordances: Leveraging broad offline data for learning visuomotor tasks. In Conference on Robot Learning, pages 106–117. PMLR, 2023

    2023

  37. [45]

    Nasiriany, S

    S. Nasiriany, S. Kirmani, T. Ding, L. Smith, Y . Zhu, D. Driess, D. Sadigh, and T. Xiao. Rt-affordance: Affordances are versatile intermediate representations for robot manipulation. arXiv preprint arXiv:2411.02704, 2024

    2024 arXiv

  38. [46]

    S. Wu, Y . Zhu, Y . Huang, K. Zhu, J. Gu, J. Yu, Y . Shi, and J. Wang. Afforddp: Generalizable diffusion policy with transferable affordance. arXiv preprint arXiv:2412.03142, 2024

    2024 arXiv

  39. [47]

    Y . Zha, S. Bhambri, and L. Guan. Contrastively learning visual attention as affordance cues from demonstrations for robotic grasping. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 7835–7842. IEEE, 2021

    2021

  40. [48]

    O. Mees, J. Borja-Diaz, and W. Burgard. Grounding language with visual affordances over unstructured data. In Proceedings of the IEEE International Conference on Robotics and Au- tomation (ICRA), London, UK, 2023

    2023

  41. [49]

    Huang, C

    W. Huang, C. Wang, R. Zhang, Y . Li, J. Wu, and L. Fei-Fei. V oxposer: Composable 3d value maps for robotic manipulation with language models. In J. Tan, M. Toussaint, and K. Darvish, editors, Proceedings of The 7th Conference on Robot Learning , volume 229 of Proceedings of M...

    2023

  42. [50]

    Huang, C

    W. Huang, C. Wang, Y . Li, R. Zhang, and L. Fei-Fei. Rekep: Spatio-temporal reasoning of relational keypoint constraints for robotic manipulation. arXiv preprint arXiv:2409.01652 , 2024

    2024 arXiv

  43. [51]

    W. Yuan, J. Duan, V . Blukis, W. Pumacay, R. Krishna, A. Murali, A. Mousavian, and D. Fox. Robopoint: A vision-language model for spatial affordance prediction for robotics. arXiv preprint arXiv:2406.10721, 2024

    2024 arXiv

  44. [52]

    Xiang, Y

    F. Xiang, Y . Qin, K. Mo, Y . Xia, H. Zhu, F. Liu, M. Liu, H. Jiang, Y . Yuan, H. Wang, L. Yi, A. X. Chang, L. J. Guibas, and H. Su. SAPIEN: A simulated part-based interactive environ- ment. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , June 2020. 12

    2020

  45. [53]

    I. Liu, E. Yang, J. Tao, R. Chen, X. Zhang, Q. Ran, Z. Liu, and H. Su. Activezero: Mixed domain learning for active stereovision with zero annotation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13033–13042, 2022

    2022

  46. [54]

    Achiam, S

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    2023 arXiv

  47. [55]

    M. A. Rahman and Y . Wang. Optimizing intersection-over-union in deep neural networks for image segmentation. In International symposium on visual computing , pages 234–244. Springer, 2016

    2016

  48. [56]

    M. J. Swain and D. H. Ballard. Color indexing. International journal of computer vision, 7(1): 11–32, 1991

    1991

  49. [57]

    C. J. Willmott and K. Matsuura. Advantages of the mean absolute error (mae) over the root mean square error (rmse) in assessing average model performance. Climate research, 30(1): 79–82, 2005

    2005

  50. [58]

    J. M. Lobo, A. Jiménez-Valverde, and R. Real. Auc: a misleading measure of the performance of predictive distribution models. Global ecology and Biogeography, 17(2):145–151, 2008

    2008

  51. [59]

    Y . Tang, W. Huang, Y . Wang, C. Li, R. Yuan, R. Zhang, J. Wu, and L. Fei-Fei. Uad: Un- supervised affordance distillation for generalization in robotic manipulation. arXiv preprint arXiv:2506.09284, 2025

    2025 arXiv

  52. [60]

    Xiang, Z

    J. Xiang, Z. Lv, S. Xu, Y . Deng, R. Wang, B. Zhang, D. Chen, X. Tong, and J. Yang. Structured 3d latents for scalable and versatile 3d generation. arXiv preprint arXiv:2412.01506, 2024. 13 Appendices A Additional Results A.1 Ablation We present ablation results on the model a...

    2024 arXiv

  53. [2020]

    URL https://proceedings.neurips.cc/paper_files/paper/2020/ file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf

    2020

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.