Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

SeqAfford: Sequential 3D Affordance Reasoning via Multimodal Large Language Model

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A single instruction becomes an ordered sequence of 3D affordance masks, in one model.

desk verdict A useful new task and benchmark for sequential 3D affordance reasoning, but the benchmark's instruction generation leaks the target affordance label, so the open-world reasoning claim is weaker than advertised. read the letter →

arxiv 2412.01550 v3 pith:FENCNNEG submitted 2024-12-02 cs.CV cs.AI

Chunlin Yu , Hanqing Wang , Ye Shi , Haoyang Luo , Sibei Yang , Jingyi Yu , Jingya Wang This is my paper · ORCID
classification cs.CVcs.AI
keywords 3Daffordancesegmentationsequentialreasoningmultimodallargelanguagemodelpointcloudgroundinginstructionunderstandingopen-worldgeneralizationbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper establishes a new task, Sequential 3D Affordance Reasoning, in which a single human instruction that implies several actions is decomposed into an ordered series of affordance masks on 3D point clouds. It builds the first instruction-based benchmark for this task, with roughly 180K instruction–point cloud pairs across 23 object categories, and proposes SeqAfford, a multimodal large language model extended with segmentation tokens that outputs both a step-by-step explanation and a mask per step. The paper claims that this architecture outperforms prior single-affordance methods on both simple and sequential settings and that it generalizes to affordance–object pairs never seen in training, evidence that the model draws on world knowledge rather than memorized mappings. A sympathetic reader would care because sequential reasoning is what an embodied agent needs to act on a sentence like 'reheat the leftovers' rather than on a pre-labeled grasp point.

What carries the argument

The load-bearing mechanism is the <SEG> token added to the multimodal language model's vocabulary, the same device used by 2D reasoning segmentation models but transplanted to 3D point clouds. Each <SEG> the model emits during text generation carries the reasoning for one affordance step; its last-layer embedding is projected and used as the query in a cross-attention layer over the point cloud's dense features, and the result is decoded into an affordance mask. A multi-granular feature propagation stage, built from farthest point sampling and upsampling, supplies the dense point features needed for this integration, allowing the coarse language-level reasoning to be grounded in fine-grained 3D geometry.

What would settle it

Construct a held-out evaluation set of human-written instructions matched to the benchmark's objects and ground-truth affordance labels but not generated by GPT-4, and without the affordance-type vocabulary (for example, 'the soup is hot, I want to warm it' instead of 'open, contain'). If SeqAfford's sequential mIoU on this set falls to the level of a baseline that simply predicts the most common mask for each object, the claim that the model reasons from world knowledge is not supported.

Watch

Extended reading notes

Core claim

SeqAfford's central claim is that a 3D multimodal large language model can reason and segment in one cohesive pass: given a point cloud and a complex instruction, the model autoregressively emits text whose <SEG> tokens mark each reasoned affordance step, and the last-layer embeddings of those tokens are passed through a multi-granular language-point integration module that fuses them with dense point features to produce the ordered masks. The paper demonstrates this on a new benchmark of 180K instruction-point cloud pairs, where SeqAfford exceeds the previous state of the art on language-guided single-affordance segmentation and, unlike all baselines, predicts sequential affordances without being given the ground-truth order. It further shows that the model keeps working when affordance-object pairings are held out from training, which the authors take as evidence of open-world generalization.

Load-bearing premise

The measured open-world generalization rests on the assumption that GPT-4-generated instructions conditioned on the object name and the target affordance type are a fair proxy for how humans would phrase requests; if real instructions use different vocabulary or blend affordances, the model's world-knowledge reasoning may be much weaker than reported.

Editorial extensions

If this is right

  • An embodied agent could take a single long-horizon command and directly obtain an ordered list of 'touch here' regions, without a separate planner to break the task into steps.
  • The same model handles both trivial single affordances and multi-step, multi-object sequences, so one system can replace several specialized affordance modules.
  • Open-world generalization to unseen affordance-object pairs would reduce the need for annotated affordance data for every new object class.
  • Because the model emits text explanations alongside masks, its step ordering is inspectable and could be used to verify or correct the plan before execution.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next test is to measure how SeqAfford degrades when instructions are rephrased by humans without any affordance vocabulary leaks; the paper's benchmark generation prompts explicitly constrain GPT-4 to 'only afford the given affordance,' so a wording-to-label shortcut may inflate the open-world numbers.
  • The ordered mask sequence could serve as a supervision signal for robot manipulation policies, for example by chaining the per-step masks into waypoints for grasping and placing.
  • The same <SEG>-token integration might transfer to other 3D dense prediction tasks such as part segmentation or referring expression segmentation, since the multi-granular language-point fusion is not affordance-specific.
  • For multi-object sequences, the paper extracts object nouns from the LLM output to order the masks; a failure analysis on noun ordering errors would clarify whether the reasoning is genuinely sequential or mostly template-following.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces Sequential 3D Affordance Reasoning, a task in which a model must take a complex human instruction, decompose it into an ordered sequence of affordances, and output a segmentation mask for each step. The authors construct a benchmark of 180K instruction-point cloud pairs by prompting GPT-4o with object names, ground-truth affordance types, rendered meshes, HOI images, and scenario descriptions over 3D AffordanceNet data. They then propose SeqAfford, a ShapeLLM-based 3D multimodal large language model extended with segmentation tokens and a multi-granular language-point integration module. Experiments report gains over single-step baselines on single and sequential affordance settings, together with ablations of the integration module and the 3D vision encoder. The paper claims open-world generalization and sequential reasoning abilities.

Significance. If the benchmark and evaluation are valid, this is a useful step toward linking 3D affordance segmentation with language-based sequential reasoning. The paper introduces the first instruction-based sequential affordance benchmark of this scale, and it demonstrates that an MLLM with segmentation tokens can be adapted to 3D dense prediction. The single-affordance results are consistent across metrics, and the MGLP ablation shows a substantial effect, which gives some confidence in the architecture. However, the benchmark-generation protocol embeds the ground-truth affordance label in the prompt that creates the test instructions, and the Unseen split does not include a sequential setting. These issues directly affect the two headline claims: world-knowledge reasoning and open-world sequential generalization. The contribution is meaningful but the central evidence needs to be strengthened before the claims are fully supported.

major comments (4)
  1. [Fig. 2 and Supplementary B.2] The instruction-generation pipeline provides GPT-4o with the object name and the ground-truth affordance type, and the prompt explicitly instructs that generated questions "can only afford the given affordance." This means the benchmark instructions are generated to be lexically and semantically unambiguous with respect to the affordance label, which may let a model solve the task by learning word-to-label associations rather than by performing the world-knowledge reasoning the paper emphasizes. The Unseen split only holds out affordance-object pairs within the same 23 categories and the same GPT-4o prompt distribution, so it does not rule out this shortcut. The authors should report lexical-overlap statistics between instruction tokens and affordance labels, evaluate on instructions written by humans or with deliberately ambiguous wording, and test whether a text-only or bag-of-words baseline can achieve nontrivial performance on the existing benchmark.
  2. [Sec. 5.2 and Table 2] The sequential evaluation is asymmetric and incomplete. The asterisk note states that baseline methods receive ground-truth decomposed instructions, while SeqAfford receives the original instruction; this makes the comparison conservative for SeqAfford, but it also means no baseline is asked to predict the sequence, so the table does not isolate the quality of SeqAfford's sequential reasoning from its per-step segmentation quality. Furthermore, no sequence-level metric is reported: the mIoU/AUC/SIM/MAE values aggregate mask predictions without indicating whether the predicted order of affordances matches the ground-truth order. The authors should add an order-aware metric such as exact-match or edit distance, and include an upper-bound condition in which SeqAfford is also given the ground-truth decomposed steps, so that the contribution of sequence prediction itself can be measured.
  3. [Sec. 3.2 and Table A1] The open-world generalization claim is stronger than what the experimental design supports. Table A1 contains a Single Unseen split but no Sequential Unseen split, so there is no evaluation of whether the model can generalize sequential reasoning to unseen affordance-object pairings. In addition, the Single Unseen split keeps the same 23 object categories and the same GPT-generated instruction distribution, which is a limited form of openness. The abstract's phrase "open-world generalization with sequential reasoning abilities" therefore requires either an additional unseen sequential evaluation or a more qualified claim that is consistent with the tested settings.
  4. [Sec. 4.2 and Table 3] The w/o MGLP ablation is not fully specified. It is unclear whether the <SEG> token is still produced in this variant and how the mask is decoded without the multi-granular language-point integration module. Because the 7.4-point mIoU drop on the single setting is used to justify the module's necessity, the authors should state exactly which components are removed and what replaces the cross-attention fusion in the ablated model.
minor comments (5)
  1. [Table 2] The header "Squential" is a typo and should read "Sequential."
  2. [Eq. (4) and Sec. 5.1] The loss-balancing weights lambda_c, lambda_b, and lambda_d are introduced in Eq. (4) but their values are never reported; please provide the default values and, ideally, a brief sensitivity check.
  3. [Table A1 and Sec. 3.2] The Unseen split is not described precisely enough: the table appears to show the same test shapes and pairs for Single Seen and Single Unseen, so the authors should clarify exactly which affordance-object pairs are removed from training and whether entire affordance types are withheld.
  4. [Table 2 and Related Work] The baseline PointRefer is referred to as LASO elsewhere in the paper; please use a single consistent name for this method.
  5. [Supplementary A] The statement that code and dataset will be released once the paper is accepted is helpful, but the contribution would be easier to assess if the benchmark are available at review time for inspection of instruction distributions and leakage statistics.

Circularity Check

1 steps flagged · score 4.0 of 10

Benchmark instructions are generated from the ground-truth affordance labels, making the open-world generalization evaluation partially self-referential; the segmentation grounding itself remains externally anchored.

  1. self definitional [Supplementary B.2 (Task prompt), Fig. 2; Section 3.2 Unseen split]
    "Task: Description: You will be provided with the name of an object. ... Along with the tool name, you will receive an affordance type that the object can afford ... Leakage Avoidance: Ensure that the generated questions tasks can only afford the given affordance. ... Instruction: With the provided OBJECT NAME: '+object+' and AFFORDANCE TYPE: '+affordance types+', generate fifteen new affordance grounding question tasks."

    The test instructions are produced by GPT-4o from a prompt whose inputs include the ground-truth AFFORDANCE TYPE, with an explicit rule that each question 'can only afford the given affordance.' Thus every evaluation input is, by construction, a paraphrase of the target label the model must infer. A model can score high mIoU/AUC by learning lexical associations between instruction words and the affordance class, without performing the sequential world-knowledge reasoning claimed in the abstract. The Unseen split only removes affordance-object pairs; it keeps the same label-conditioned prompt distribution, so it does not break the loop.

full rationale

No equation-level derivation exists in this paper, so the high end of the circularity scale does not apply. The model is compared against external baselines on fixed metrics, and the mask labels come from 3D AffordanceNet, not from the model or the instruction generator. The one load-bearing circularity concern is the benchmark construction: GPT-4o receives the object name and the ground-truth AFFORDANCE TYPE and is told to write questions that 'can only afford the given affordance' (Supp. B.2, Fig. 2). Consequently the test instructions are conditioned on the very labels the model is supposed to derive, so the 'open-world generalization with sequential reasoning abilities' claim is partly self-referential and may reflect wording-to-label shortcuts. The Unseen split does not remove this confound because it only holds out affordance-object pairings, not the label-conditioned prompt distribution. I therefore flag one self-definitional step and score 4: the central segmentation result retains independent content, but the headline reasoning/generalization claim is not independently established by the benchmark.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim depends on the quality of the synthetic benchmark and on the transferability of pretrained 3D representations. No physical entities are introduced. The listed free parameters are training hyperparameters whose sensitivity is not reported.

free parameters (3)
  • loss balancing weights lambda_c, lambda_b, lambda_d = not reported
    Eq. (4) combines language, binary cross-entropy, and Dice losses; the weights are chosen by hand and no sensitivity study is reported.
  • LoRA rank = 8
    Section 5 sets the rank to 8 by default; the paper does not ablate this value.
  • number of training epochs = 10
    Section 5 trains for 10 epochs on one A100; no early-stopping or epoch sensitivity analysis is reported.
assumptions (4)
  • ad hoc to paper GPT-4o-generated instructions correctly and uniquely express the intended affordance.
    Figure 2 and Supplementary B.2 show prompts containing object names, affordance types, and leakage-avoidance rules; the benchmark's validity depends on these synthetic instructions being faithful to the labels.
  • domain assumption 3D AffordanceNet masks are accurate ground truth for affordance regions.
    Section 3.1 states all point clouds and affordance annotations are sourced entirely from 3D AffordanceNet; any source errors propagate to the benchmark and evaluation.
  • domain assumption Pretrained ShapeLLM and Uni3D representations transfer to fine-grained affordance segmentation.
    Section 4.2 builds the model on ShapeLLM and Uni3D and freezes the 3D encoder; open-world generalization is expected from these pretrained features.
  • domain assumption The order of affordances in the generated text matches the correct execution order.
    Supplementary E.2 uses the order of nouns in the LLM output as the order of affordance reasoning; this assumes the text order is the correct task order.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SeqAfford: Sequential 3D Affordance Reasoning via Multimodal Large Language Model." pith.science (2026). https://pith.science/paper/FENCNNEG

@misc{pith2026241201550,
  author       = {Pith},
  title        = {Pith review of: SeqAfford: Sequential 3D Affordance Reasoning via Multimodal Large Language Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FENCNNEG}},
  note         = {Machine review of arXiv:2412.01550}
}
read the original abstract

3D affordance segmentation aims to link human instructions to touchable regions of 3D objects for embodied manipulations. Existing efforts typically adhere to single-object, single-affordance paradigms, where each affordance type or explicit instruction strictly corresponds to a specific affordance region and are unable to handle long-horizon tasks. Such a paradigm cannot actively reason about complex user intentions that often imply sequential affordances. In this paper, we introduce the Sequential 3D Affordance Reasoning task, which extends the traditional paradigm by reasoning from cumbersome user intentions and then decomposing them into a series of segmentation maps. Toward this, we construct the first instruction-based affordance segmentation benchmark that includes reasoning over both single and sequential affordances, comprising 180K instruction-point cloud pairs. Based on the benchmark, we propose our model, SeqAfford, to unlock the 3D multi-modal large language model with additional affordance segmentation abilities, which ensures reasoning with world knowledge and fine-grained affordance grounding in a cohesive framework. We further introduce a multi-granular language-point integration module to endow 3D dense prediction. Extensive experimental evaluations show that our model excels over well-established methods and exhibits open-world generalization with sequential reasoning abilities.

Figures

Figures reproduced from arXiv: 2412.01550 by the authors.

Figure 1
Figure 1. Sequential 3D affordance reasoning task with different types of interactions. We introduce SeqAfford, a Multi-Modal Language Model (MLLM) capable of serialized affordance inference implied in human instructions: 1) Single Affordance Reasoning; 2) Sequential Affordance Reasoning; 3) Sequential Affordance Reasoning with Multiple Objects Abstract 3D affordance segmentation aims to link human instructions to touchable r… view at source ↗
Figure 2
Figure 2. Preparing the instructions. To better utilize the world knowledge of GPT4, We prompt GPT-4o to generate diverse instructions based on 4 types of system prompts containing different modalities as input. Instructions are generated based on input prompts with modalities from a) purely textual affordance type, object name; b) the mesh-rendered image of the object; c) the mesh-rendered image and HOI images that reveal af… view at source ↗
Figure 3
Figure 3. Main Pipeline. Given the point clouds of the target objects and a piece of complex human instruction, SeqAfford first reasons from this instruction and decomposes it into several hidden <SEG> tokens extracted from the last-layer embeddings, each representing an intermediate affordance segmentation result. Then, for each <SEG>, the point features extracted by the 3D vision encoder dynamically interact with the <SEG> … view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Multi-Granular Language-Point Integration Module. We propose an interaction module between <SEG> tokens from LLM and point features from the 3D vision encoder, to synergize both reasoning and segmentation in a cohesive framework. This module consists of the multi-granu…
Figure 5
Figure 5. Figure 5: Qualitative results of our model. SeqAfford understands human instruction and accurately segments the target affordance. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. AffordDP: Generalizable Diffusion Policy with Transferable Affordance

    cs.RO 2024-12 conditional novelty 6.0 of 10

    A diffusion-based manipulation policy conditioned on transferred 3D contact points and post-contact trajectories, with adaptive affordance-guided sampling, generalizes to unseen object instances and categories.

Reference graph

Works this paper leans on

55 extracted references · 28 canonical work pages · cited by 1 Pith paper

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023. 4

    arXiv 2023

  2. [2]

    Flamingo: a visual language model for few-shot learning

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems, 35:23716–23736,

  3. [3]

    3d-affordancellm: Harnessing large language models for open-vocabulary affordance detec- tion in 3d worlds

    Hengshuo Chu, Xiang Deng, Xiaoyang Chen, Yinchuan Li, Jianye Hao, and Liqiang Nie. 3d-affordancellm: Harnessing large language models for open-vocabulary affordance detec- tion in 3d worlds. arXiv preprint arXiv:2502.20041, 2025. 2

    arXiv 2025

  4. [4]

    Iris: Interactive responsive intelligent segmentation for 3d affordance analysis

    Meng Chu and Xuan Zhang. Iris: Interactive responsive intelligent segmentation for 3d affordance analysis. arXiv preprint arXiv:2409.10078, 2024. 3

    arXiv 2024

  5. [5]

    Flashattention: Fast and memory-efficient exact attention with io-awareness

    Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christo- pher Ré. Flashattention: Fast and memory-efficient exact attention with io-awareness. Advances in Neural Information Processing Systems, 35:16344–16359, 2022. 6

    work page 2022

  6. [6]

    Scene- fun3d: fine-grained functionality and affordance understand- ing in 3d scenes

    Alexandros Delitzas, Ayca Takmaz, Federico Tombari, Robert Sumner, Marc Pollefeys, and Francis Engelmann. Scene- fun3d: fine-grained functionality and affordance understand- ing in 3d scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14531– 14542, 2024. 2

    work page 2024

  7. [7]

    3d affordancenet: A benchmark for visual object affordance understanding

    Shengheng Deng, Xun Xu, Chaozheng Wu, Ke Chen, and Kui Jia. 3d affordancenet: A benchmark for visual object affordance understanding. In proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1778–1787, 2021. 2, 4

    work page 2021

  8. [8]

    Learning 2d invariant affordance knowledge for 3d affordance ground- ing

    Xianqiang Gao, Pingrui Zhang, Delin Qu, Dong Wang, Zhi- gang Wang, Yan Ding, Bin Zhao, and Xuelong Li. Learning 2d invariant affordance knowledge for 3d affordance ground- ing. arXiv preprint arXiv:2408.13024, 2024. 2

    arXiv 2024

Show all 55 references
  1. [9]

    3d-llm: Injecting the 3d world into large language models

    Yining Hong, Haoyu Zhen, Peihao Chen, Shuhong Zheng, Yilun Du, Zhenfang Chen, and Chuang Gan. 3d-llm: Injecting the 3d world into large language models. Advances in Neural Information Processing Systems, 36:20482–20494, 2023. 3

    2023

  2. [10]

    Lora: Low-rank adaptation of large language models.arXiv preprint arXiv:2106.09685, 2021

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models.arXiv preprint arXiv:2106.09685, 2021. 6

    2021 arXiv

  3. [11]

    Robo-abc: Affordance generalization beyond categories via semantic correspondence for robot ma- nipulation

    Yuanchen Ju, Kaizhe Hu, Guowei Zhang, Gu Zhang, Mingrun Jiang, and Huazhe Xu. Robo-abc: Affordance generalization beyond categories via semantic correspondence for robot ma- nipulation. In European Conference on Computer Vision , pages 222–239. Springer, 2024. 2

    2024

  4. [12]

    Bert: Pre-training of deep bidirectional transform- ers for language understanding

    Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. Bert: Pre-training of deep bidirectional transform- ers for language understanding. InProceedings of naacL-HLT, page 2. Minneapolis, Minnesota, 2019. 2

    2019

  5. [13]

    Lisa: Reasoning segmentation via large language model

    Xin Lai, Zhuotao Tian, Yukang Chen, Yanwei Li, Yuhui Yuan, Shu Liu, and Jiaya Jia. Lisa: Reasoning segmentation via large language model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 9579–9589, 2024. 5

    2024

  6. [14]

    One-shot open affordance learning with foundation models

    Gen Li, Deqing Sun, Laura Sevilla-Lara, and Varun Jampani. One-shot open affordance learning with foundation models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3086–3096, 2024. 2

    2024

  7. [15]

    Referring transformer: A one-step approach to multi-task visual grounding

    Muchen Li and Leonid Sigal. Referring transformer: A one-step approach to multi-task visual grounding. Advances in neural information processing systems, 34:19652–19664,

  8. [16]

    Laso: Language-guided affordance seg- mentation on 3d object

    Yicong Li, Na Zhao, Junbin Xiao, Chun Feng, Xiang Wang, and Tat-seng Chua. Laso: Language-guided affordance seg- mentation on 3d object. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 14251–14260, 2024. 2, 4, 5, 7, 9

    2024

  9. [17]

    Gres: Gen- eralized referring expression segmentation

    Chang Liu, Henghui Ding, and Xudong Jiang. Gres: Gen- eralized referring expression segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 23592–23601, 2023. 7

    2023

  10. [18]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024. 3

    2024

  11. [19]

    Openshape: Scaling up 3d shape representation towards open-world understanding

    Minghua Liu, Ruoxi Shi, Kaiming Kuang, Yinhao Zhu, Xu- anlin Li, Shizhong Han, Hong Cai, Fatih Porikli, and Hao Su. Openshape: Scaling up 3d shape representation towards open-world understanding. Advances in neural information processing systems, 36, 2024. 8

    2024

  12. [20]

    Roberta: A robustly optimized bert pretraining approach

    Yinhan Liu. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 364, 2019. 2

    1907 arXiv

  13. [21]

    Interngpt: Solving vision-centric tasks by interacting with chatgpt beyond language

    Zhaoyang Liu, Yinan He, Wenhai Wang, Weiyun Wang, Yi Wang, Shoufa Chen, Qinglong Zhang, Zeqiang Lai, Yang Yang, Qingyun Li, et al. Interngpt: Solving vision-centric tasks by interacting with chatgpt beyond language. arXiv preprint arXiv:2305.05662, 2023. 3

    2023 arXiv

  14. [22]

    Auc: a misleading measure of the performance of predictive distribution models

    Jorge M Lobo, Alberto Jiménez-Valverde, and Raimundo Real. Auc: a misleading measure of the performance of predictive distribution models. Global ecology and Biogeog- raphy, 17(2):145–151, 2008. 7

    2008

  15. [23]

    Decoupled weight decay regularization

    I Loshchilov. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. 6

    2017 arXiv

  16. [24]

    Geal: Generalizable 3d affordance learning with cross- modal consistency

    Dongyue Lu, Lingdong Kong, Tianxin Huang, and Gim Hee Lee. Geal: Generalizable 3d affordance learning with cross- modal consistency. arXiv preprint arXiv:2412.09511, 2024. 2

    2024 arXiv

  17. [25]

    3d-sps: Single-stage 3d visual grounding via referred point progressive selection

    Junyu Luo, Jiahui Fu, Xianghao Kong, Chen Gao, Haibing Ren, Hao Shen, Huaxia Xia, and Si Liu. 3d-sps: Single-stage 3d visual grounding via referred point progressive selection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16454–16...

    2022

  18. [26]

    Partnet: A large- scale benchmark for fine-grained and hierarchical part-level 3d object understanding

    Kaichun Mo, Shilin Zhu, Angel X Chang, Li Yi, Subarna Tripathi, Leonidas J Guibas, and Hao Su. Partnet: A large- scale benchmark for fine-grained and hierarchical part-level 3d object understanding. In Proceedings of the IEEE/CVF conference on computer vision and pattern recog...

    2019

  19. [27]

    O2o-afford: Annotation-free large-scale object- object affordance learning

    Kaichun Mo, Yuzhe Qin, Fanbo Xiang, Hao Su, and Leonidas Guibas. O2o-afford: Annotation-free large-scale object- object affordance learning. In Conference on robot learning, pages 1666–1677. PMLR, 2022. 4

    2022

  20. [28]

    Rt-affordance: Affordances are versatile intermedi- ate representations for robot manipulation

    Soroush Nasiriany, Sean Kirmani, Tianli Ding, Laura Smith, Yuke Zhu, Danny Driess, Dorsa Sadigh, and Ted Xiao. Rt-affordance: Affordances are versatile intermedi- ate representations for robot manipulation. arXiv preprint arXiv:2411.02704, 2024. 3

    2024 arXiv

  21. [29]

    Open-vocabulary affordance detection in 3d point clouds

    Toan Nguyen, Minh Nhat Vu, An Vuong, Dzung Nguyen, Thieu V o, Ngan Le, and Anh Nguyen. Open-vocabulary affordance detection in 3d point clouds. In 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5692–5698. IEEE, 2023. 2, 5

    2023

  22. [30]

    Where2explore: Few-shot affordance learning for unseen novel categories of articulated objects

    Chuanruo Ning, Ruihai Wu, Haoran Lu, Kaichun Mo, and Hao Dong. Where2explore: Few-shot affordance learning for unseen novel categories of articulated objects. Advances in Neural Information Processing Systems, 36:4585–4596, 2023. 2

    2023

  23. [31]

    Training language models to follow instructions with human feedback.Advances in neural information processing systems, 35:27730–27744,

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Car- roll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback.Advances in neural information processing systems, 35:277...

  24. [32]

    Pointnet++: Deep hierarchical feature learning on point sets in a metric space

    Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Advances in neural information processing systems, 30, 2017. 6, 2

    2017

  25. [33]

    Contrast with reconstruct: Contrastive 3d representation learning guided by generative pretraining

    Zekun Qi, Runpei Dong, Guofan Fan, Zheng Ge, Xiangyu Zhang, Kaisheng Ma, and Li Yi. Contrast with reconstruct: Contrastive 3d representation learning guided by generative pretraining. In International Conference on Machine Learn- ing, pages 28223–28243. PMLR, 2023. 5

    2023

  26. [34]

    Shapellm: Universal 3d object understanding for embodied interaction

    Zekun Qi, Runpei Dong, Shaochen Zhang, Haoran Geng, Chunrui Han, Zheng Ge, Li Yi, and Kaisheng Ma. Shapellm: Universal 3d object understanding for embodied interaction. arXiv preprint arXiv:2402.17766, 2024. 2, 3, 5, 6, 8

    2024 arXiv

  27. [35]

    Affordancellm: Grounding affordance from vision language models

    Shengyi Qian, Weifeng Chen, Min Bai, Xiong Zhou, Zhuowen Tu, and Li Erran Li. Affordancellm: Grounding affordance from vision language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7587–7597, 2024. 2

    2024

  28. [36]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  29. [37]

    Optimizing intersection- over-union in deep neural networks for image segmentation

    Md Atiqur Rahman and Yang Wang. Optimizing intersection- over-union in deep neural networks for image segmentation. In International symposium on visual computing, pages 234–

  30. [38]

    Great: Geometry-intention collab- orative inference for open-vocabulary 3d object affordance grounding

    Yawen Shao, Wei Zhai, Yuhang Yang, Hongchen Luo, Yang Cao, and Zheng-Jun Zha. Great: Geometry-intention collab- orative inference for open-vocabulary 3d object affordance grounding. arXiv preprint arXiv:2411.19626, 2024. 2

    2024 arXiv

  31. [39]

    Color indexing

    Michael J Swain and Dana H Ballard. Color indexing. Inter- national journal of computer vision, 7(1):11–32, 1991. 7

    1991

  32. [40]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Mar- tinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Roz- ière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. 5

    2023 arXiv

  33. [41]

    Visionllm: Large language model is also an open- ended decoder for vision-centric tasks

    Wenhai Wang, Zhe Chen, Xiaokang Chen, Jiannan Wu, Xizhou Zhu, Gang Zeng, Ping Luo, Tong Lu, Jie Zhou, Yu Qiao, et al. Visionllm: Large language model is also an open- ended decoder for vision-centric tasks. Advances in Neural Information Processing Systems, 36, 2024. 3

    2024

  34. [42]

    Dynamic graph cnn for learning on point clouds

    Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon. Dynamic graph cnn for learning on point clouds. ACM Transactions on Graphics (tog), 38(5):1–12, 2019. 6, 2, 5

    2019

  35. [43]

    Advantages of the mean absolute error (mae) over the root mean square error (rmse) in assessing average model performance

    Cort J Willmott and Kenji Matsuura. Advantages of the mean absolute error (mae) over the root mean square error (rmse) in assessing average model performance. Climate research, 30(1):79–82, 2005. 7

    2005

  36. [44]

    Learning environment-aware affor- dance for 3d articulated object manipulation under occlu- sions

    Ruihai Wu, Kai Cheng, Yan Zhao, Chuanruo Ning, Guanqi Zhan, and Hao Dong. Learning environment-aware affor- dance for 3d articulated object manipulation under occlu- sions. Advances in Neural Information Processing Systems, 36:60966–60983, 2023. 2

    2023

  37. [45]

    Afforddp: Gener- alizable diffusion policy with transferable affordance

    Shijie Wu, Yihang Zhu, Yunao Huang, Kaizhen Zhu, Jiayuan Gu, Jingyi Yu, Ye Shi, and Jingya Wang. Afforddp: Gener- alizable diffusion policy with transferable affordance. arXiv preprint arXiv:2412.03142, 2024. 2

    2024 arXiv

  38. [46]

    Partafford: Part-level affordance discovery from 3d objects

    Chao Xu, Yixin Chen, He Wang, Song-Chun Zhu, Yixin Zhu, and Siyuan Huang. Partafford: Part-level affordance discovery from 3d objects. arXiv preprint arXiv:2202.13519,

    arXiv

  39. [47]

    Weakly-supervised affordance grounding guided by part-level semantic priors

    Peiran Xu and MU Yadong. Weakly-supervised affordance grounding guided by part-level semantic priors. In The Thir- teenth International Conference on Learning Representations. 2

  40. [48]

    PointLLM: Empowering large language models to understand point clouds

    Runsen Xu, Xiaolong Wang, Tai Wang, Yilun Chen, Jiang- miao Pang, and Dahua Lin. PointLLM: Empowering large language models to understand point clouds. arXiv preprint arXiv:2308.16911, 2023. 2, 3

    2023 arXiv

  41. [49]

    Ulip: Learning a unified representation of language, images, and point clouds for 3d understanding

    Le Xue, Mingfei Gao, Chen Xing, Roberto Martín-Martín, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, and Silvio Savarese. Ulip: Learning a unified representation of language, images, and point clouds for 3d understanding. In Proceedings of the IEEE/CVF conference on c...

    2023

  42. [50]

    Grounding 3d object affordance from 2d interactions in images

    Yuhang Yang, Wei Zhai, Hongchen Luo, Yang Cao, Jiebo Luo, and Zheng-Jun Zha. Grounding 3d object affordance from 2d interactions in images. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10905– 10915, 2023. 2, 4, 5, 7

    2023

  43. [51]

    Lemon: Learning 3d human-object interac- tion relation from 2d images

    Yuhang Yang, Wei Zhai, Hongchen Luo, Yang Cao, and Zheng-Jun Zha. Lemon: Learning 3d human-object interac- tion relation from 2d images. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 16284–16295, 2024. 2

    2024

  44. [52]

    Ferret: Refer and ground anything anywhere at any granularity

    Haoxuan You, Haotian Zhang, Zhe Gan, Xianzhi Du, Bowen Zhang, Zirui Wang, Liangliang Cao, Shih-Fu Chang, and Yinfei Yang. Ferret: Refer and ground anything anywhere at any granularity. arXiv preprint arXiv:2310.07704, 2023. 3

    2023 arXiv

  45. [53]

    Uni3d: A unified baseline for multi-dataset 3d object detection

    Bo Zhang, Jiakang Yuan, Botian Shi, Tao Chen, Yikang Li, and Yu Qiao. Uni3d: A unified baseline for multi-dataset 3d object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9253– 9262, 2023. 8, 2

    2023

  46. [54]

    Gpt4roi: Instruction tuning large language model on region- of-interest

    Shilong Zhang, Peize Sun, Shoufa Chen, Min Xiao, Wenqi Shao, Wenwei Zhang, Yu Liu, Kai Chen, and Ping Luo. Gpt4roi: Instruction tuning large language model on region- of-interest. arXiv preprint arXiv:2307.03601, 2023. 3

    2023 arXiv

  47. [55]

    B.2. Task In the task module, we have preset the textual Prompt template as follows:

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mo- hamed Elhoseiny. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592, 2023. 2, 3 SeqAfford: Sequential 3D Affordance Reasoning via Multimodal Large Languag...

    2023 arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.