REVIEW 5 major objections 6 minor 1 cited by
PanoAffordanceNet: Towards Holistic Affordance Grounding in 360{\deg} Indoor Environments
T0 review · 5 major / 6 minor · reviewed 2026-08-02 · deepseek-v4-flash
Pith's one-line read PanoAffordanceNet brings affordance grounding to whole 360° indoor scenes, and a new dataset shows it works.
desk verdict A genuinely new panoramic affordance task and dataset, but the headline comparison against baselines is uncontrolled and the artifacts aren't out yet; worth a major-revision round, not a desk reject. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
DASM and OSDH. DASM (Distortion-Aware Spectral Modulator) uses Laplacian and Gaussian filters to split visual features into high- and low-frequency branches, then applies latitude-dependent gates to counteract ERP stretching near the poles while sharpening boundaries near the equator. OSDH (Omni-Spherical Densification Head) builds a cosine-similarity affinity matrix from spherically projected visual tokens, selects top-k confident seeds, and propagates activations via max pooling to restore topologically continuous affordance regions. A multi-level training objective—pixel-wise BCE, distributional KL divergence, and region-text contrastive loss—ties the modules together under low supervisio
What would settle it
Annotate a subset of the same 360° panoramas with dense pixel-level affordance masks, then compute the overlap between the keypoint-blurred heatmaps and the dense masks (e.g., IoU or correlation). Also measure inter-annotator agreement on keypoint placement. If the overlap is low or annotator variance is high, the benchmark signal is unreliable.
Extended reading notes
Core claim
The central claim is that panoramic affordance grounding becomes tractable when the model explicitly accounts for the geometric distortions of equirectangular projection and the topological sparsity of functional regions. PanoAffordanceNet does this with a Distortion-Aware Spectral Modulator (DASM) that separates and rebalances high- and low-frequency features by latitude, and an Omni-Spherical Densification Head (OSDH) that propagates sparse seed activations through cosine self-similarity to recover continuous affordance regions. On the newly built 360-AGD benchmark, the framework reports KLD 1.270 versus 2.868 on the easy split and 1.306 versus 2.965 on the hard split compared with the bes
Load-bearing premise
The ground-truth heatmaps are generated by blurring keypoints that annotators placed inside non-occluded interaction regions; if those keypoint-derived heatmaps do not faithfully represent the true functional areas—especially for large or multi-part affordances such as sit or lie—every reported metric inherits that bias.
Editorial extensions
If this is right
- Embodied robots could infer interaction possibilities across a whole room from a single omnidirectional image, enabling global action planning.
- The 360-AGD dataset provides a standardized benchmark for scene-level affordance grounding, allowing future methods to be compared on common footing.
- The method's competitive results on perspective AGD20K suggest a panoramic-trained model can transfer to conventional views, potentially serving as a unified affordance backbone.
- Keypoint-derived heatmap supervision proves sufficient to train dense panoramic affordance grounding, lowering annotation cost compared to pixel-level masks.
Reading between the lines
- The reported margins are so large that part of the effect may stem from the keypoint-blurred supervision signal rather than from the model itself; a dense-label evaluation would separate these contributions.
- Because the architecture builds on DINOv2 and CLIP with LoRA and a prompt learner, it may extend to open-vocabulary affordances beyond the 19 annotated classes by swapping text embeddings—an avenue the paper does not investigate.
- Treating multiple disjoint regions of the same affordance as one class may reward merging behavior; evaluating region-level consistency (e.g., per-connected-component agreement) could produce a more complete picture.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces a new task, holistic affordance grounding in 360° indoor panoramas, and proposes PanoAffordanceNet, an end-to-end one-shot framework composed of a LoRA-adapted dual encoder, a Distortion-Aware Spectral Modulator (DASM), a Spherical-Aware Hierarchical Decoder with an Omni-Spherical Densification Head (OSDH), and a multi-level training objective combining BCE, KL, and region-text contrastive losses. The authors also construct 360-AGD, a new panoramic affordance grounding dataset with keypoint-derived heatmap supervision, and report large gains over OOAL and OS-AGDO on this dataset while remaining competitive on the perspective AGD20K benchmark.
Significance. If the empirical results hold, this paper would provide the first benchmark and strong baseline for scene-level affordance grounding in 360° environments, a timely and useful contribution for embodied AI. The proposed modules target meaningful challenges (ERP distortion, sparse activations, semantic drift) and the generalization results on AGD20K, despite training on a different domain, suggest that the method has real merit. However, the central claims currently rest on an uncontrolled baseline comparison and an unvalidated annotation protocol, so the significance cannot be fully assessed until these are addressed.
major comments (5)
- [§V-B, Table I(a)] The headline comparison is uncontrolled. PanoAffordanceNet is trained end-to-end on 360-AGD with L_BCE, L_KL, and L_RTC for 20k iterations, while OOAL and OS-AGDO are only described as 'adapted for equirectangular projections.' No fine-tuning on 360-AGD or target-domain support-set adaptation is reported for the baselines. The large KLD/SIM/NSS gaps could therefore be due to the baselines never training on ERP images or the annotation distribution, rather than to architectural superiority. The authors must either train the baselines under the same protocol (e.g., same 360-AGD training data and losses) or clearly frame the comparison as domain transfer and temper the 'significantly outperforms' claim.
- [§IV Data Annotation; §V-A] The 360-AGD annotation protocol is the empirical foundation of the paper, but it lacks any validation. The paper does not report the number of panoramic images, the number of keypoints per affordance class, the Gaussian kernel parameter σ used to generate heatmaps, or any inter-annotator agreement. Since these keypoint-derived heatmaps are used both as training targets and as evaluation ground truth, all reported metrics are self-referential. If keypoints are sparse within large functional regions, metrics like NSS and KLD may reward concentrated predictions rather than complete affordance areas. The authors should provide annotation statistics, agreement measures, and a comparison against dense pixel-level labels on a subset.
- [§III-C, Eqs. (2)-(3), Fig. 2] DASM is claimed to perform 'latitude-dependent calibration' for ERP distortion, but the formulation contains no latitude conditioning. Eq. (2) applies a global Gaussian low-pass and a global Laplacian high-pass; Eq. (3) fuses them with language-driven and self-adaptive gates. There is no polar/equatorial weighting, no position encoding, and no explicit dependence on latitude in the equations or figure. The claims that HFEM sharpens 'equatorial regions' and LFSM stabilizes 'near the poles' are not supported by the described mechanism. Either add an explicit latitude-aware mechanism or soften the distortion-calibration claims to what the architecture actually implements.
- [§III-D, Eqs. (5)-(7)] OSDH is described as operating on the 'spherical manifold' and restoring 'topological continuity,' but Eq. (5) is a plain cosine similarity in the flattened ERP feature space, and Eq. (7) propagates maxima over top-k seeds without any spherical geodesic distance or wraparound-aware adjacency. The construction does not encode the spherical topology that the paper claims to exploit. This is a central design claim; the authors should either incorporate an explicit spherical adjacency or rephrase the contribution as generic self-similarity densification.
- [§V, Tables I-III] No error bars, multiple seeds, or significance tests are reported. Several ablation differences are very small (e.g., Table II rows 3–4 differ by 0.021 KLD; Table V KLD from top-k=5 to 10 differs by 0.002). Without variance estimates, it is impossible to know whether these differences are meaningful. The paper should report mean and standard deviation over at least three runs for the main comparisons and ablations. Additionally, the 'one-shot' protocol is never defined: the proposed method is trained on 360-AGD for 20k iterations, while the baselines are one-shot methods. This protocol ambiguity should be clarified.
minor comments (6)
- [Fig. 2] The figure contains an unresolved placeholder 'zz在此处键入公式。' and '????' in the OSDH block. This must be fixed before publication.
- [Eq. (4)] The notation D^{-0.5} is used without definition. Please specify what D is (e.g., a diagonal scaling matrix) and how it is computed.
- [Table III] The row labels and checkmarks are ambiguous. The text says 'Training solely with pixel-level supervision L_BCE establishes a baseline,' but the first row appears to check L_KL rather than L_BCE. Please align the table columns with the described configurations.
- [§IV] The dataset section gives no quantitative statistics: no image counts per split, no class frequency table, no resolution breakdown after preprocessing. The word clouds in Fig. 4 are not a substitute for reporting dataset composition.
- [Table I(b)] The claim of 'high competitiveness' on the AGD20K Unseen Split is strained: the method's KLD is 1.185 versus OOAL's 1.070. If SIM/NSS are emphasized, please state that explicitly and avoid implying overall superiority.
- [References] Reference [25] contains a typo: 'and ohters' should be 'and others.'
Circularity Check
No significant circularity: the paper's claims are empirical and self-contained, with no prediction reducing to its fitted inputs or to a load-bearing self-citation chain.
full rationale
I walked the paper's derivation chain: the method equations (Eqs. 1-11) define an architecture and a multi-term loss; no predicted quantity is, by construction, equal to a fitted parameter or to the ground-truth annotation. The paper does not fit a parameter to a subset of data and then rename it as a prediction; the reported KLD/SIM/NSS numbers are standard supervised evaluations on a self-created benchmark. The 360-AGD dataset is indeed constructed by the authors and evaluated with Gaussian-blurred keypoint heatmaps used both for training and testing, but this is a standard supervised protocol rather than a circular reduction: the model's output is not derived from the ground truth by an equation that makes the metric vacuous. The paper's own limitations—such as ignoring heavily occluded interaction regions and lacking dense pixel-level annotations—are annotation-quality concerns, not circularity. Self-citations appear in related work (e.g., [25] on panoramic segmentation) and in a baseline reference ([35]), but they are contextual or comparative, not load-bearing for the central claim. The main weakness is that baseline methods are only described as 'adapted' and are not shown to receive the same training data as PanoAffordanceNet; this is a fairness/control concern for the empirical comparison, not a circularity of the derivation. On the defined circularity criteria, I find no step where the paper's reasoning reduces to its own inputs.
Assumptions & free parameters
free parameters (5)
- Loss weights λ1, λ2, λ3 =
not reported
- LoRA rank r =
16
- Seed count top-k =
10 (implied; not explicit)
- Gaussian kernel σ for ground-truth heatmaps =
not reported
- Learnable scalars α and λ_k, and CoOp prompt context vectors =
learned
assumptions (4)
- domain assumption ERP distortion can be approximately separated and corrected by high/low-frequency spectral modulation.
- domain assumption Keypoint-driven Gaussian heatmaps are a valid dense ground-truth for affordance grounding.
- domain assumption Pre-trained DINOv2 and CLIP features transfer to panoramic indoor affordance grounding with LoRA/prompt adaptation.
- domain assumption Cosine self-similarity in the visual feature space is a reliable structural prior for densifying affordance regions.
Cite this review
Pith. "Pith review of PanoAffordanceNet: Towards Holistic Affordance Grounding in 360{\deg} Indoor Environments." pith.science (2026). https://pith.science/paper/TGPYQ75Q
@misc{pith2026260309760,
author = {Pith},
title = {Pith review of: PanoAffordanceNet: Towards Holistic Affordance Grounding in 360\deg Indoor Environments},
year = {2026},
howpublished = {\url{https://pith.science/paper/TGPYQ75Q}},
note = {Machine review of arXiv:2603.09760}
}
read the original abstract
Global perception is essential for embodied agents in 360{\deg} spaces, yet current affordance grounding remains largely object-centric and restricted to perspective views. To bridge this gap, we introduce a novel task: Holistic Affordance Grounding in 360{\deg} Indoor Environments. This task faces unique challenges, including severe geometric distortions from Equirectangular Projection (ERP), semantic dispersion, and cross-scale alignment difficulties. We propose PanoAffordanceNet, an end-to-end framework featuring a Distortion-Aware Spectral Modulator (DASM) for latitude-dependent calibration and an Omni-Spherical Densification Head (OSDH) to restore topological continuity from sparse activations. By integrating multi-level constraints comprising pixel-wise, distributional, and region-text contrastive objectives, our framework effectively suppresses semantic drift under low supervision. Furthermore, we construct 360-AGD, the first high-quality panoramic affordance grounding dataset. Extensive experiments demonstrate that PanoAffordanceNet significantly outperforms existing methods, establishing a solid baseline for scene-level perception in embodied intelligence. The source code and benchmark dataset will be made publicly available at https://github.com/GL-ZHU925/PanoAffordanceNet.
Figures
Figures from the paper (3 more)
Forward citations
Cited by 1 Pith paper
-
Panoramic Scene Understanding: A Survey from Distortion-Aware Engineering to Sphere-Native Modeling
Survey organizing panoramic scene analysis literature by architectural design and training paradigm, identifying the absence of methods achieving both strict spherical equivariance and full reuse of perspective-pretra...
Reference graph
Works this paper leans on
-
[1]
A survey of embodied ai: From simulators to research tasks,
J. Duan, S. Yu, H. L. Tan, H. Zhu, and C. Tan, “A survey of embodied ai: From simulators to research tasks,”IEEE Transactions on Emerging Topics in Computational Intelligence, vol. 6, no. 2, pp. 230–244, 2022
2022
-
[2]
Thinking in 360 °: Humanoid visual search in the wild,
H. Yu, Y . Han, X. Zhanget al., “Thinking in 360 °: Humanoid visual search in the wild,”arXiv preprint arXiv:2511.20351, 2025
arXiv 2025
-
[3]
Speaker-follower models for vision-and-language navigation,
D. Friedet al., “Speaker-follower models for vision-and-language navigation,” inProc. NeurIPS, 2018, pp. 3318–3329
2018
-
[4]
Route-centric ant-inspired memories enable panoramic route-following in a car-like robot,
G. G. Gattaux, A. Wystrach, J. R. Serres, and F. Ruffier, “Route-centric ant-inspired memories enable panoramic route-following in a car-like robot,”Nature Communications, vol. 16, no. 1, p. 8328, 2025
2025
-
[5]
Affordance detection of tool parts from geometric features,
A. Myers, C. L. Teo, C. Ferm ¨uller, and Y . Aloimonos, “Affordance detection of tool parts from geometric features,” inProc. ICRA, 2015, pp. 1374–1381
2015
-
[6]
The theory of affordances:(1979),
J. J. Gibson, “The theory of affordances:(1979),” inThe people, place, and space reader. Routledge, 2014, pp. 56–60
1979
-
[7]
LOCATE: Localize and transfer object parts for weakly supervised affordance grounding,
G. Li, V . Jampani, D. Sun, and L. Sevilla-Lara, “LOCATE: Localize and transfer object parts for weakly supervised affordance grounding,” inProc. CVPR, 2023, pp. 10 922–10 931
2023
-
[8]
Weakly supervised multimodal affordance grounding for egocentric images,
L. Xu, Y . Gao, W. Song, and A. Hao, “Weakly supervised multimodal affordance grounding for egocentric images,” inProc. AAAI, vol. 38, no. 6, 2024, pp. 6324–6332
2024
Show all 36 references
-
[9]
One-shot open affordance learning with foundation models,
G. Li, D. Sun, L. Sevilla-Lara, and V . Jampani, “One-shot open affordance learning with foundation models,” inProc. CVPR, 2024, pp. 3086–3096
2024
-
[10]
Affordan- ceLLM: Grounding affordance from vision language models,
S. Qian, W. Chen, M. Bai, X. Zhou, Z. Tu, and L. E. Li, “Affordan- ceLLM: Grounding affordance from vision language models,” inProc. CVPR, 2024, pp. 7587–7597
2024
-
[11]
SphereNet: Learning spherical representations for detection and classification in omnidirec- tional images,
B. Coors, A. P. Condurache, and A. Geiger, “SphereNet: Learning spherical representations for detection and classification in omnidirec- tional images,” inProc. ECCV, 2018, pp. 518–533
2018
-
[12]
Geometric exploitation for indoor panoramic semantic segmentation,
D. Cao Dinh, S. J. Kim, and K. Cho, “Geometric exploitation for indoor panoramic semantic segmentation,” inProc. NeurIPS, 2024, pp. 26 355–26 376
2024
-
[13]
Learning affordance grounding from exocentric images,
H. Luo, W. Zhai, J. Zhanget al., “Learning affordance grounding from exocentric images,” inProc. CVPR, 2022, pp. 2252–2261
2022
-
[14]
AffordanceNet: An end-to-end deep learning approach for object affordance detection,
T.-T. Do, A. Nguyen, and I. Reid, “AffordanceNet: An end-to-end deep learning approach for object affordance detection,” inProc. ICRA, 2018, pp. 5882–5889
2018
-
[15]
Weakly supervised affordance detection,
J. Sawatzky, A. Srikantha, and J. Gall, “Weakly supervised affordance detection,” inProc. CVPR, 2017, pp. 2795–2804
2017
-
[16]
Resource-efficient affordance grounding with com- plementary depth and semantic prompts,
Y . Huanget al., “Resource-efficient affordance grounding with com- plementary depth and semantic prompts,” inProc. IROS, 2025, pp. 7788–7795
2025
-
[17]
Selective contrastive learning for weakly supervised affordance grounding,
W. Moon, H. S. Seong, and J.-P. Heo, “Selective contrastive learning for weakly supervised affordance grounding,” inProc. ICCV, 2025, pp. 5210–5220
2025
-
[18]
Weakly-supervised affordance grounding guided by part-level semantic priors,
P. Xu and Y . MU, “Weakly-supervised affordance grounding guided by part-level semantic priors,” inProc. ICLR, 2025
2025
-
[19]
Affordancesam: Segment anything once more in affordance grounding,
D. Jiang, Z. Wang, H. Li, S. Dang, T. Ma, W. Wei, G. Dai, L. Zhang, and M. Wang, “Affordancesam: Segment anything once more in affordance grounding,”arXiv preprint arXiv:2504.15650, 2025
2025 arXiv
-
[20]
Worldafford: Affordance grounding based on natural language instructions,
C. Chen, Y . Cong, and Z. Kan, “Worldafford: Affordance grounding based on natural language instructions,” inProc. ICTAI, 2024, pp. 822–828
2024
-
[21]
Seeing the unseen: Visual common sense for semantic placement,
R. Ramrakhya, A. Kembhavi, D. Batra, Z. Kira, K.-H. Zeng, and L. Weihs, “Seeing the unseen: Visual common sense for semantic placement,” inProc. CVPR, 2024, pp. 16 273–16 283
2024
-
[22]
Scene- Fun3D: Fine-grained functionality and affordance understanding in 3D scenes,
A. Delitzas, A. Takmaz, F. Tombari, R. Sumneret al., “Scene- Fun3D: Fine-grained functionality and affordance understanding in 3D scenes,” inProc. CVPR, 2024, pp. 14 531–14 542
2024
-
[23]
Grounding 3D scene affordance from egocentric interactions,
C. Liuet al., “Grounding 3D scene affordance from egocentric interactions,”arXiv preprint arXiv:2409.19650, 2024
2024 arXiv
-
[24]
AffordBot: 3D fine-grained embodied reasoning via multimodal large language models,
X. Wang, X. Yang, Y . Xu, Y . Wu, Z. Li, and N. Zhao, “AffordBot: 3D fine-grained embodied reasoning via multimodal large language models,”arXiv preprint arXiv:2511.10017, 2025
2025
-
[25]
Bending reality: Distortion-aware transformers for adapting to panoramic semantic segmentation,
J. Zhang, K. Yang, C. Ma, S. Reiß, and ohters, “Bending reality: Distortion-aware transformers for adapting to panoramic semantic segmentation,” inProc. CVPR, 2022, pp. 16 917–16 927
2022
-
[26]
PanoContext-Former: Panoramic total scene understanding with a transformer,
Y . Dong, C. Fang, L. Bo, Z. Dong, and P. Tan, “PanoContext-Former: Panoramic total scene understanding with a transformer,” inProc. CVPR, 2024, pp. 28 087–28 097
2024
-
[27]
DINOv2: Learning robust visual features without supervision,
M. Oquabet al., “DINOv2: Learning robust visual features without supervision,”Transactions on Machine Learning Research Journal, pp. 1–31, 2023
2023
-
[28]
LoRA: Low-rank adaptation of large language models,
E. J. Huet al., “LoRA: Low-rank adaptation of large language models,” inProc. ICLR, 2022
2022
-
[29]
Learning transferable visual models from natural language supervision,
A. Radfordet al., “Learning transferable visual models from natural language supervision,” inProc. ICML, 2021, pp. 8748–8763
2021
-
[30]
Learning to prompt for vision-language models,
K. Zhou, J. Yang, C. C. Loy, and Z. Liu, “Learning to prompt for vision-language models,”International Journal of Computer Vision, vol. 130, no. 9, pp. 2337–2348, 2022
2022
-
[31]
360-Indoor: Towards learning real-world objects in 360° indoor equirectangular images,
S.-H. Chou, C. Sun, W.-Y . Chang, W.-T. Hsu, M. Sun, and J. Fu, “360-Indoor: Towards learning real-world objects in 360° indoor equirectangular images,” inProc. WACV, 2020, pp. 845–853
2020
-
[32]
Gibson env: Real-world perception for embodied agents,
F. Xia, A. R. Zamir, Z. Heet al., “Gibson env: Real-world perception for embodied agents,” inProc. CVPR, 2018, pp. 9068–9079
2018
-
[33]
PanoContext: A whole-room 3D context model for panoramic scene understanding,
Y . Zhang, S. Song, P. Tan, and J. Xiao, “PanoContext: A whole-room 3D context model for panoramic scene understanding,” inProc. ECCV, 2014, pp. 668–686
2014
-
[34]
Recognizing scene viewpoint using panoramic place representation,
J. Xiao, K. A. Ehinger, A. Oliva, and A. Torralba, “Recognizing scene viewpoint using panoramic place representation,” inProc. CVPR, 2012, pp. 2695–2702
2012
-
[35]
One-shot affordance grounding of deformable objects in egocentric organizing scenes,
W. Jia, F. Yanget al., “One-shot affordance grounding of deformable objects in egocentric organizing scenes,” inProc. IROS, 2025, pp. 21 243–21 250
2025
-
[36]
Closed-loop transfer for weakly-supervised affordance grounding,
J. Tang, Z. Wei, G. Zheng, and S. Yang, “Closed-loop transfer for weakly-supervised affordance grounding,” inProc. ICCV, 2025, pp. 9530–9539
2025
Reviewed August 2, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.