REVIEW 2 major objections 3 minor 50 references
Is the Geometry Doing the Work? An Operating-Point Audit of Hierarchy in Hyperbolic Vision-Language Models
T0 review · 2 major / 3 minor · reviewed 2026-08-02 · deepseek-v4-flash
Pith's one-line read The hyperbolic geometry in audited vision-language models is effectively dormant: embeddings remain near-Euclidean and trained entailment cones are saturated.
desk verdict Operating-point audit that lands, but the cone-saturation mechanism is compromised by a reciprocal-inconsistent aperture equation. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The key machinery is the dimensionless operating coordinate u=√cρ (with local distortion factor H(u)=u/asinh(u)), which lets the audit separate learning the curvature scalar from actually using hyperbolic geometry; the entailment-cone half-aperture ω=arcsin(min{1,2K/(√cρ)}) with K=0.1, whose saturation edge √cρ≤2K gives a closed-form criterion for when cones are trivially wide; and a gradient decomposition that attributes curvature movement to individual loss terms. Together these turn a benchmark question into an operating-point question, and ground the paper's proposed five-number geometry report.
What would settle it
Inspect the cone-aperture formula in each released implementation and measure the trained cone apertures while halving c with norms held fixed; if apertures shrink rather than grow, the low-curvature shortcut is inverted and the saturation-edge explanation (√cρ≤2K) collapses.
Extended reading notes
Core claim
Central claim: audited hyperbolic vision-language formulations do not demonstrate an operative radial or cone-based hierarchy. Effective geometry is governed by u=√cρ, and every converged checkpoint sits at u≈0.1–0.37—below the u≈0.84 10%-distortion point; unclamping curvature changes c and norms but not this band. Trained entailment cones saturate at π/2, so low violation rates are trivial containment, shuffle-controlled tests find no pair-specific radial ordering, and traversal yields only weak branch-conditioned order. The mechanism is a low-curvature shortcut: cone half-aperture widens as √cρ shrinks toward saturation edge 2K=0.2, so entailment suppresses violations by lowering curvature
Load-bearing premise
The mechanism account depends on the implementations using the reciprocal cone-aperture formula (aperture grows as 1/√cρ); if the formula as printed in Eq. (10) is what the code actually uses, lowering curvature would narrow the cones and the entire low-curvature-shortcut explanation would run backwards.
Editorial extensions
If this is right
- If these diagnostics are accepted, prior claims that hyperbolic VLMs encode hierarchy based on taxonomy-distance correlation, leaf-level accuracy, or zero violation rates need re-examination: those signals are compatible with angular structure and saturated cones.
- Future hyperbolic VLM claims should report the five-number geometry report (operating point, cone-saturation state, directed violations, shuffle-controlled radial excess, radial increment beyond angle); without it, a positive hierarchy claim is not mechanism-tested.
- The low-curvature shortcut implies that simply adding entailment losses or lowering the curvature floor will not activate hierarchy; objectives must decouple aperture width from norm/curvature and include radial-growth control.
- Because entailment-off training also collapses curvature, fixing the cone loss alone cannot stabilize a nonlocal operating point; the contrastive/alignment objective must be modified as well.
- The analytic saturation edge √cρ≤2K gives a parameter-free check: models whose trained parent coordinates lie at or below 0.2 cannot have informative trained cones.
Reading between the lines
- Beyond the paper: the u=√cρ operating-point test transfers to any hyperbolic embedding claim (word embeddings, knowledge graphs): if learned radii never push u past about 0.8, negative-curvature structure is probably not doing the work.
- A direct test the paper leaves open is a K-sweep: if the saturation edge 2K causally anchors trained parent coordinates, varying K should move the observed endpoints; if the endpoints stay fixed, the edge is a correlation, not a mechanism.
- The aperture formula in Eq. (10) deserves code-level verification; if the non-reciprocal form is actually implemented, the shortcut's direction reverses and the saturation classification needs re-deriving.
- A testable extension: replace hyperbolic encoders with Euclidean encoders plus a learned radial scale; if downstream hierarchy metrics are preserved, angular and supervision effects are sufficient and the hyperbolic geometry is not needed.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper audits whether MERU, HyCoCLIP, and PHyCLIP hyperbolic vision-language models actually use the radial/cone hierarchy mechanism they are motivated by. Using released checkpoints and matched from-scratch interventions on a fixed GRIT snapshot, it reports that all converged checkpoints remain near-Euclidean in the dimensionless radius u=√cρ (largest image-side value 0.37, far below the 10%-distortion marker near 0.84), that unclamping curvature does not move the operating point, that trained entailment cones are saturated or nearly saturated, that shuffle-controlled directed radial tests are null or unreplicated across seeds, and that calibrated semantic traversal detects only partial branch-conditioned order. The paper attributes the curvature collapse partly to a low-curvature shortcut in the entailment objective, and proposes a five-number geometry report for future hierarchy claims.
Significance. If the central claims survive, this is an important negative result with an unusually rigorous diagnostic apparatus: preregistered traversal thresholds, planted synthetic controls for sensitivity, power/MDE analyses, matched within-snapshot interventions, and a reproducibility suite with checkpoint hashes and analytic-versus-code tests. The paper carefully separates angular semantic organization from radial hierarchy and gives a mechanistic account of why the audited formulations leave the geometry dormant. The main obstacle is the internally inconsistent aperture formula, which is load-bearing for the low-curvature-shortcut mechanism and for the interpretation of near-zero entailment violations as trivial containment.
major comments (2)
- [§7.3, Eq. (10)] Eq. (10) prints ω(ρ)=arcsin(min{1,2K√cρ}), but the saturation criterion used in Tables 4, 13, 14 and in the closed-form boundary later in the same section is ω=arcsin(min{1,2K/(√cρ)}), which saturates at π/2 when √cρ≤2K. The two forms are reciprocals. Under the printed form, lowering curvature narrows the cone, and at the measured text-side u≈0.12–0.20 the trained text-parent cones would have half-apertures of only ~0.02–0.04 rad. In that regime, the reported 0–0.4% text→image violation rates would be evidence of finely learned directed order, directly contradicting the paper's central claim. The low-curvature shortcut and the 'saturated cone ⇒ trivial containment' interpretation both depend on the reciprocal form. The manuscript contains both formulas without identifying which one is implemented. This must be resolved by checking the released/from-scratch code and correcting Eq. (10) or
- [§7.3/Tables 4, 13, 14] The saturation classification used throughout the cone diagnostics is the reciprocal form (saturated when √cρ≤2K with K=0.1). The same section states that the analytic identity and endpoint match carry the mechanistic claim. Because the endpoint evidence (trained box-image parents ending within 0.013 of 2K) is derived from this same aperture identity, the internal inconsistency in Eq. (10) propagates into the main evidence for the low-curvature shortcut. Please state explicitly which aperture expression each implementation uses, provide the code-level verification promised in the reproducibility statement, and, if Eq. (10) is a typo, mark it as such and correct every later reference.
minor comments (3)
- [§7.3 text after Eq. (10)] The sentence 'As foreshadowed in Section 5.2, reducing c widens the aperture' is inconsistent with the printed Eq. (10), under which reducing c narrows the aperture. This cross-reference should be fixed together with the formula.
- [Appendix A / §7.5] The notation table and the five-number geometry report both use the reciprocal aperture form. Once Eq. (10) is corrected, please ensure the notation table, the closed-form edge, and all table captions use one consistent expression.
- [Table 14 caption] The saturation definition (ω≥π/2−0.01) is given only in the caption. Consider stating it in Section 5.2 or Section 7.3, since the interpretation of 'saturated' is central to the argument.
Circularity Check
No significant circularity: the audit is self-contained, uses external benchmarks and planted controls, and explicitly disclaims the one analytic identity it relies on.
full rationale
The paper's central claims are measurements and mechanistic analyses that do not reduce to their own inputs. The near-Euclidean operating point is measured from released checkpoints and matched from-scratch runs via u=√cρ and H(u)=u/asinh(u); the paper explicitly notes that H(u) and √cρ are monotone functions of the same quantity, so no independent prediction is being made from a fitted quantity. The cone-saturation diagnosis is based on the closed-form criterion √cρ≤2K with the models' fixed K=0.1 and on measured parent coordinates; the paper openly states, 'This is an identity, not an independent prediction; the falsifiable observation is where the trained parent coordinates end.' The gradient decomposition is computed from per-loss gradients on unmodified training trajectories and is corroborated by entailment-off ablations and planted positive controls. No load-bearing self-citations appear, and the paper directly addresses its own self-sealing concern by citing planted controls and measured sensitivity (e.g., MDE80). The reciprocal-inconsistent aperture formula in Eq. (10) versus the reciprocal form used elsewhere is a real internal-consistency and correctness risk that could invert the mechanism interpretation if implementations follow Eq. (10), but it is not a circular derivation and therefore does not affect the circularity score.
Assumptions & free parameters
assumptions (4)
- standard math The Lorentz/Poincaré exponential-map distortion H(u)=u/asinh(u) with u=√cρ correctly measures how strongly an embedding experiences hyperbolic geometry.
- domain assumption The entailment-cone aperture is ω=arcsin(min{1,2K/(√cρ)}) with K=0.1, so saturation occurs at √cρ≤2K.
- domain assumption The current-GRIT snapshot (13.1M of the documented 20.5M pairs) is a sufficient matched basis for the from-scratch interventions and for connecting released checkpoints to the mechanism.
- domain assumption Only pair-specific radial order beyond a shuffle null, and non-saturated cone activity, count as operative hierarchy; marginal level-wise norm codes are not sufficient.
Cite this review
Pith. "Pith review of Is the Geometry Doing the Work? An Operating-Point Audit of Hierarchy in Hyperbolic Vision-Language Models." pith.science (2026). https://pith.science/paper/AAVRH44L
@misc{pith2026260705268,
author = {Pith},
title = {Pith review of: Is the Geometry Doing the Work? An Operating-Point Audit of Hierarchy in Hyperbolic Vision-Language Models},
year = {2026},
howpublished = {\url{https://pith.science/paper/AAVRH44L}},
note = {Machine review of arXiv:2607.05268}
}
abstract
Hyperbolic vision-language models are designed to encode abstraction geometrically: general concepts near the origin, specific ones farther out, and entailment cones representing directed order. We ask whether trained MERU, HyCoCLIP, and PHyCLIP models actually use these mechanisms. We audit seven released checkpoints and matched from-scratch interventions, using diagnostics that distinguish active hyperbolic geometry from angular structure and supervision effects. All audited converged checkpoints remain near-Euclidean in the dimensionless radius $u=\sqrt{c}\rho$, which measures how strongly embeddings experience hyperbolic geometry: the largest observed image-side value is $0.37$ -- well below $u\approx0.84$, where local metric distortion reaches $10\%$. Releasing the curvature floor changes curvature and norms but not this regime, with mixed, generally modest downstream shifts. Trained entailment cones are saturated or nearly saturated, so low violation rates can arise from trivially wide cones rather than learned order. Preregistered semantic traversal detects weak within-branch order but no operative full-hierarchy readout. Shuffle-controlled tests detect no pair-specific radial ordering in released checkpoints, and no positive result is consistent across all three matched ViT-B seeds. We trace this to a low-curvature shortcut: lowering curvature widens entailment cones and suppresses violations without learning order. In the probed trajectories, gradient decomposition identifies entailment as the dominant curvature-lowering pressure during collapse. Yet curvature contracts even when entailment is removed, so the shortcut is not the sole cause. Under our diagnostics, the audited formulations do not demonstrate an operative radial or cone-based hierarchy. We distill the audit into a five-number geometry report for evaluating future hierarchy claims.
Figures
Figures from the paper (7 more)
Reference graph
Works this paper leans on
-
[1]
Comparing euclidean and hyperbolic embeddings on the wordnet nouns hypernymy graph
Sameer Bansal and Adrian Benton. Comparing euclidean and hyperbolic embeddings on the wordnet nouns hypernymy graph. In Proceedings of the Second Workshop on Insights from Negative Results in NLP, pp.\ 49--53, 2021
2021
-
[2]
Metric spaces of non-positive curvature, volume 319
Martin R Bridson and Andr \'e Haefliger. Metric spaces of non-positive curvature, volume 319. Springer Science & Business Media, 1999
1999
-
[3]
Embedding geometries of contrastive language-image pre-training
Jason Chuan-Chih Chou and Nahid Alam. Embedding geometries of contrastive language-image pre-training. In European Conference on Computer Vision, pp.\ 399--416. Springer, 2024
2024
-
[4]
Imagenet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp.\ 248--255. Ieee, 2009
2009
-
[5]
Hyperbolic image-text representations
Karan Desai, Maximilian Nickel, Tanmay Rajpurohit, Justin Johnson, and Shanmukha Ramakrishna Vedantam. Hyperbolic image-text representations. In International Conference on Machine Learning, pp.\ 7694--7731. PMLR, 2023
2023
-
[6]
Hyperbolic entailment cones for learning hierarchical embeddings
Octavian Ganea, Gary B \'e cigneul, and Thomas Hofmann. Hyperbolic entailment cones for learning hierarchical embeddings. In International conference on machine learning, pp.\ 1646--1655. PMLR, 2018
2018
-
[8]
Learning mixed-curvature representations in product spaces
Albert Gu, Frederic Sala, Beliz Gunel, and Christopher R \'e . Learning mixed-curvature representations in product spaces. In International Conference on Learning Representations (ICLR), 2019
2019
-
[9]
Yunhui Guo, Xudong Wang, Yubei Chen, and Stella X. Yu. Clipped hyperbolic classifiers are super-hyperbolic classifiers. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022
2022
Show all 50 references
-
[10]
Sugarcrepe: Fixing hackable benchmarks for vision-language compositionality
Cheng-Yu Hsieh, Jieyu Zhang, Zixian Ma, Aniruddha Kembhavi, and Ranjay Krishna. Sugarcrepe: Fixing hackable benchmarks for vision-language compositionality. Advances in neural information processing systems, 36: 0 31096--31116, 2023
2023
-
[12]
Intriguing properties of hyperbolic embeddings in vision-language models
Sarah Ibrahimi, Mina Ghadimi Atigh, Nanne Van Noord, Pascal Mettes, and Marcel Worring. Intriguing properties of hyperbolic embeddings in vision-language models. Transactions on Machine Learning Research, 2024
2024
-
[13]
Learning multiple layers of features from tiny images
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009
2009
-
[14]
Microsoft coco: Common objects in context
Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll \'a r, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pp.\ 740--755. Springer, 2014
2014
-
[15]
Wordnet: a lexical database for english
George A Miller. Wordnet: a lexical database for english. Communications of the ACM, 38 0 (11): 0 39--41, 1995
1995
-
[16]
Poincar \'e embeddings for learning hierarchical representations
Maximilian Nickel and Douwe Kiela. Poincar \'e embeddings for learning hierarchical representations. Advances in neural information processing systems, 30, 2017
2017
-
[17]
Compositional entailment learning for hyperbolic vision-language models
Avik Pal, Max Van Spengler, Guido D'Amely di Melendugno, Alessandro Flaborea, Fabio Galasso, and Pascal Mettes. Compositional entailment learning for hyperbolic vision-language models. In International Conference on Learning Representations, volume 2025, pp.\ 87371--87399, 2025
2025
-
[18]
Grounding multimodal large language models to the world
Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, Qixiang Ye, and Furu Wei. Grounding multimodal large language models to the world. In International Conference on Learning Representations, volume 2024, pp.\ 51575--51598, 2024
2024
-
[19]
Learning transferable visual models from natural language supervision
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp...
2021
-
[20]
Accept the modality gap: An exploration in the hyperbolic space
Sameera Ramasinghe, Violetta Shevchenko, Gil Avraham, and Ajanthan Thalaiyasingam. Accept the modality gap: An exploration in the hyperbolic space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 27263--27272, 2024
2024
-
[21]
Imagenet large scale visual recognition challenge
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115 0 (3): 0 211--252, 2015
2015
-
[22]
Representation tradeoffs for hyperbolic embeddings
Frederic Sala, Christopher De Sa, Albert Gu, and Christopher R \'e . Representation tradeoffs for hyperbolic embeddings. In International Conference on Machine Learning (ICML), 2018
2018
-
[23]
Low distortion delaunay embedding of trees in hyperbolic plane
Rik Sarkar. Low distortion delaunay embedding of trees in hyperbolic plane. In International symposium on graph drawing, pp.\ 355--366. Springer, 2011
2011
-
[25]
PHyCLIP : _1 -product of hyperbolic factors unifies hierarchy and compositionality in vision-language representation learning
Daiki Yoshikawa and Takashi Matsubara. PHyCLIP : _1 -product of hyperbolic factors unifies hierarchy and compositionality in vision-language representation learning. arXiv preprint arXiv:2510.08919, 2025
2025
-
[26]
From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions
Peter Young, Alice Lai, Micah Hodosh, and Julia Hockenmaier. From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions. Transactions of the association for computational linguistics, 2: 0 67--78, 2014
2014
-
[28]
Communications of the ACM , volume=
WordNet: a lexical database for English , author=. Communications of the ACM , volume=. 1995 , publisher=
1995
-
[29]
Advances in neural information processing systems , volume=
Sugarcrepe: Fixing hackable benchmarks for vision-language compositionality , author=. Advances in neural information processing systems , volume=
-
[30]
arXiv preprint arXiv:2207.00221 , year=
Vl-checklist: Evaluating pre-trained vision-language models with objects, attributes and relations , author=. arXiv preprint arXiv:2207.00221 , year=
-
[31]
International journal of computer vision , volume=
Imagenet large scale visual recognition challenge , author=. International journal of computer vision , volume=. 2015 , publisher=
2015
-
[32]
International Conference on Learning Representations (ICLR) , year=
Learning Mixed-Curvature Representations in Product Spaces , author=. International Conference on Learning Representations (ICLR) , year=
-
[33]
International Conference on Machine Learning (ICML) , year=
Representation Tradeoffs for Hyperbolic Embeddings , author=. International Conference on Machine Learning (ICML) , year=
-
[34]
European conference on computer vision , pages=
Microsoft coco: Common objects in context , author=. European conference on computer vision , pages=. 2014 , organization=
2014
-
[35]
1999 , publisher=
Metric spaces of non-positive curvature , author=. 1999 , publisher=
1999
-
[36]
Transactions of the association for computational linguistics , volume=
From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions , author=. Transactions of the association for computational linguistics , volume=
-
[37]
2009 , publisher=
Learning multiple layers of features from tiny images , author=. 2009 , publisher=
2009
-
[38]
2009 IEEE conference on computer vision and pattern recognition , pages=
Imagenet: A large-scale hierarchical image database , author=. 2009 IEEE conference on computer vision and pattern recognition , pages=. 2009 , organization=
2009
-
[39]
International conference on machine learning , pages=
Learning transferable visual models from natural language supervision , author=. International conference on machine learning , pages=. 2021 , organization=
2021
-
[40]
International Conference on Learning Representations , volume=
Grounding multimodal large language models to the world , author=. International Conference on Learning Representations , volume=
-
[41]
International Conference on Machine Learning , pages=
Hyperbolic image-text representations , author=. International Conference on Machine Learning , pages=. 2023 , organization=
2023
-
[42]
International Conference on Learning Representations , volume=
Compositional entailment learning for hyperbolic vision-language models , author=. International Conference on Learning Representations , volume=
-
[43]
Yoshikawa, Daiki and Matsubara, Takashi , journal=
-
[44]
International symposium on graph drawing , pages=
Low distortion delaunay embedding of trees in hyperbolic plane , author=. International symposium on graph drawing , pages=. 2011 , organization=
2011
-
[45]
Nickel, Maximilian and Kiela, Douwe , journal=. Poincar
-
[46]
International conference on machine learning , pages=
Hyperbolic entailment cones for learning hierarchical embeddings , author=. International conference on machine learning , pages=. 2018 , organization=
2018
-
[47]
Transactions on Machine Learning Research , year=
Intriguing properties of hyperbolic embeddings in vision-language models , author=. Transactions on Machine Learning Research , year=
-
[48]
arXiv preprint arXiv:1511.06361 , year=
Order-embeddings of images and language , author=. arXiv preprint arXiv:1511.06361 , year=
-
[49]
Proceedings of the Second Workshop on Insights from Negative Results in NLP , pages=
Comparing Euclidean and hyperbolic embeddings on the WordNet nouns hypernymy graph , author=. Proceedings of the Second Workshop on Insights from Negative Results in NLP , pages=
-
[50]
arXiv preprint arXiv:2303.02995 , year=
Hiclip: Contrastive language-image pretraining with hierarchy-aware attention , author=. arXiv preprint arXiv:2303.02995 , year=
-
[51]
European Conference on Computer Vision , pages=
Embedding geometries of contrastive language-image pre-training , author=. European Conference on Computer Vision , pages=. 2024 , organization=
2024
-
[52]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
Accept the modality gap: An exploration in the hyperbolic space , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
-
[53]
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year=
Clipped Hyperbolic Classifiers Are Super-Hyperbolic Classifiers , author=. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year=
-
[54]
arXiv preprint arXiv:2603.23311 , year=
ARGENT: Adaptive Hierarchical Image-Text Representations , author=. arXiv preprint arXiv:2603.23311 , year=
Reviewed August 2, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.