Pith. sign in

REVIEW 4 major objections 6 minor 19 references

Exploring the Stratified Space Structure of an RL Game with the Volume Growth Transform

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A transformer trained on a visual RL game embeds observations in a stratified space, not a manifold, with local dimension spiking at sub-goals.

desk verdict Interesting hypothesis about stratified RL latent spaces, but the evidence is not yet load-bearing and the supporting theorem has a serious flaw. read the letter →

arxiv 2507.22010 v1 pith:FYOHWGWM submitted 2025-07-29 math.AT cs.AIcs.CGcs.LGmath.DG

Justin Curry , Brennan Lagasse , Ngoc B. Lam , Gregory Cox , David Rosenbluth , Alberto Speranzon This is my paper
classification math.ATcs.AIcs.CGcs.LGmath.DG MSC 57N8055R8068T07
keywords stratifiedspacesvolumegrowthtransformlocaldimensionreinforcementlearningtokenembeddingsmanifoldhypothesistransformerlatentspacegeometry
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that the latent token space of a transformer trained to play a visual reinforcement-learning game is not a manifold with one intrinsic dimension, but a stratified space whose local dimension changes from point to point. It adapts the volume growth transform, previously applied to large language models, to image tokens from a two-coin variant of the Searing Spotlights game, and finds local dimensions clustered in bands from about 6 to 21. It also follows local dimension through single episodes and reports that low values coincide with stable sub-strategy execution while spikes coincide with coin collection, new spotlights, or moments of apparent indecision. If the interpretation holds, the distribution of local dimensions becomes a geometric indicator of complexity for RL environments, and averaging intrinsic dimension over all tokens is the wrong summary statistic.

What carries the argument

The central object is the volume growth transform $\mathrm{VGT}_x(s) = \log \mu(B_x(e^s))$, which records how the measure of a ball around a token grows as the radius is increased exponentially. The paper estimates it by counting neighboring tokens in Euclidean distance and fits the log-log curve with a linear least-squares problem in $\log \hat{H}_x$ and $\hat{n}_x$; the fitted slope $\hat{n}_x$ is interpreted as the local dimension. The argument is carried by two structural tools: the fiber-bundle hypothesis, which predicts only slope decreases in growth curves and therefore serves as the null hypothesis that spike-like curves rebut, and the realization theorem, which constructs a stratified space realizing any admissible piecewise-linear growth curve by gluing half-discs of varying dimensions at zenith and center points. The theorem shows the volume-growth data are compatible with stratified geometry.

What would settle it

Run the identical volume-growth pipeline on synthetic point clouds that are known manifolds (for example, a low-dimensional sphere or torus embedded in $\mathbb{R}^{256}$ with the same token count and density), using the same volume range 50–90. If the pipeline yields a multimodal dimension histogram or slope increases in the growth curves, then the method cannot distinguish manifolds from stratified spaces and the empirical claim collapses. Alternatively, re-estimate local dimensions on fresh episodes using a radius interval fixed before seeing any growth curves and check whether the four clusters and the coin-collection spikes reproduce.

Watch

Extended reading notes

Core claim

The central discovery is that the token embedding space of a transformer-based PPO agent trained on a two-coin variant of the Searing Spotlights game is not a manifold and not a fiber bundle, but a stratified space whose local dimension varies across the token set. Local dimensions estimated by counting tokens within growing Euclidean balls cluster into four ranges (roughly 6–8, 9–10, 11–13, and 14–21), and individual growth curves such as token 4040 show slope increases after decreases, which the fiber-bundle hypothesis forbids. Along individual episodes, the local dimension function stays low during stable sub-strategy execution and spikes at moments of sub-goal achievement or high environmental complexity, such as when a coin is collected or multiple spotlights enter. The paper also proves a realization theorem: any non-decreasing piecewise-linear growth curve with natural-number slopes can be realized as the volume growth transform of some stratified space.

Load-bearing premise

The load-bearing premise is that the least-squares fit in Equation (5) over the volume range 50 to 90 recovers the true local dimension: the curvature correction $O(r^4)$ must be negligible, token density must be roughly constant, and the choice of that range—made by inspecting the same growth curves it is used to measure—must not be selecting the pattern it reports.

Editorial extensions

If this is right

  • For this game, the manifold hypothesis and the fiber-bundle hypothesis are rejected: dimension histograms are multimodal, and some growth curves show slope increases that a fiber-bundle model cannot produce.
  • Local dimension should be reported as a distribution over tokens, not averaged into a single intrinsic dimension for the dataset.
  • Along an episode, a rise in local dimension is a candidate event marker for sub-goal completion, increased obstacle complexity, or policy indecision, which could segment trajectories without reward labels.
  • Because high-dimensional tokens mark situations the agent finds complex, an adaptive training schedule could select those frames as extra training examples.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct extension the paper leaves implicit: if local dimension tracks decision conflict rather than raw object count, the same volume-growth pipeline could serve as a proxy for policy entropy in other transformer-based RL agents, without needing access to the policy head.
  • The realization theorem implies that volume-growth curves are far from a complete invariant of stratification: many different stratified spaces realize the same growth curve, so local-dimension estimates should be paired with other structure, such as persistent homology of the token cloud, before claiming a specific decomposition into strata.
  • A testable refinement: retrain the agent with reward shaping that removes the spotlight penalty; if spikes near obstacles persist, high dimension tracks perceptual complexity, whereas if they vanish, it tracks goal-related uncertainty.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper adapts the volume growth transform (VGT) of Robinson et al. to the token embedding space of a transformer-based PPO agent trained on a modified two-coin version of the Searing Spotlights environment. For each token, local dimension is estimated by a least-squares fit of log neighbor-count versus log radius over a chosen volume range, and the resulting histograms and trajectory plots are interpreted as evidence that the latent space is not a manifold but a stratified space. The paper also contributes a theorem claiming that any non-decreasing piecewise-linear function can be realized exactly as the VGT of a stratified space at a point, with the proof deferred to Appendix A.

Significance. If the empirical findings survive scrutiny, this would be the first report of stratified-structure evidence in the visual latent space of an RL agent, and the proposed link between local-dimension bursts and sub-goal events is a concrete, falsifiable geometric indicator of task complexity. The adaptation of the VGT from LLMs to RL is a natural and potentially useful extension, and the realization theorem, if correct, would strengthen the mathematical foundations of the method. The paper is also honest about its heuristic choices, explicitly describing the post-hoc selection of the volume range in Section 3.1. However, the empirical evidence currently lacks the controls needed to rule out estimator artifacts, and the theorem's proof is incomplete as written, so the central claims are not yet established.

major comments (4)
  1. [Appendix A, Theorem 1] The construction glues half-discs at a point but does not specify a metric or a measure on the quotient space, which are required for the VGT to be defined. With the natural quotient metric, a ball centered at c^{n1} that has entered the second half-disc has volume approximately C_{n1} r^{n1} + C_{n2} (r - r1)^{n2}, whose logarithm is not affine in log r and therefore cannot equal the stated piecewise-linear f. The proof only asserts the equality. Please either supply a correct construction (for example, using a metric or measure that makes the cumulative volume exactly piecewise exponential in log r) or weaken the theorem to a statement about asymptotic slopes, and justify the claim in either case.
  2. [Section 3.1, Equation (5)] The volume range [50, 90] used for the central local-dimension estimates was selected 'based on an inspection of the local dimension plots in Figure 5,' i.e., on the same curves being fitted. No sensitivity analysis is reported for neighboring ranges, and no null baseline is provided. Applying the same pipeline with the same selection heuristic to a point cloud sampled from a single smooth n-manifold in R^256 with roughly 4500 tokens would very plausibly also produce a spread of fitted slopes because of boundary and density effects, and the heuristic would again select a linear-looking window. I ask for (i) a synthetic single-manifold baseline with matched token count and ambient dimension, (ii) bootstrap or repeated-seed error bars on the histograms in Figure 3, and (iii) estimates for several volume ranges reported side by side. If the multimodal structure persists under these controls, the stratified claim would be substantially stronger.
  3. [Section 2, Equation (3)] The identification of the least-squares slope with the true local dimension requires that the O(r^4) term is negligible over the entire fitted interval, that the point density is uniform at the probed scales (implicit in the constant M in Equation (4)), and that the fit range is not itself responsible for the apparent linearity. None of these assumptions is checked. For a finite sample of about 4500 tokens in a 256-dimensional ambient space, the useful radius range may be narrow, and a 'most linear' regime can arise from the middle of a crossover between scales rather than from manifold structure. Please report the fitted intercepts, residuals, and neighbor-density profiles for representative tokens, and, if possible, validate the slope-recovery step on synthetic point clouds with known dimension.
  4. [Section 3.2, Figures 6a and 6b] The trajectory-level interpretation is based on visual inspection of two hand-picked episodes. The text makes broader claims, such as that local-dimension spikes 'generally occur before the agent reaches a goal state' (Section 4) and that higher dimension correlates with spotlight complexity, but no statistical test is offered. A systematic analysis over the 250 collected episodes, comparing local dimension around coin-collection events and spotlight-count changes against a permutation or bootstrap null, would be needed to support these conclusions. As written, the two traces cannot distinguish a genuine signal from arbitrary temporal fluctuations in the estimator.
minor comments (6)
  1. [Section 3.1, Equation (5)] Please define p and the indexing of the radii r_{i,x} explicitly; the left-hand side (0, log 2, ..., log p)^T is clear only after reading that 'there are i tokens within distance r_{i,x}', but p is never formally introduced.
  2. [Figure 6] The figure has no caption explaining the plotted quantity, the time axis, or the meaning of the highlighted regions, and the spikes discussed in the text are not marked directly on the figure; this makes the trajectory analysis difficult to follow.
  3. [Figure 5] The figure does not label which curve corresponds to which token, even though the text refers to tokens 2434, 2833, 4040, and 4105; please add labels or a legend.
  4. [Appendix B, Reference [9]] The reference to Lurie's Higher Algebra is listed as 'Not stated (available online), 2017'; please provide a stable version identifier or a more formal citation, and likewise check the other arXiv preprints for version numbers.
  5. [General] The abstract and introduction use inconsistent capitalization ('Searing Spotlight' versus 'Searing Spotlights'); please standardize the game-name spelling.
  6. [Section 3.1] The comparison of the two density estimates in Figure 3 is visual only; please state the sample sizes, bin widths, and a quantitative similarity measure if the consistency of the two approaches is claimed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the manifold hypothesis is tested against an external null model, Theorem 1 is an independent construction, and the post-hoc volume-range choice is a validation concern rather than a circular reduction.

full rationale

The paper's central empirical claim is not circular. It imports an external manifold null model from Gray's asymptotic volume law, Equations (1)-(3), and tests whether the slopes fitted in Equation (5) cluster around a single integer. The stratified-space conclusion is inferred from the multimodality of the resulting histogram and from slope increases in individual VGT curves; neither of these outputs is fed back into the estimator as an input. The only post-hoc choice is the volume window [50,90], selected 'based on an inspection of the local dimension plots in Figure 5' as the range with 'the most linear behavior across tokens' (Section 3.1). This is a legitimate model-selection and validation concern, but it is not a constructional circularity: choosing a linear-looking window could bias individual slope estimates, yet it does not by itself force the four-cluster histogram or the trajectory spike locations. Theorem 1 is an independent existence result with an explicit stratified-space construction using chains of half-discs, and it is proved in the appendix rather than assumed. The cited methodology of Robinson et al. [14, 15] is external prior work, not self-citation, and no load-bearing argument reduces to an unverified self-citation. If the local-dimension estimates are unreliable, that is a correctness or robustness issue, not circularity. No step can be exhibited where a prediction is equivalent to its input by construction.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The central claim rests on the validity of the VGT local-dimension estimator and on the scale range chosen after inspecting the data. The mathematical realization theorem adds nothing to the empirical evidence and has an unproven construction. No new entities are introduced.

free parameters (4)
  • Volume range for local dimension estimation = [50, 90]
    Chosen after inspecting the same volume-growth curves to find the most linear range (Section 3.1). All reported dimensions use this range.
  • Radius range (alternative estimator) = [40, 60]
    Used for the left histogram of Figure 3; not used for the main analysis.
  • Density parameter M = unknown
    Introduced in Eq (4) as an unknown density and folded into the per-token intercept H_x in Eq (5). Not fitted, but affects the estimated dimension if density varies.
  • Per-token intercept log H_x = estimated by least squares for each token
    Fitted jointly with n_x in Eq (5); sensitive to range choice.
assumptions (6)
  • domain assumption Gray's small-ball volume expansion for Riemannian manifolds (Eq (1)) applies to the token embedding point cloud.
    Used to derive Eq (3) and the least-squares estimator; token embeddings are not known a priori to be Riemannian samples.
  • domain assumption The O(r^4) remainder and the Ricci term are negligible over the chosen range.
    Eq (5) drops these terms without diagnostics; essential for the linear fit to recover n_x.
  • domain assumption Token density is uniform enough that counting tokens approximates volume up to a constant M.
    Eq (4) assumes a constant density M; the paper gives no density variation analysis.
  • ad hoc to paper The glued half-disc construction in Appendix A defines a metric measure space whose VGT equals the stated piecewise linear f.
    The proof does not specify a metric or measure on the quotient and does not demonstrate the equality; the volume of a ball in the chain of half-discs is not obviously exactly piecewise linear in log radius.
  • domain assumption Slope increases after slope decreases in a VGT curve are inconsistent with fiber bundles and indicate a flare.
    Used to classify tokens 2434, 2833, 4040, and 4105 as lying on flares; no mathematical characterization is given for point clouds.
  • domain assumption The local dimension estimates and their clusters are stable under choices of neighborhood scale.
    The paper compares two ranges but does not vary the volume range systematically; cluster boundaries [6,8], [9,10], [11,13], [14,21] are read off by eye.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Exploring the Stratified Space Structure of an RL Game with the Volume Growth Transform." pith.science (2026). https://pith.science/paper/FYOHWGWM

@misc{pith2026250722010,
  author       = {Pith},
  title        = {Pith review of: Exploring the Stratified Space Structure of an RL Game with the Volume Growth Transform},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FYOHWGWM}},
  note         = {Machine review of arXiv:2507.22010}
}
read the original abstract

In this work, we explore the structure of the embedding space of a transformer model trained for playing a particular reinforcement learning (RL) game. Specifically, we investigate how a transformer-based Proximal Policy Optimization (PPO) model embeds visual inputs in a simple environment where an agent must collect "coins" while avoiding dynamic obstacles consisting of "spotlights." By adapting Robinson et al.'s study of the volume growth transform for LLMs to the RL setting, we find that the token embedding space for our visual coin collecting game is also not a manifold, and is better modeled as a stratified space, where local dimension can vary from point to point. We further strengthen Robinson's method by proving that fairly general volume growth curves can be realized by stratified spaces. Finally, we carry out an analysis that suggests that as an RL agent acts, its latent representation alternates between periods of low local dimension, while following a fixed sub-strategy, and bursts of high local dimension, where the agent achieves a sub-goal (e.g., collecting an object) or where the environmental complexity increases (e.g., more obstacles appear). Consequently, our work suggests that the distribution of dimensions in a stratified latent space may provide a new geometric indicator of complexity for RL games.

Figures

Figures reproduced from arXiv: 2507.22010 by the authors.

Figure 1
Figure 1. A cartoon example of the fiber bundle hypothesis and the corresponding piecewise [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. (Left): The ‘Two-Coin’ game. (Right): The network architecture used to train the agent. We modified the Searing Spotlights en￾vironment to have two coins in the room, but with different colors. This allows us to look for patterns in the local dimension curve and try to con￾nect signals there with repeatable ac￾tions during game play, e.g., coin col￾lection. We also increased visibility of the room, so that we could … view at source ↗
Figure 3
Figure 3. Local dimension probability density estimate when solving the least square prob [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: For each local dimension “cluster” we show 20 tokens (observations). Note that [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Volume vs radius for various tokens/observations. Note that for tokens 2434, 2833, [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Two simulation traces, (a) and (b), showing how the local dimension of the tokens [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: Illustration of the Transformer-XL-based PPO model implemented in [ [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: PPO-TransformerXL results on Searing Spotlights Environment. Success corre [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

19 extracted references · 15 canonical work pages

  1. [1]

    Local structures on stratified spaces

    David Ayala, John Francis, and Hiro Lee Tanaka. “Local structures on stratified spaces”. In:Advances in Mathematics307 (2017), pp. 903–1028

    work page 2017

  2. [2]

    Integral geometry of tame sets

    Ludwig Bröcker and Martin Kuppe. “Integral geometry of tame sets”. In:Geometriae Dedicata 82.1 (2000), pp. 285–323

    work page 2000

  3. [3]

    Zihang Dai et al.Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context. 2019. arXiv: 1901.02860 [cs.LG] . url: https://arxiv.org/abs/1901. 02860

    arXiv 2019

  4. [4]

    Singular intersection homology

    Greg Friedman. Singular intersection homology. Vol. 33. Cambridge University Press, 2020

    work page 2020

  5. [5]

    The volume of a small geodesic ball of a Riemannian manifold

    Alfred Gray. “The volume of a small geodesic ball of a Riemannian manifold.” In: Michigan Mathematical Journal20.4 (1974), pp. 329–344

    work page 1974

  6. [6]

    Differential topology

    Victor Guillemin and Alan Pollack. Differential topology. Vol. 370. American Mathe- matical Soc., 2010

    work page 2010

  7. [7]

    Local analytic dimensions of a subanalytic set

    Heisuke Hironaka. “Local analytic dimensions of a subanalytic set”. In:Proc. Japan Acad. Ser. A Math. Sci.62.6 (1986), pp. 73–75

    work page 1986

  8. [8]

    Topology of Word Em- beddings: Singularities Reflect Polysemy

    Alexander Jakubowski, Milica Gasic, and Marcus Zibrowius. “Topology of Word Em- beddings: Singularities Reflect Polysemy”. In:Proceedings of the Ninth Joint Confer- ence on Lexical and Computational Semantics. 2020, pp. 103–113

    work page 2020

Show all 19 references
  1. [9]

    Higher Algebra

    Jacob Lurie. Higher Algebra. Available at https : / / people . math . harvard . edu / ~lurie/paper/HA/HA.pdf. Not stated (available online), 2017

    2017

  2. [10]

    Whitney stratifications are conically smooth

    Guglielmo Nocera and Marco Volpe. “Whitney stratifications are conically smooth”. In: Selecta Mathematica29.5 (2023), p. 68

    2023

  3. [11]

    GitHub Repository for Memory Gym: Towards Endless Tasks to Benchmark Memory Capabilities of Agents

    Marco Pleines et al. “GitHub Repository for Memory Gym: Towards Endless Tasks to Benchmark Memory Capabilities of Agents”.https://github.com/MarcoMeter/ endless-memory-gym. 2024. 11

    2024

  4. [12]

    Memory Gym: Towards Endless Tasks to Benchmark Memory Capabilities of Agents

    Marco Pleines et al. “Memory Gym: Towards Endless Tasks to Benchmark Memory Capabilities of Agents”. In:Journal of Machine Learning Research26 (2025), pp. 1–40

    2025

  5. [13]

    The Intrinsic Dimension of Images and Its Impact on Learning

    Phil Pope et al. “The Intrinsic Dimension of Images and Its Impact on Learning”. In: International Conference on Learning Representations. 2021

    2021

  6. [14]

    Token embeddings violate the manifold hypothesis

    Michael Robinson, Sourya Dey, and Tony Chiang. “Token embeddings violate the manifold hypothesis”. In:arXiv preprint arXiv:2504.01002(2025)

    2025

  7. [15]

    The structure of the token space for large language models

    Michael Robinson, Sourya Dey, and Shauna Sweet. “The structure of the token space for large language models”. In:arXiv preprint arXiv:2410.08993(2024)

    2024 arXiv

  8. [16]

    Manifolds

    Loring W Tu. “Manifolds”. In:An Introduction to Manifolds. Springer, 2011, pp. 47– 83

    2011

  9. [17]

    Topological singularity detection at multi- ple scales

    Julius Von Rohrscheidt and Bastian Rieck. “Topological singularity detection at multi- ple scales”. In:Proceedings of the 40th International Conference on Machine Learning. ICML’23. Honolulu, Hawaii, USA: JMLR.org, 2023

    2023

  10. [18]

    On stratifications and poset-stratified spaces

    Lukas Waas, Jon Woolf, and Shoji Yokura. “On stratifications and poset-stratified spaces”. In:arXiv preprint arXiv:2407.17690(2024)

    2024 arXiv

  11. [19]

    Elementary Structure of Real Algebraic Varieties

    Hassler Whitney. “Elementary Structure of Real Algebraic Varieties”. In:Annals of Mathematics 66.3 (1957), pp. 545–556. A Manifolds, Fiber Bundles, and Stratified Spaces We provide a brief introduction to several classes of spaces that can be used to model the latent space of ...

    1957

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.