Pith. sign in

REVIEW 3 major objections 3 minor 48 references

An Analysis of Residual-Stream Geometry Across Transformer Depth

T0 review · 3 major / 3 minor · reviewed 2026-08-01 · deepseek-v4-flash

Pith's one-line read Across six language models, token representations move most in early and late layers while the overall cloud rotation stays nearly constant with depth.

desk verdict Good measurement-first paper on residual-stream depth curves, but the Procrustes-residual claim needs a scale-normalized control before it carries weight. read the letter →

arxiv 2607.18348 v1 pith:Q2MJXQ4G submitted 2026-07-20 cs.LG cs.AI

classification cs.LGcs.AI
keywords residualstreamtransformerinterpretabilityProcrustesanalysislayer-wisegeometrymultilingualrepresentationsdepthcodegenerationtranslation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a transformer's residual stream is best understood transition-by-transition: treating each layer change as a geometric transformation of the whole token cloud reveals stable depth regularities. In six instruction-tuned models, on code generation and English-to-four-language translation, representations move more in early and late layers, less in the middle third, and this depth schedule is nearly identical across tasks within each model. The size of the best-fit global rotation is almost flat across depth, while the non-rigid residual left after alignment varies strongly and peaks at the final transition; in generation, non-English targets show larger final-layer movement and residual than English. The authors carefully limit the claim to descriptive geometry, not computational effort or causality. A sympathetic reader cares because these regularities define a target that any future explanation of depth-related computation must reproduce.

What carries the argument

Orthogonal Procrustes decomposition of each layer transition is the central object. At each depth ℓ the token cloud is centered and the rotation R*_ℓ best mapping source to target is found; rotation magnitude Rℓ = ||R*_ℓ - I||_F quantifies the rigid component, the average Euclidean distance between aligned source and centered target tokens is the Procrustes residual, and Cℓ = ||D_ℓ||_F²/(q||D_ℓ||₂²) with D_ℓ = R*_ℓ - I measures how concentrated the rotation is across planes. Relative displacement δℓ(t) = ||h_{ℓ+1}(t)-h_ℓ(t)||/||h_ℓ(t)|| controls for norm growth. Together these separate 'how far, how rigid, how non-rigid, and how spread' for every transition.

What would settle it

Compute the same Procrustes residual after normalizing each layer's cloud to unit scale (or after subtracting a random-cloud baseline); if the final-transition peak and the English/non-English gap shrink to near zero, the reported depth modulation is an artifact of norm growth rather than non-rigid change. A second check: run the pipeline on a randomly initialized untrained model and compare the depth curves.

Watch

Extended reading notes

Core claim

The central claim is that layer-to-layer changes in the residual stream can be separated into a globally rigid rotation and a non-rigid residual, and that this separation exposes reproducible depth curves. Relative displacement δℓ(t) = ||h_{ℓ+1}-h_ℓ||/||h_ℓ|| is strongly layer-dependent, typically larger early and late with a quieter middle third, and nearly invariant across condition within a model. The rotation magnitude Rℓ = ||R*_ℓ - I||_F stays within a predeclared 3% peak-to-trough bound in every model and phase, while Procrustes residual and rotation-angle concentration remain depth-modulated, with residual peaking at the final transition in every model and phase. The paper presents th

Load-bearing premise

The load-bearing premise is that the Procrustes residual, computed as an absolute Euclidean distance between aligned source and centered target tokens without normalizing by per-layer cloud scale, reflects non-rigid geometry rather than the growth of residual-stream norms with depth in Pre-LN transformers.

Editorial extensions

If this is right

  • If the depth curves are as stable as reported, the early-middle-late rhythm is a property of the model family, not of the prompt, so layer-skipping or layer-looping methods can be tested against whether they preserve or disrupt this measured pattern.
  • Near-constant rotation magnitude combined with depth-varying residual and angle concentration implies that flat rotational scale cannot be used as evidence about depth structure; the informative signal is the dissociation.
  • During generation, the English/non-English separation in final-layer displacement and residual is consistent with English-centric processing accounts, but the paper frames it as an observational association restricted to Indo-European pairs.
  • The statistical tests (hierarchical bootstrap and layer permutation with Holm correction) support the robustness of the modulation, so the regularities are not flukes of a single condition.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension I would draw: compute the same metrics on a randomly initialized (untrained) model of the same architecture; if the early/late displacement curve and flat rotation persist, much of the pattern is a geometric baseline rather than a learned computational schedule.
  • The final-transition residual peak and the non-English gap may be inflated by the growth of residual-stream norms with depth, since the Procrustes residual is an absolute Euclidean distance; normalizing by per-layer cloud scale (or whitening each cloud before alignment) would separate non-rigid geometry from norm growth.
  • If the depth schedule is condition-stable but model-dependent, then model families might be characterized by distinct geometric signatures, which could serve as a cheap diagnostic for whether an intervention (fine-tuning, distillation, extra compute) changed the model's internal organization.
  • I read the middle-layer slowdown as aligning with prior observations that middle layers are more language-neutral; a direct test would be whether the slowdown vanishes for non-linguistic tasks or for in-distribution English-only data.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper proposes a transition-centered geometric analysis of transformer residual streams. For each layer transition it measures relative displacement (normalized by the source norm), curvature, the magnitude of the optimal orthogonal Procrustes rotation, the Procrustes residual after rotation-only alignment, and a proxy for rotation-angle concentration. These metrics are computed on six instruction-tuned Pre-LN models across code-generation and English-centric translation tasks. The paper reports four descriptive regularities: relative displacement is depth-structured with larger values early and late and a quieter middle third; rotation magnitude is nearly constant across depth; Procrustes residual and angle concentration remain depth-modulated, with residual peaking at the final transition; and during generation, non-English targets produce larger final-layer displacement and residual. The authors present these as descriptive geometric regularities, not causal or computational-effort claims, and support them with hierarchical bootstrap and layer-permutation tests with Holm correction.

Significance. If the reported regularities survive scrutiny, the framework is a useful descriptive addition to residual-stream analysis: it is a direct measurement with no fitted parameters, the statistical validation is explicit, and the authors are appropriately cautious about causal interpretation. The predeclared rotation-constancy threshold and the acknowledgement that rotation flatness may be a high-dimensional geometric baseline are strengths. However, the central residual-based regularities are compromised by the lack of scale normalization, as detailed in the major comments. The paper's most distinctive claim—that non-rigid residual structure is depth-modulated and peaks at the final transition—is not yet supported by the evidence as presented.

major comments (3)
  1. [Section 3.2, after Eq. (3); Figures 7–8; Table 1] The Procrustes residual is an absolute Euclidean distance between each aligned source token and its centered target, with no normalization by cloud scale. Section 3.1 explicitly states that residual-stream norms commonly grow with depth in Pre-LN models, and Eq. (1) deliberately normalizes by the source norm for exactly this reason. The unnormalized residual therefore conflates non-rigid geometric mismatch with the overall scale of the target cloud. The final-transition peak in every model and phase, and the larger non-English residual in Figure 8, are exactly what a pure norm-growth baseline would predict. The layer-permutation test in §3.3 tests whether absolute residual values vary across depth, not whether the depth structure survives scale normalization. Consequently, the abstract's third regularity—that Procrustes residual is depth-modulated and peaks at the final transition—is uns
  2. [Section 5.2 and Figure 3] The claim that the residual's maximizing transition is the final layer for every model in both phases is presented as a striking regularity. But this uniformity is the hallmark of monotone norm growth. Because no norm curves are provided, the depth-modulation test cannot distinguish 'non-rigid mismatch grows near the output' from 'vector norms grow near the output.' The authors already acknowledge in Section 5.2 that rotation magnitude is near a high-dimensional concentration scale and therefore do not interpret flatness alone as evidence of a learned invariant; the analogous caution is missing for the residual. A scale-matched baseline or a matched random-rotation baseline with the same norm profile is needed before the final-peak claim can stand.
  3. [Discussion, 'What the measurements show'] The paper's informative dissociation—flat rotation magnitude alongside depth-structured residual and angle concentration—is used to argue that the depth structure is not trivial. If the residual is scale-dominated, the dissociation reduces to 'rotational direction is stable while vector norms grow with depth,' which is a much weaker statement and not a geometric property of cloud shape. The limitations paragraph appropriately notes that the metrics are not validated as computational-effort measures, but it does not address the scale confound for the residual. The dissociation argument should be revisited after the scale-normalized reanalysis requested above, or the corresponding claims in the Abstract and §5.2 should be weakened.
minor comments (3)
  1. [Section 4.1, footnote 1] The footnote reads 'Implementation details here' and appears to be a placeholder. Supply the missing implementation details, including hook placement, dtype, and any preprocessing.
  2. [Equation (2)] The denominator is ||v||_2^2 while the numerator is a norm, so the curvature quantity is not scale-free and depends on the arbitrary scale of the residual stream. Since curvature is only descriptive, this is not blocking, but the definition should be clarified or the denominator changed to ||v||_2 for a standard scale-invariant turning measure.
  3. [Section 5.1, Figure 1] The claim that coding and translation 'share the same depth curve' is supported mainly by visual comparison across two separate panels. Consider overlaying the conditions or reporting a quantitative shape-similarity measure, since condition-invariance is part of the abstract's claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: depth curves are direct measurements; thresholds are predeclared and no fitted quantity is reported as a prediction.

full rationale

The paper's central claims are descriptive measurements of residual-stream activations: relative displacement (Eq. 1), Procrustes rotation magnitude (Eq. 3), Procrustes residual, and angle concentration are computed directly from recorded hidden states. There is no fitted parameter that is later called a prediction, and no subset of the data is used to estimate a quantity that is then claimed as an independent forecast. The 3% rotation-constancy bound is explicitly predeclared in Section 3.3 ('Before looking at the data, we defined rotation as practically constant when V < 0.03'), so it is a fixed criterion rather than a fitted threshold. The bootstrap and permutation tests compare observed depth curves with nulls generated from the same measurements; that is standard descriptive inference, not circularity. The only self-citation is Bhattacharya and Bojar (2023), used as contextual support for a middle-layer language-neutrality association; the load-bearing depth curves and residual/displacement findings do not reduce to that citation and would stand without it. The skeptic's concern that the Procrustes residual is not normalized by per-layer cloud scale is a validity and interpretation issue, not a derivation that reduces to its inputs: the final-transition peak may reflect residual-stream norm growth, but the paper does not define the residual in terms of the claimed conclusion. It even controls norm growth for relative displacement (Eq. 1) while omitting the same control for residual, which is a limitation, not a circular step. No circular step is therefore identified.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new physical or architectural entities. Its assumptions are mostly standard measurement conventions for residual-stream analysis; the least supported is the unnormalized residual interpretation. The only hand-chosen number is the 3% constancy threshold.

free parameters (1)
  • Rotation-constancy threshold = 0.03
    Hand-chosen practical bound for peak-to-trough variation V (Eq. 5). Determines the 'rotation magnitude is nearly constant' claim; not fitted to data, but arbitrary and not preregistered.
assumptions (5)
  • domain assumption Residual-stream activations captured by hooks after each layer's attention and MLP represent the layer's output state
    Section 4.3. If hooks fire at a different sublayer boundary or MPS float16 changes operation order, all depth curves shift.
  • domain assumption Centering token clouds before orthogonal Procrustes and using the unnormalized residual as non-rigid mismatch is meaningful
    Section 3.2. No per-layer norm normalization; absolute residual conflates cloud-scale growth with non-rigid shape change.
  • domain assumption Tokens within a prompt and prompts within conditions are exchangeable for the hierarchical bootstrap
    Section 3.3. Resampling conditions then prompts assumes a clean hierarchy; autocorrelation across autoregressive decoding steps of one prompt is not modeled.
  • domain assumption Greedy decoding and MPS float16 precision do not materially distort the geometric metrics
    Section 4.3. No precision or sampling ablation is provided.
  • standard math The high-dimensional Procrustes concentration scale sqrt(2d) is the relevant null baseline for interpreting rotation flatness
    Section 6. Used to temper the flatness claim, but no random-matrix benchmark experiment is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of An Analysis of Residual-Stream Geometry Across Transformer Depth." pith.science (2026). https://pith.science/paper/Q2MJXQ4G

@misc{pith2026260718348,
  author       = {Pith},
  title        = {Pith review of: An Analysis of Residual-Stream Geometry Across Transformer Depth},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Q2MJXQ4G}},
  note         = {Machine review of arXiv:2607.18348}
}
read the original abstract

We propose a transition-centred geometric analysis of transformer residual streams. Relative displacement measures how \emph{far} representations move between consecutive layers, and orthogonal Procrustes analysis separates each transition into a rigid rotation and a non-rigid residual. Across six instruction-tuned models, on code generation and cross-lingual translation, these measurements reveal reproducible depth regularities. Relative displacement is strongly layer-dependent; typically larger early and late, with a quieter middle third; and nearly invariant across conditions within each model. Rotation magnitude is nearly constant across depth, while Procrustes residual and angle concentration remain depth-modulated, with residual peaking at the final transition. During generation, non-English targets show larger final-layer displacement and residual than English targets. We present these as descriptive geometric regularities, not as measures of computational effort or causal explanations. The contribution is a measurement framework for residual-stream transitions and evidence that, in the settings studied here, depth curves are model-dependent and largely condition-stable.

Figures

Figures reproduced from arXiv: 2607.18348 by the authors.

Figure 2
Figure 2. Depth structure of relative displacement (top) [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 1
Figure 1. Relative displacement during prefill. Within [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 6
Figure 6. Rotation magnitude during translation pre [PITH_FULL_IMAGE:figures/full_fig_p006_6.png] view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: Relative displacement during translation [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 9
Figure 9. Figure 9: Rotation-angle concentration during trans [PITH_FULL_IMAGE:figures/full_fig_p007_9.png]
Figure 8
Figure 8. Figure 8: Procrustes residual during translation gen [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

48 extracted references · 1 canonical work pages

  1. [1]

    Advances in neural information processing systems , volume=

    Attention is all you need , author=. Advances in neural information processing systems , volume=

  2. [2]

    arXiv preprint arXiv:2501.16496 , year=

    Open problems in mechanistic interpretability , author=. arXiv preprint arXiv:2501.16496 , year=

  3. [3]

    AI and Ethics , volume=

    History, development, and principles of large language models: an introductory survey , author=. AI and Ethics , volume=. 2025 , publisher=

  4. [4]

    Frontiers of Computer Science , volume=

    A survey of large language models , author=. Frontiers of Computer Science , volume=. 2026 , publisher=

  5. [5]

    Transactions on Machine Learning Research , year=

    Mechanistic Interpretability for AI Safety-A Review , author=. Transactions on Machine Learning Research , year=

  6. [6]

    Advances in Neural Information Processing Systems , volume=

    Interpbench: Semi-synthetic transformers for evaluating mechanistic interpretability techniques , author=. Advances in Neural Information Processing Systems , volume=

  7. [7]

    arXiv preprint arXiv:2405.10928 , year=

    The local interaction basis: Identifying computationally-relevant and sparsely interacting features in neural networks , author=. arXiv preprint arXiv:2405.10928 , year=

  8. [8]

    Distill , volume=

    An overview of early vision in inceptionv1 , author=. Distill , volume=

Show all 48 references
  1. [9]

    Transformer Circuits Thread , volume=

    A mathematical framework for transformer circuits , author=. Transformer Circuits Thread , volume=

  2. [10]

    Distill , volume=

    Zoom in: An introduction to circuits , author=. Distill , volume=

  3. [11]

    International Conference on Learning Representations , volume=

    Not all language model features are one-dimensionally linear , author=. International Conference on Learning Representations , volume=

  4. [12]

    arXiv preprint arXiv:2509.17665 , year=

    Mechanistic Interpretability with SAEs: Probing Religion, Violence, and Geography in Large Language Models , author=. arXiv preprint arXiv:2509.17665 , year=

  5. [13]

    arXiv preprint arXiv:2303.08112 , year=

    Eliciting latent predictions from transformers with the tuned lens , author=. arXiv preprint arXiv:2303.08112 , year=

  6. [14]

    LessWrong , year =

    interpreting. LessWrong , year =

  7. [15]

    International conference on machine learning , pages=

    Similarity of neural network representations revisited , author=. International conference on machine learning , pages=. 2019 , organization=

  8. [16]

    Psychometrika , volume=

    A generalized solution of the orthogonal procrustes problem , author=. Psychometrika , volume=. 1966 , publisher=

  9. [17]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Do llamas work in english? on the latent language of multilingual transformers , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  10. [18]

    and Tu, Zhuowen and Bergen, Benjamin K

    Chang, Tyler A. and Tu, Zhuowen and Bergen, Benjamin K. The Geometry of Multilingual Language Model Representations. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 2022. doi:10.18653/v1/2022.emnlp-main.9

  11. [19]

    Unveiling Multilinguality in Transformer Models: Exploring Language Specificity in Feed-Forward Networks

    Bhattacharya, Sunit and Bojar, Ond r ej. Unveiling Multilinguality in Transformer Models: Exploring Language Specificity in Feed-Forward Networks. Proceedings of the 6th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP. 2023. doi:10.18653/v1/2023.blackb...

  12. [20]

    ICLR 2025 Workshop on Building Trust in Language Models and Applications , year=

    Do Multilingual LLMs Think In English? , author=. ICLR 2025 Workshop on Building Trust in Language Models and Applications , year=

  13. [21]

    arXiv preprint arXiv:2605.31220 , year=

    Shared Doubt: Zero-shot Cross-Lingual Confidence Estimation for Language Models , author=. arXiv preprint arXiv:2605.31220 , year=

  14. [22]

    The Eleventh International Conference on Learning Representations , year=

    Progress measures for grokking via mechanistic interpretability , author=. The Eleventh International Conference on Learning Representations , year=

  15. [23]

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages=

    A mechanistic interpretation of arithmetic reasoning in language models using causal mediation analysis , author=. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages=

  16. [24]

    arXiv preprint arXiv:2510.09782 , year=

    The geometry of reasoning: Flowing logics in representation space , author=. arXiv preprint arXiv:2510.09782 , year=

  17. [25]

    Annual review of condensed matter physics , volume=

    Statistical mechanics of deep learning , author=. Annual review of condensed matter physics , volume=. 2020 , publisher=

  18. [26]

    Scientific Reports , year=

    Unraveling the geometry of visual relational reasoning , author=. Scientific Reports , year=

  19. [27]

    arXiv preprint arXiv:2501.10573 , year=

    The geometry of tokens in internal representations of large language models , author=. arXiv preprint arXiv:2501.10573 , year=

  20. [28]

    International conference on machine learning , pages=

    On layer normalization in the transformer architecture , author=. International conference on machine learning , pages=. 2020 , organization=

  21. [29]

    , author=

    Large language models implicitly learn to straighten neural sentence trajectories to construct a predictive representation of natural language. , author=. Advances in Neural Information Processing Systems , volume=

  22. [30]

    arXiv preprint arXiv:2502.12131 , year=

    Transformer Dynamics: A neuroscientific approach to interpretability of large language models , author=. arXiv preprint arXiv:2502.12131 , year=

  23. [31]

    arXiv preprint arXiv:2603.01326 , year=

    Truth as a trajectory: What internal representations reveal about large language model reasoning , author=. arXiv preprint arXiv:2603.01326 , year=

  24. [32]

    arXiv preprint arXiv:2601.22364 , year=

    Context Structure Reshapes the Representational Geometry of Language Models , author=. arXiv preprint arXiv:2601.22364 , year=

  25. [33]

    Mathqa: Towards interpretable math word problem solving with operation-based formalisms , author=. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) , pages=

  26. [34]

    E uroparl: A Parallel Corpus for Statistical Machine Translation

    Koehn, Philipp. E uroparl: A Parallel Corpus for Statistical Machine Translation. Proceedings of Machine Translation Summit X: Papers. 2005

  27. [35]

    Acta Crystallographica Section A: Crystal Physics, Diffraction, Theoretical and General Crystallography , volume=

    A solution for the best rotation to relate two sets of vectors , author=. Acta Crystallographica Section A: Crystal Physics, Diffraction, Theoretical and General Crystallography , volume=. 1976 , publisher=

  28. [36]

    IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=

    Least-squares estimation of transformation parameters between two point patterns , author=. IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=. 1991 , publisher=

  29. [37]

    Journal of Mathematical Imaging and Vision , volume=

    Metrics for 3D rotations: Comparison and analysis , author=. Journal of Mathematical Imaging and Vision , volume=. 2009 , publisher=

  30. [38]

    1998 , publisher=

    Statistical shape analysis , author=. 1998 , publisher=

  31. [39]

    2001 , publisher=

    Geometric methods and applications: for computer science and engineering , author=. 2001 , publisher=

  32. [40]

    15th European Signal Processing Conference , pages=

    The effective rank: A measure of effective dimensionality , author=. 15th European Signal Processing Conference , pages=. 2007 , organization=

  33. [41]

    International Conference on Learning Representations , volume=

    Livecodebench: Holistic and contamination free evaluation of large language models for code , author=. International Conference on Learning Representations , volume=

  34. [42]

    How Contextual are Contextualized Word Representations? C omparing the Geometry of BERT , ELM o, and GPT -2 Embeddings

    Ethayarajh, Kawin. How Contextual are Contextualized Word Representations? C omparing the Geometry of BERT , ELM o, and GPT -2 Embeddings. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natur...

  35. [43]

    arXiv preprint arXiv:1807.03819 , year=

    Universal transformers , author=. arXiv preprint arXiv:1807.03819 , year=

  36. [44]

    Advances in Neural Information Processing Systems , volume=

    Do language models use their depth efficiently? , author=. Advances in Neural Information Processing Systems , volume=

  37. [45]

    International Conference on Learning Representations , volume=

    Reasoning with latent thoughts: On the power of looped transformers , author=. International Conference on Learning Representations , volume=

  38. [46]

    Proceedings of the 58th annual meeting of the association for computational linguistics , pages=

    Emerging cross-lingual structure in pretrained language models , author=. Proceedings of the 58th annual meeting of the association for computational linguistics , pages=

  39. [47]

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages=

    Mega: Multilingual evaluation of generative ai , author=. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages=

  40. [48]

    Advances in neural information processing systems , volume=

    Chain-of-thought prompting elicits reasoning in large language models , author=. Advances in neural information processing systems , volume=

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.