arxiv: 2602.20338 · v2 · submitted 2026-02-23 · 💻 cs.LG

Recognition: 2 theorem links

· Lean Theorem

Emergent Manifold Separability during Reasoning in Large Language Models

Chanwoo Chun , Alexandre Polo , SueYeon Chung

Authors on Pith no claims yet

Pith reviewed 2026-05-15 20:10 UTC · model grok-4.3

classification 💻 cs.LG

keywords chain-of-thought reasoningmanifold capacity theorylinear separabilityrepresentation geometrylarge language modelsdynamic manifold managementcompositional tasksresidual stream

0 comments

The pith

Reasoning in large language models produces a transient geometric pulse that untangles concept manifolds into linearly separable subspaces immediately before each computational step and then rapidly compresses them.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper applies Manifold Capacity Theory to track how latent representations evolve during chain-of-thought reasoning on both a controlled Boolean logic task and a natural-language eligibility task. Across multiple open-weight models, the authors find that manifolds of relevant concepts become linearly separable right before the model performs a reasoning operation and are then compressed afterward. This pattern is distinct from linear probe accuracy, which stays elevated long after the computation has occurred. The work frames this as Dynamic Manifold Management, in which the model modulates representational capacity to manage bandwidth in the residual stream throughout the reasoning chain. The central claim is that this geometric pulse is a signature of active computation rather than passive information storage.

Core claim

On two compositional reasoning tasks and across several open-weight models, the latent representations exhibit a transient geometric pulse: concept manifolds are untangled into linearly separable subspaces immediately prior to each computational step and rapidly compressed thereafter. This behavior diverges from standard linear probe accuracy, which remains high long after computation, indicating a distinction between information that is merely retrievable and information that is geometrically prepared for processing. The authors interpret the pattern as Dynamic Manifold Management, a mechanism in which the model dynamically modulates representational capacity to optimize residual-streamband

What carries the argument

Manifold Capacity Theory (MCT), which quantifies the linear separability of latent representations without requiring trained probes.

If this is right

The model uses a temporary increase in representational capacity to prepare information for processing at precise moments in the chain.
Geometric compression after each step helps maintain bandwidth efficiency in the residual stream across multiple reasoning operations.
Information remains linearly decodable after compression, yet the model no longer treats it as immediately processable.
This dynamic occurs consistently on both synthetic Boolean trees and natural-language eligibility tasks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Similar pulses might appear in other sequential decision tasks such as planning or multi-step arithmetic.
Architectural modifications that strengthen or weaken this pulse could be tested as a way to improve or degrade reasoning performance.
The distinction between retrievable and geometrically prepared representations may help explain why some prompts succeed or fail at eliciting correct chains of thought.

Load-bearing premise

The observed changes in manifold geometry are caused by the specific computational demands of reasoning rather than by token generation, layer normalization, or other general architectural operations.

What would settle it

If the same transient pulse of linear separability appears at comparable rates during non-reasoning token generation tasks or random sequences that lack compositional structure, the claim that the pulse is tied to reasoning steps would be falsified.

Figures

Figures reproduced from arXiv: 2602.20338 by Alexandre Polo, Chanwoo Chun, SueYeon Chung.

**Figure 1.** Figure 1: Manifold Untangling during Chain-of-Thought. Before any token is generated (left) the latent representations of tasks corresponding to answers A and B are entangled and hard to distinguish resulting in low capacity. As the model generates the CoT, it progressively untangles these representations. By the final token (right), the two manifolds are linearly separable and capacity is high. 2 Related Work 2.1 … view at source ↗

**Figure 2.** Figure 2: a. Example of a Boolean logic tree with height h = 2. Left: The corresponding text input provided to the model, where internal nodes are labeled [01] through [03]. Right: Schematic of the tree structure. b. Accuracy on Boolean logic trees of varying height. Performance with CoT (purple) remains near-perfect (> 98%), while standard prompting (No CoT, orange) degrades significantly as tree depth increases (6… view at source ↗

**Figure 3.** Figure 3: Dynamic Modulation of Manifold Geometry during CoT. a. Manifold capacity (α) tracked across the Chain-of-Thought sequence (layer 20, tree height h = 4, see Supplementary for height 5). Lines are colored by node ID. Capacity for a specific node peaks sharply at two distinct moments: first when the node is intrinsically computed, and second when it is processed by its parent. b. Detailed analysis of Node 11 … view at source ↗

**Figure 4.** Figure 4: Dynamics of separability metrics during Solve and Recall events. Data is aligned to the moment a node is computed (Left column: a, c) or when it is recalled by its parent (Right column: b, d). (a, b) Manifold Capacity: Shows a sharp, transient peak at the moment of computation and recall, dropping quickly to baseline in between. (c, d) Hard-margin SVM probe test accuracy: While accuracy also peaks at compu… view at source ↗

**Figure 5.** Figure 5: presents the capacity change averaged over all nodes along the reasoning trace, aligned with respect to the moment where the node is solved. We identify two distinct phases: 1. Reasoning Phase: At the header and logic structural tokens of a specific node, the increase in separability originates in the middle layers (Layers 13-15). Then at the result structural token (immediately before the answer is spelle… view at source ↗

**Figure 6.** Figure 6: Correlation between Attention and Manifold Capacity. a. Attention score heatmap between graph nodes. The x-axis represents the target node currently being solved; the y-axis represents the source nodes in the context. The diagonal structure reflects the model attending to recent context, while off-diagonal points represent the retrieval of specific dependencies. b. Scatter plot of Attention Score versus Ma… view at source ↗

read the original abstract

Chain-of-Thought (CoT) prompting significantly improves reasoning in Large Language Models, yet the temporal dynamics of the underlying representation geometry remain poorly understood. We investigate these dynamics by applying Manifold Capacity Theory (MCT) to two compositional reasoning tasks: a controlled Boolean logic tree that supports deep mechanistic analysis, and a natural-language eligibility task in which the model has to extract attributes from prose, compare them to thresholds, and compose the local decisions through a fixed evaluation tree. MCT lets us quantify the linear separability of latent representations without the confounding factors of probe training. On both tasks, and across several open-weight models, reasoning manifests as a transient geometric pulse: concept manifolds are untangled into linearly separable subspaces immediately prior to computation and rapidly compressed thereafter. This behavior diverges from standard linear probe accuracy, which remains high long after computation, suggesting a fundamental distinction between information that is merely retrievable and information that is geometrically prepared for processing. We interpret this phenomenon as Dynamic Manifold Management, a mechanism where the model dynamically modulates representational capacity to optimize the bandwidth of the residual stream throughout the reasoning chain.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper observes a transient pulse of manifold separability timed to reasoning steps that diverges from probe accuracy, but the evidence is thin without controls for generation artifacts.

read the letter

The core observation is a short-lived rise in linear separability of concept manifolds right before the computation steps in chain-of-thought, followed by rapid compression. This shows up on the Boolean logic tree and the eligibility task, and it parts ways with linear probe accuracy that stays elevated afterward. They measure this with Manifold Capacity Theory across a few open models, which avoids the usual probe-training confounds and gives a direct geometric read on how representations get prepared for processing. That timing contrast is the main new piece, and applying MCT to compositional reasoning tasks is a reasonable move. The paper does a clean job laying out the two tasks and the basic pattern. The soft spot is exactly the one in the stress-test note. Nothing in the abstract points to matched non-reasoning generation sequences, position-controlled baselines, or layer-norm ablations, so the pulse could easily be a generic feature of how residual streams evolve during any longer autoregressive run rather than evidence for reasoning-specific dynamic management. Without those checks the interpretive label stays loose. The abstract also gives no numbers, error bars, or statistical tests, which makes it hard to judge how reliable the pulse is. This is for readers already working on geometric or mechanistic accounts of transformer reasoning. Someone following MCT or CoT interpretability work would find the observation worth seeing, but only after the methods and controls are filled in. I would send it to peer review if the full paper supplies the missing baselines and quantitative detail, because the geometric timing idea is worth testing even if the current write-up is preliminary.

Referee Report

2 major / 1 minor

Summary. The paper claims that Chain-of-Thought reasoning in LLMs produces a transient geometric pulse, measured via Manifold Capacity Theory (MCT), in which concept manifolds become linearly separable immediately prior to computation and are then rapidly compressed; this trajectory is observed on a Boolean logic tree and a natural-language eligibility task across open-weight models and diverges from the sustained high accuracy of linear probes, which the authors interpret as evidence for Dynamic Manifold Management that optimizes residual-stream bandwidth.

Significance. If the reported pulse is shown to be reasoning-specific rather than a generic generation artifact, the work would supply a geometric account of why CoT improves performance and would distinguish information that is merely retrievable from information that is geometrically prepared for processing. The use of MCT to obtain a probe-independent separability metric is a methodological strength that avoids training-related confounds.

major comments (2)

[Experimental setup and results] The central claim that the separability pulse is specifically tied to the computational steps of reasoning (rather than autoregressive token generation or layer-wise updates) requires matched-length non-reasoning control sequences and position-matched baselines; the experimental description supplies no indication of such controls, leaving open the possibility that the observed MCT trajectory is a generic property of any sufficiently long generation sequence.
[Discussion and interpretation] The interpretive label 'Dynamic Manifold Management' is not derived from fitted MCT parameters or self-referential equations; the manuscript must clarify how the pulse is quantitatively distinguished from alternative explanations such as residual-stream normalization or token-position effects before the mechanism can be treated as a distinct phenomenon.

minor comments (1)

[Abstract and results] The abstract states that the behavior 'diverges from standard linear probe accuracy' but supplies no quantitative comparison (timing offset, magnitude difference, or statistical test); this detail should be added to the results section with error bars.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments, which highlight important controls and interpretive clarifications needed to strengthen the claims. We address each point below and will revise the manuscript to incorporate the suggested experiments and distinctions.

read point-by-point responses

Referee: [Experimental setup and results] The central claim that the separability pulse is specifically tied to the computational steps of reasoning (rather than autoregressive token generation or layer-wise updates) requires matched-length non-reasoning control sequences and position-matched baselines; the experimental description supplies no indication of such controls, leaving open the possibility that the observed MCT trajectory is a generic property of any sufficiently long generation sequence.

Authors: We agree that matched controls are essential to establish specificity. While the original experiments align the pulse timing with explicit computational steps in the Boolean logic tree (e.g., at layers where AND/OR operations occur) and show divergence from sustained probe accuracy, we did not include explicit non-reasoning controls of matched length. In the revision we will add: (i) matched-length non-reasoning sequences such as repetitive token generation and simple copying tasks, and (ii) position-matched baselines by extracting MCT metrics at equivalent token positions in non-reasoning prompts. These controls will be reported alongside the existing results to isolate reasoning-specific effects. revision: yes
Referee: [Discussion and interpretation] The interpretive label 'Dynamic Manifold Management' is not derived from fitted MCT parameters or self-referential equations; the manuscript must clarify how the pulse is quantitatively distinguished from alternative explanations such as residual-stream normalization or token-position effects before the mechanism can be treated as a distinct phenomenon.

Authors: We acknowledge that 'Dynamic Manifold Management' is a descriptive label rather than a formally derived model. In the revised discussion we will: (i) report MCT separability metrics computed before and after explicit residual-stream normalization to rule out normalization artifacts, (ii) include position-controlled analyses by shuffling token order or comparing across fixed positions in sequences of varying length, and (iii) demonstrate that pulse onset aligns with the timing of specific computational operations in the reasoning tree (rather than uniform layer-wise updates). These quantitative comparisons will be added to distinguish the phenomenon from the listed alternatives. revision: yes

Circularity Check

0 steps flagged

No significant circularity; central claim is empirical measurement

full rationale

The paper applies Manifold Capacity Theory (MCT) as an external tool to quantify linear separability of latent representations on two reasoning tasks, reporting an observed transient pulse in manifold geometry as a direct empirical result across models. The interpretive label 'Dynamic Manifold Management' is introduced post-hoc to describe the measured behavior and does not appear as a fitted parameter, self-referential equation, or quantity derived by construction from the inputs. No load-bearing steps in the provided abstract or described chain reduce to self-citation, ansatz smuggling, or renaming of known results; the derivation remains self-contained as an observation rather than a tautological prediction.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

The central claim rests on the applicability of Manifold Capacity Theory to LLM activations and on the interpretation that the observed pulse constitutes a functional management mechanism.

axioms (1)

domain assumption Manifold Capacity Theory quantifies linear separability of latent representations without confounding effects from probe training
This is the methodological foundation stated in the abstract for measuring the pulse.

invented entities (1)

Dynamic Manifold Management no independent evidence
purpose: Interpretive mechanism explaining the transient untangling and compression of manifolds during reasoning
Proposed label for the observed phenomenon; no independent falsifiable prediction or external evidence is supplied in the abstract.

pith-pipeline@v0.9.0 · 5491 in / 1312 out tokens · 32050 ms · 2026-05-15T20:10:06.577156+00:00 · methodology

discussion (0)

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

IndisputableMonolith/Foundation/AbsoluteFloorClosure.lean absolute_floor_iff_bare_distinguishability echoes

?

echoes
ECHOES: this paper passage has the same mathematical shape or conceptual pattern as the Recognition theorem, but is not a direct formal dependency.

reasoning manifests as a transient geometric pulse: concept manifolds are untangled into linearly separable subspaces immediately prior to computation and rapidly compressed thereafter
IndisputableMonolith/Cost/FunctionalEquation.lean washburn_uniqueness_aczel unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

Manifold Capacity Theory (MCT) ... quantifies the linear separability of latent representations without the confounding factors of probe training

What do these tags mean?

matches: The paper's claim is directly supported by a theorem in the formal canon.
supports: The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends: The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses: The paper appears to rely on the theorem as machinery.
contradicts: The paper's claim conflicts with a theorem or certificate in the canon.
unclear: Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

20 extracted references · 20 canonical work pages · 9 internal anchors

[1]

Knowing before saying: Llm representations encode information about chain-of-thought success before completion.arXiv preprint arXiv:2505.24362,

Anum Afzal, Florian Matthes, Gal Chechik, and Yftah Ziser. Knowing before saying: Llm representations encode information about chain-of-thought success before completion.arXiv preprint arXiv:2505.24362,

work page arXiv
[2]

The internal state of an llm knows when it’s lying.arXiv preprint arXiv:2304.13734,

Amos Azaria and Tom Mitchell. The internal state of an llm knows when it’s lying.arXiv preprint arXiv:2304.13734,

work page arXiv
[3]

Discovering Latent Knowledge in Language Models Without Supervision

Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt. Discovering latent knowledge in language models without supervision.arXiv preprint arXiv:2212.03827,

work page internal anchor Pith review arXiv
[4]

Unveiling the key factors for distilling chain-of-thought reasoning

Xinghao Chen, Zhijing Sun, Guo Wenjin, Miaoran Zhang, Yanjun Chen, Yirong Sun, Hui Su, Yijie Pan, Dietrich Klakow, Wenjie Li, and Xiaoyu Shen. Unveiling the key factors for distilling chain-of-thought reasoning. InFindings of the Association for Computational Linguistics: ACL 2025, pages 15094–15119. Association for Computational Linguistics, July

work page 2025
[5]

Chi-Ning Chou, Royoung Kim, Luke A Arend, Yao-Yuan Yang, Brett D Mensh, Won Mok Shim, Matthew G Perich, and SueYeon Chung

doi: 10.18653/v1/2025.findings-acl.782. Chi-Ning Chou, Royoung Kim, Luke A Arend, Yao-Yuan Yang, Brett D Mensh, Won Mok Shim, Matthew G Perich, and SueYeon Chung. Geometry linked to untangling efficiency reveals structure and computation in neural populations. bioRxiv, pages 2024–02,

work page doi:10.18653/v1/2025.findings-acl.782 2025
[6]

Toy Models of Superposition

Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, Roger Grosse, Sam McCandlish, Jared Kaplan, Dario Amodei, Martin Wattenberg, and Christopher Olah. Toy models of superposition.arXiv preprint arXiv:2209.10652,

work page internal anchor Pith review Pith/arXiv arXiv
[7]

Language models represent space and time.arXiv preprint arXiv:2310.02207,

Wes Gurnee and Max Tegmark. Language models represent space and time.arXiv preprint arXiv:2310.02207,

work page arXiv
[8]

Designing and interpreting probes with control tasks.arXiv preprint arXiv:1909.03368,

John Hewitt and Percy Liang. Designing and interpreting probes with control tasks.arXiv preprint arXiv:1909.03368,

work page arXiv 1909
[9]

John Hewitt and Christopher D. Manning. A structural probe for finding syntax in word representations. InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, V olume 1 (Long and Short Papers), pages 4129–4138. Association for Computational Linguistics, June

work page 2019
[10]

The geometry of prompting: Unveiling distinct mechanisms of task adaptation in language models.arXiv preprint arXiv:2502.08009,

Artem Kirsanov, Chi-Ning Chou, Kyunghyun Cho, and SueYeon Chung. The geometry of prompting: Unveiling distinct mechanisms of task adaptation in language models.arXiv preprint arXiv:2502.08009,

work page arXiv
[11]

Measuring Faithfulness in Chain-of-Thought Reasoning

Tamera Lanham, Anna Chen, Ansh Radhakrishnan, Benoit Steiner, Carson Denison, Danny Hernandez, Dustin Li, Esin Durmus, Evan Hubinger, Jackson Kernion, Kamil˙e Lukoši¯ut˙e, Karina Nguyen, Newton Cheng, Nicholas Joseph, Nicholas Schiefer, Oliver Rausch, Robin Larson, Sam McCandlish, Sandipan Kundu, Saurav Kadavath, Shannon Yang, Thomas Henighan, Timothy Max...

work page internal anchor Pith review Pith/arXiv arXiv
[12]

Ministral 3

Alexander H. Liu, Kartik Khandelwal, Sandeep Subramanian, Victor Jouaul, et al. Ministral 3.arXiv preprint arXiv:2601.08584,

work page internal anchor Pith review Pith/arXiv arXiv
[13]

The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets

Samuel Marks and Max Tegmark. The geometry of truth: Emergent linear structure in large language model representa- tions of true/false datasets.arXiv preprint arXiv:2310.06824,

work page internal anchor Pith review Pith/arXiv arXiv
[14]

In-context Learning and Induction Heads

Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Scott Johnston, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, a...

work page internal anchor Pith review Pith/arXiv arXiv
[15]

The intrinsic dimension of images and its impact on learning.arXiv preprint arXiv:2104.08894,

Phillip Pope, Chen Zhu, Ahmed Abdelkader, Micah Goldblum, and Tom Goldstein. The intrinsic dimension of images and its impact on learning.arXiv preprint arXiv:2104.08894,

work page arXiv
[16]

On the geometry of generalization and memorization in deep neural networks.arXiv preprint arXiv:2105.14602,

Cory Stephenson, Suchismita Padhy, Abhinav Ganesh, Yue Hui, Hanlin Tang, and SueYeon Chung. On the geometry of generalization and memorization in deep neural networks.arXiv preprint arXiv:2105.14602,

work page arXiv
[17]

Miles Turpin, Julian Michael, Ethan Perez, and Samuel R. Bowman. Language models don’t always say what they think: Unfaithful explanations in chain-of-thought prompting.arXiv preprint arXiv:2305.04388,

work page internal anchor Pith review Pith/arXiv arXiv
[18]

No Training Required: Exploring Random Encoders for Sentence Classification

John Wieting and Douwe Kiela. No training required: Exploring random encoders for sentence classification.arXiv preprint arXiv:1901.10444,

work page internal anchor Pith review Pith/arXiv arXiv 1901
[19]

Reasoning models know when they’re right: Probing hidden states for self-verification.arXiv preprint arXiv:2504.05419,

Anqi Zhang, Yulin Chen, Jane Pan, Chen Zhao, Aurojit Panda, Jinyang Li, and He He. Reasoning models know when they’re right: Probing hidden states for self-verification.arXiv preprint arXiv:2504.05419,

work page arXiv
[20]

Language Modeling Teaches You More Syntax than Translation Does: Lessons Learned Through Auxiliary Task Analysis

12 PREPRINT. UNDERREVIEW. Kelly W Zhang and Samuel R Bowman. Language modeling teaches you more syntax than translation does: Lessons learned through auxiliary task analysis.arXiv preprint arXiv:1809.10040,

work page internal anchor Pith review Pith/arXiv arXiv