Manifold steering along activation geometry induces behavioral trajectories matching the natural manifold of outputs, while linear steering produces off-manifold unnatural behaviors.
Dynamics of transient structure in in-context linear regression transformers
6 Pith papers cite this work. Polarity classification is still indexing.
6
Pith papers citing it
citation-role summary
background 1
citation-polarity summary
years
2026 6roles
background 1polarities
background 1representative citing papers
In a controlled synthetic setting, transformers implement in-distribution task inference via convex combinations of task vectors and out-of-distribution inference via nearly orthogonal extrapolative representations.
Temporal diversity in task distribution during training increases generalization bias over memorization in transformers for in-context linear regression.
Small transformers on HMM prediction tasks exhibit correlated scaling between performance and linear encoding of belief distributions in residual activations.
This review synthesizes representative advances in high-dimensional statistics, highlights common themes and open problems, and points to key entry works.
citing papers explorer
-
Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behavior
Manifold steering along activation geometry induces behavioral trajectories matching the natural manifold of outputs, while linear steering produces off-manifold unnatural behaviors.
-
Task Vector Geometry Underlies Dual Modes of Task Inference in Transformers
In a controlled synthetic setting, transformers implement in-distribution task inference via convex combinations of task vectors and out-of-distribution inference via nearly orthogonal extrapolative representations.
-
Temporal Task Diversity: Inductive Biases Under Non-Stationarity in Synthetic Sequence Modelling
Temporal diversity in task distribution during training increases generalization bias over memorization in transformers for in-context linear regression.
-
Structure and Scale in Simplicial Sequence Modelling
Small transformers on HMM prediction tasks exhibit correlated scaling between performance and linear encoding of belief distributions in residual activations.
-
High-Dimensional Statistics: Reflections on Progress and Open Problems
This review synthesizes representative advances in high-dimensional statistics, highlights common themes and open problems, and points to key entry works.
- Mechanistic Anomaly Detection via Functional Attribution