Pith. sign in

REVIEW 3 major objections 4 minor 27 references

Spectral Principal Paths: A Spectral Perspective on Linear Representation Formation in LLMs

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that the linear concept directions observed in LLM hidden states are inherited from a concept-aligned direction already present in token embeddings, then preserved across layers by flowing along a spectral principal path…

desk verdict A genuinely new spectral framing of linear representations, but the main stability theorem is false as stated and the empirical support is thin. read the letter →

arxiv 2506.08543 v4 pith:M475D7EF submitted 2025-06-10 cs.CV

Bowei Tian , Xuntao Lyu , Meng Liu , Hongyi Wang , Ang Li This is my paper · ORCID
classification cs.CV
keywords linearrepresentationhypothesisspectralprincipalpathsingularvaluedecompositioninterpretabilitylargelanguagemodelsinput-spacelinearitycontextincoherenceWedinsinThetatheorem
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that the linear representations observed in LLM hidden states are not created by the network but inherited: concept-aligned directions are already present in token embeddings, and the transformer stack preserves them along a spectral principal path. It formalizes this as the Input-Space Linearity Hypothesis and proves, via Wedin's sinΘ theorem, that the leading singular direction of a concept-difference matrix stays nearly fixed across layers when a spectral gap and context incoherence hold. From this it derives the Linear Representation Hypothesis: the deep representation difference is proportional to the product of leading singular values times one dominant direction, so concepts become linearly separable. The authors test the prerequisite conditions on four decoder-only LLMs and show that breaking context incoherence degrades the spectral gap and cross-layer stability, as predicted.

What carries the argument

The central object is the Spectral Principal Path (SPP): given $M$ paired sequences differing only in concept tokens, form the difference matrix $D^{(l)}$ whose rows are last-token hidden-state differences for each pair; the SPP is the leading right singular vector $v_1^{(l)}$ of $D^{(l)}$. The argument is carried by the Wedin $\sin\Theta$ perturbation theorem, which bounds the rotation of this leading singular vector when $D^{(l+1)}$ is decomposed into a mean-Jacobian-propagated signal $D^{(l)}\bar{J}^{(l)\top}$ plus context fluctuation and a higher-order remainder. The stability theorem requires a spectral gap ($\sigma_1^{(l)} \gg \sigma_2^{(l)}$) and context incoherence (the empirical second-moment matrix of per-sample fluctuations has small operator norm $\eta^{(l)}$). Together these make the network behave like a multi-stage power iteration, so non-principal spectral paths decay exponentially and the concept signal converges to a single dominant direction.

What would settle it

Estimate the input-space concept direction $v_{\mathrm{in}}$ directly from token embeddings before any transformer layer, then measure its alignment with the SPP direction $v_1^{(l)}$ at several intermediate layers; if the alignment is low, or if a model trained on embeddings with randomized concept directions still develops stable linear concept representations at depth, the claim that input-space linearity is necessary for the Linear Representation Hypothesis is contradicted.

Watch

Extended reading notes

Core claim

The central claim is that the Linear Representation Hypothesis is not an emergent learning outcome but a spectral inheritance: under the Input-Space Linearity Hypothesis, each token's embedding is a concept-agnostic base plus a concept-aligned term $c\cdot \alpha_i v_{\mathrm{in}}$ plus isotropic noise, so a shared concept direction $v_{\mathrm{in}}$ exists in the input space. The paper proves that the first attention layer aggregates this distributed signal into a concentrated direction at the last token, and then proves that the leading right singular vector of the difference matrix $D^{(l)}$ rotates by at most $\sigma_1^{(l)}(\delta^{(l)}+\sqrt{\eta^{(l)}}+\sqrt{\zeta^{(l)}})/\gamma^{(l+1)}$ between layers, where $\gamma$ is the spectral gap and $\eta, \zeta$ measure context fluctuation and higher-order remainder. Theorem 4.8 then concludes that the deep representation satisfies the Linear Representation Hypothesis: $\Delta h^{(L)}$ is proportional to the product of the leading singular values times the leading left singular vector $u_1^{(L-1)}$, so the semantic difference collapses into a one-dimensional subspace. In short, the paper claims that concept directions in LLMs are inherited from input-space linear structure and propagated through dominant spectral directions, with stability guaranteed by spectral gap and context incoherence.

Load-bearing premise

The load-bearing premise is the Input-Space Linearity Hypothesis: every token embedding is exactly a concept-agnostic base plus a per-token strength times a single shared concept direction, plus Gaussian noise; if input embeddings are not linearly organized this way, the SPP mechanism and the derivation of the Linear Representation Hypothesis do not get off the ground.

Editorial extensions

If this is right

  • If the Input-Space Linearity Hypothesis holds, linear probes and steering vectors work because the network preserves input-space directions rather than inventing them, so interventions at the embedding level should transfer predictably to deep-layer behavior.
  • The per-layer bound implies a worst-case cumulative angular drift that grows linearly in depth, but a random-walk estimate that grows only as $\sqrt{L}$; the paper argues that the empirically observed stability is consistent with the random-walk regime.
  • Both prerequisite conditions are measurable per layer (the spectral-gap ratio $\sigma_1/\sigma_2$ and the operator norm of the fluctuation second-moment matrix), giving practitioners a concrete way to predict whether a given model and concept should exhibit stable linear representations.
  • Because the deep representation difference becomes proportional to the product of leading singular values times $u_1^{(L-1)}$, concept directions at depth are predicted to remain aligned with the SPP across layers and to scale with the cumulative spectral gain.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension: treating the SPP direction as a causal handle, ablating or amplifying the leading singular direction of $D^{(l)}$ at an intermediate layer should shift downstream concept behavior, and the theorem's bound gives a quantitative prediction for how much drift to expect.
  • The framework implies that concepts not linearly organized in the input embedding space, such as compositional or culturally situated concepts, may fail to form stable spectral principal paths, which would explain why some concepts are more linearly probeable than others.
  • The same spectral argument may transfer beyond decoder-only LLMs to any residual architecture with attention-based aggregation, since only residual Jacobians and attention pooling are used; the paper hints at multimodal extensions but does not prove the transfer.
  • One could test the theory by artificially shrinking the spectral gap, for instance by adding a strong second concept direction, and predicting the precise layer at which the SPP stops tracking the original concept.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes the Input-Space Linearity Hypothesis (ISLH), which posits that concept-aligned directions exist in the input embedding space and are propagated through transformer layers along Spectral Principal Paths (SPP). The main theoretical claim is Theorem 4.5, a stability bound on the angular drift of the leading singular vector of a difference matrix between adjacent layers, and Theorem 4.8, which derives the Linear Representation Hypothesis (LRH) from ISLH via SPP. The paper also reports empirical measurements of spectral gaps and angular stability on four LLMs, plus a context-homogeneity ablation. The central derivation, however, rests on a stability theorem that is false as stated, and the LRH derivation is not a rigorous consequence of the stated assumptions.

Significance. If the theorems were correct, the paper would provide a conditional, testable mechanism for how linear concept directions persist across transformer layers, complementing the largely descriptive LRH literature. The authors deserve credit for attempting to make each assumption empirically measurable and for including a causal-style ablation on context coherence. Nevertheless, because the main stability theorem is false as stated and the LRH derivation contains substantial unproven steps, the theoretical contribution is not currently sound. The paper's conditional framing (ISLH as a hypothesis) is honest, but the promised 'rigorous stability guarantees' are not delivered.

major comments (3)
  1. [Theorem 4.8 and its proof in §4.5] The bound on the fluctuation term is not a consequence of Conditions 1–2. In the proof of Theorem 4.5, Step 3(a) claims ‖D^(l)_fluct‖_op ≤ σ_1^(l) √η^(l), citing Condition 2 and the rank-1 approximation d_k ≈ c_k v_1. When the rank-1 approximation is exact and the mean Jacobian perturbation and remainder vanish (d_k = c_k v_1, \barΔ=0, ξ=0), the rows of D^(l)_fluct equal the fluctuation vectors f_k, so ‖D^(l)_fluct‖_op^2 = λ_max(∑_k f_k f_k^⊤) = M η^(l). The claimed inequality then reduces to √M ≤ σ_1^(l), which is not scale-invariant and is false in explicit examples: take M=100, d=4096, d_k = 0.1 c_k v_1, E_k iid N(0, 0.01/d I); then σ_1 ≈ 1, η ≈ 2.4×10^(-8), ‖D_fluct‖ ≈ 1.6×10^(-3), but σ_1√η ≈ 1.6×10^(-4). Thus the bound in Eq. (9) is false as stated, and Theorem 4.8, which invokes Theorem 4.5, loses its premise.
  2. [Lemma 4.2, Eq. (4)] The proof of Theorem 4.8 does not constitute a valid derivation of LRH. Eq. (12) treats the singular value decomposition of the product of Jacobians as if it factorizes into products of individual singular values and inter-layer alignment factors; this identity does not hold for non-commuting matrices, and no justification is given. Furthermore, even if the principal direction u_1^(L-1) were stable, the fact that the average or aggregate difference Δh^(L) is proportional to u_1^(L-1) does not imply that every individual counterfactual difference h_+,(L)_k − h_−,(L)_k lies in Cone(u_1^(L-1)), which is what the LRH definition in Eq. (1) requires. The proof's '≈' steps are unquantified, so the claimed formal derivation is at best heuristic.
  3. [Lemma 4.2, Eq. (4)] The proof of Lemma 4.2 treats the softmax attention weights A_{N,i} as constants when computing E[h|c], but these weights depend on the input tokens and hence on the concept label c. As a result, the equality in Eq. (4) does not follow from the ISLH model as written; it requires an additional assumption (e.g., that attention weights are concept-independent) or a carefully stated first-order approximation. Since \tilde v_1 is the seed direction for the SPP, this gap affects the foundation of the framework.
minor comments (4)
  1. [Appendix A.2, Step 2] The proof of Theorem 4.5 in Step 2 says 'By Theorem 4.5, this direction deviates from v_1^(l) by at most arcsin(δ^(l))'; this is circular and should instead read 'by the definition of δ^(l)'.
  2. [Appendix A.3.2, Proposition A.4] The justification 'as τ^2 ≤ B^2 d by Jensen' in the variance bound is inaccurate; the inequality follows directly from the almost-sure norm bound ∥f_k∥ ≤ B, not from Jensen's inequality.
  3. [Section 5.2] The empirical section does not compute the right-hand side of Eq. (9) or compare the measured spectral gap and incoherence to the bound's thresholds; the reported destructive-interference ratio of 2.7–2.9× is not directly connected to the Wedin bound's requirements.
  4. [Section 5 / Appendix B] The experimental description is confusing: COCO is an image dataset, yet the concepts studied (honesty, fairness) and the token-wise scoring in Appendix B suggest text or multimodal prompts; the dataset construction should be clarified.

Circularity Check

2 steps flagged · score 6.0 of 10

Theorem 4.5's proof assumes the rank-1 row structure (d_k ≈ c_k v_1) that Theorem 4.8 then uses to conclude LRH; ISLH is LRH relocated to the input, so the origin claim is assumed rather than derived.

  1. other [Appendix A.2, Step 3(a) of the proof of Theorem 4.5; used in Theorem 4.8]
    "(a) ∥D(l) fluct∥op ≤σ(l)1 sqrt(η(l)) (from Condition 2 and the rank-1 approximation d(l)k ≈ c(l)k v(l)1);"

    The key fluctuation bound is justified by assuming d_k^(l) ≈ c_k^(l) v_1^(l), i.e. that every row of D^(l) lies along the leading singular direction. That row-wise one-dimensionality is exactly the linear-representation structure that Theorem 4.8 claims to derive (Δh^(L) ∝ u_1^(L−1), a single 1D subspace). The approximation is not listed among Conditions 1–2 or the ζ-condition, and Condition 2 bounds a different object, Γ^(l) built from layer-transition differences f_k. Spectral gap alone does not imply row-wise rank-1. Thus the stability bound that supplies Θ_1≈1 and the decay of k>1 terms in Theorem 4.8 is obtained only by assuming the target conclusion. The chain ISLH → SPP stability → LRH therefore reduces at its critical step to this rank-1 ansatz.

  2. renaming known result [Definition 4.1 (Section 4.1); Abstract; Theorem 4.8]
    "Definition 4.1: "The ISLH posits that the linear structure described by LRH is a concept-aligned direction already present in the input embedding space: x_i^(c)=μ_i+c·α_i v_in+ϵ_i, ∀i∈{1,...,N}""

    The paper frames ISLH as the answer to 'how these representations originate', but Definition 4.1 defines ISLH as the very 'linear structure described by LRH' transplanted into the input embedding, parameterized token-wise. Theorem 4.8 then derives deep-layer LRH from this input-space linearity, so the deep linearity is a propagated restatement of the input hypothesis rather than an independent account of its origin. The Limitations section concedes that 'both LRH and ISLH itself remains a hypothesis', confirming that the origin claim is assumption-presupposing. This is a relocation/renaming of the known LRH hypothesis rather than a first-principles derivation of it.

full rationale

The empirical portion is genuinely external and non-circular: spectral gaps, destructive-interference ratios, and cross-layer alignments are measured on four public LLMs, and the Diverse-vs-Homogeneous ablation is a real controlled comparison. No self-citation chain or imported uniqueness theorem is used. The theoretical derivation, however, is not self-contained in its central step. In Appendix A.2 the proof of Theorem 4.5 bounds D_fluct using the unstated rank-1 row approximation d_k ≈ c_k v_1^(l); this is precisely the one-dimensional LRH structure that Theorem 4.8 claims to conclude, so the stability guarantee and the subsequent derivation of LRH are partially circular by construction. Separately, the advertised explanation of where linear representations 'originate' is obtained by positing ISLH, which is defined as the LRH linear structure already present in the input space; the paper's own Limitations acknowledges this remains an unvalidated hypothesis. These two issues make the central claim partially circular, while leaving the empirical measurements and the propagation calculation as independent content. Consistent with the rubric, this is a 6 rather than a 0-2: one or more of the paper's theoretical 'predictions' reduce, at a load-bearing proof step, to an assumption equivalent to the target conclusion.

Assumptions & free parameters 2 free parameters · 6 assumptions · 0 invented entities

No new physical or ontological entities are postulated. SPP is a mathematical object (leading right singular vector of a difference matrix), not an exogenous entity with independent evidence. The ledger therefore records only the posited input-space structure and the assumptions used in the proofs.

free parameters (2)
  • alpha_i (token-specific concept strength)
    ISLH (Eq. 3) assumes each token has a scalar alpha_i multiplying the shared concept direction v_in. This value is never estimated; the aggregation result in Lemma 4.2 depends on it.
  • sigma^2 (input noise variance)
    Eq. 3 assumes independent Gaussian noise with variance sigma^2 on every input embedding. No estimation or diagnostic for the noise model is given.
assumptions (6)
  • ad hoc to paper Input-Space Linearity Hypothesis (ISLH): x_i = mu_i + c alpha_i v_in + epsilon_i with a single concept direction v_in.
    The paper's central premise, stated in Definition 4.1, and only indirectly tested through later spectral quantities. It is not derived from training or architecture.
  • domain assumption Attention weights A_{N,i} are approximately independent of the concept value c when computing the expected difference in Lemma 4.2.
    The proof of Lemma 4.2 (Appendix A.1) averages over c without considering that softmax weights depend on query/key vectors, which can vary with c.
  • domain assumption Layer transition is first-order linearizable: d^{l+1} = J_k d^l + xi with small higher-order remainder.
    Eq. 6 and Theorem 4.5 assume the Jacobian approximation holds for finite concept differences; the paper concedes tight worst-case bounds are lacking.
  • ad hoc to paper Rank-1 approximation d_k^l is close to c_k v_1^l is accurate enough to bound the fluctuation term in Theorem 4.5.
    Used in Appendix A.2 to link Condition 2 (built from f_k) to the actual perturbation E_k d_k; no error bound is supplied for this approximation.
  • domain assumption Per-sample fluctuations are roughly independent and isotropic for the O(tau^2/d) bound in Proposition A.4.
    The concentration result assumes zero-mean isotropic fluctuations; the empirical test only reports a ratio, not isotropy or independence.
  • standard math Wedin sinTheta theorem and Weyl's inequality give the stated singular-vector and singular-value perturbation bounds.
    Standard matrix perturbation results, cited as Wedin (1972) and used in Theorem 4.5 and Lemma A.2.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Spectral Principal Paths: A Spectral Perspective on Linear Representation Formation in LLMs." pith.science (2026). https://pith.science/paper/M475D7EF

@misc{pith2026250608543,
  author       = {Pith},
  title        = {Pith review of: Spectral Principal Paths: A Spectral Perspective on Linear Representation Formation in LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/M475D7EF}},
  note         = {Machine review of arXiv:2506.08543}
}
abstract

High-level representations have become a central focus in enhancing AI transparency and control, shifting attention from individual neurons or circuits to structured semantic directions that align with human-interpretable concepts. While the Linear Representation Hypothesis (LRH) suggests that such directions emerge in representations, it remains unclear how these representations originate and why they become increasingly stable across layers. To solve this issue, we introduce the Input-Space Linearity Hypothesis, positing that concept-aligned directions originate in the input space and are steadily maintained with increasing depth. We then propose the Spectral Principal Path (SPP) framework, which formalizes how deep networks progressively distill linear representations along the spectral principal directions. We provide rigorous stability guarantees for the SPP based on the Wedin $\sin\Theta$ perturbation theorem, identifying testable conditions, including spectral gap and context incoherence, that jointly ensure layer-wise directional preservation. By bridging theoretical analysis with empirical evidence, this work identifies a spectral view of how linear representations arise in LLMs, and suggests potential implications for concept-level controllable, robust, and coherent approaches to fairness and transparency in modern AI systems.

Figures

Figures reproduced from arXiv: 2506.08543 by the authors.

Figure 1
Figure 1. Overview of the Spectral Principal Path framework. The conceptual seed v (0) 1 originates in the input space (Phase I: ISLH) and is robustly propagated through network layers via the mean Jacobian ¯J (l) (Phase II). The stability of this path is mathematically guaranteed by (I) Spectral Gap and (II) Context Incoherence, which collectively suppress the destructive influence of context fluctuations. Ultimately, the cu… view at source ↗
Figure 2
Figure 2. Spectral dominance of the concept direction across four LLMs. (a) Spectral gap [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Context incoherence verification. (a) Individual perturbation norm. (b) Collective [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Token-wise scores for abstract concepts generated by a VLM. Green indicates a [PITH_FULL_IMAGE:figures/full_fig_p018_4.png]
Figure 5
Figure 5. Figure 5: VLM response for fearlessness concept with ocean image. Green = high fearlessness [PITH_FULL_IMAGE:figures/full_fig_p019_5.png]
Figure 6
Figure 6. Figure 6: VLM response for power concept with U.S. Capitol image. Red = high power [PITH_FULL_IMAGE:figures/full_fig_p020_6.png]
Figure 7
Figure 7. Figure 7: Attention matrix visualization across different layers. [PITH_FULL_IMAGE:figures/full_fig_p020_7.png]
Figure 8
Figure 8. Figure 8: Temporal LAT Scans for Honesty, Power, Fearlessness, and Fairness. The left [PITH_FULL_IMAGE:figures/full_fig_p021_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

27 extracted references · 7 canonical work pages

  1. [2]

    Then, with probability at least1−2 exp(−t 2/2), η(l) ≤ τ2 d + B2 (t+ √ d)√ M·d

    are indepen- dent, zero-mean random vectors in Rd satisfying ∥f(l) k ∥ ≤Band E[f(l) k (f(l) k )⊤] = τ2 d Id (isotropic). Then, with probability at least1−2 exp(−t 2/2), η(l) ≤ τ2 d + B2 (t+ √ d)√ M·d . (25) For M≫d , this gives η(l) →τ 2/d, which is small when individual fluctuations are spread across many dimensions. Proof. Recall that Γ(l) = 1 M ∑M k=1 ...

    work page 2015

  2. [3]

    Evalu- ating large language models trained on code.arXiv preprint arXiv:2107.03374,

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evalu- ating large language models trained on code.arXiv preprint arXiv:2107.03374,

    arXiv

  3. [4]

    19 Under review as a conference paper at COLM 2026 Figure 6: VLM response for power concept with U.S

    To quantify this process, we compute token-wise projection scores following RepE (Zou et al., 2023), measuring how closely activations align with concept directions at each layer. 19 Under review as a conference paper at COLM 2026 Figure 6: VLM response for power concept with U.S. Capitol image. Red = high power score, green = low. Figure 7: Attention mat...

    work page 2023

  4. [6]

    The llama 3 herd of models.arXiv preprint arXiv:2407.21783,

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783,

    arXiv

  5. [9]

    10 Under review as a conference paper at COLM 2026 Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, L´elio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timoth´ee Lacroix, and...

    work page 2026

  6. [10]

    Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton

    URL https://arxiv.org/abs/2310.06825. Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of neural network representations revisited. InInternational conference on machine learning, pp. 3519–3529. PMlR,

    arXiv

  7. [11]

    Does circuit analysis interpretability scale? evidence from multiple choice capabilities in chinchilla.arXiv preprint arXiv:2307.09458,

    Tom Lieberum, Matthew Rahtz, J´anos Kram´ar, Neel Nanda, Geoffrey Irving, Rohin Shah, and Vladimir Mikulik. Does circuit analysis interpretability scale? evidence from multiple choice capabilities in chinchilla.arXiv preprint arXiv:2307.09458,

    arXiv

  8. [13]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer vision–ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part v 13, pp. 740–755. Springer,

    work page 2014

Show all 27 references
  1. [16]

    Dinov2: Learning robust visual features without supervision.arXiv preprint arXiv:2304.07193,

    Maxime Oquab, Timoth´ee Darcet, Th´eo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khali- dov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision.arXiv preprint arXiv:2304.07193,

    arXiv

  2. [17]

    The linear representation hypothesis and the geometry of large language models.arXiv preprint arXiv:2311.03658,

    Kiho Park, Yo Joong Choe, and Victor Veitch. The linear representation hypothesis and the geometry of large language models.arXiv preprint arXiv:2311.03658,

    arXiv

  3. [18]

    Unsupervised representation learning with deep convolutional generative adversarial networks.arXiv preprint arXiv:1511.06434,

    11 Under review as a conference paper at COLM 2026 Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks.arXiv preprint arXiv:1511.06434,

    2026 arXiv

  4. [21]

    Striving for simplicity: The all convolutional net.arXiv preprint arXiv:1412.6806,

    Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin Riedmiller. Striving for simplicity: The all convolutional net.arXiv preprint arXiv:1412.6806,

    arXiv

  5. [23]

    Ethical and social risks of harm from language models.arXiv preprint arXiv:2112.04359,

    Laura Weidinger, John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang, Myra Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh, et al. Ethical and social risks of harm from language models.arXiv preprint arXiv:2112.04359,

    arXiv

  6. [24]

    Representation engineering: A top-down approach to ai transparency.arXiv preprint arXiv:2310.01405,

    Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, et al. Representation engineering: A top-down approach to ai transparency.arXiv preprint arXiv:2310.01405,

    arXiv

  7. [25]

    signal” matrix D(l) ¯J(l)⊤ and a “perturbation

    12 Under review as a conference paper at COLM 2026 Ethics Statement This work is theoretical and empirical in nature; we do not collect human subject data, and all experiments are conducted on publicly available models and datasets. We foresee no direct harm arising from this ...

    2026

  8. [1959]

    Simcse: Simple contrastive learning of sentence embeddings.arXiv preprint arXiv:2104.08821,

    Tianyu Gao, Xingcheng Yao, and Danqi Chen. Simcse: Simple contrastive learning of sentence embeddings.arXiv preprint arXiv:2104.08821,

    arXiv

  9. [1996]

    Truthfulqa: Measuring how models mimic human falsehoods.arXiv preprint arXiv:2109.07958,

    Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human falsehoods.arXiv preprint arXiv:2109.07958,

    arXiv

  10. [2013]

    Progress measures for grokking via mechanistic interpretability.arXiv preprint arXiv:2301.05217,

    Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, and Jacob Steinhardt. Progress measures for grokking via mechanistic interpretability.arXiv preprint arXiv:2301.05217,

    arXiv

  11. [2014]

    Intriguing properties of neural networks.arXiv preprint arXiv:1312.6199,

    Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks.arXiv preprint arXiv:1312.6199,

    arXiv

  12. [2016]

    Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, et al. Qwen2. 5-coder technical report.arXiv preprint arXiv:2409.12186,

    arXiv

  13. [2017]

    Bert has a moral compass: Improvements of ethical and moral values of machines.arXiv preprint arXiv:1912.05238,

    Patrick Schramowski, Cigdem Turan, Sophie Jentzsch, Constantin Rothkopf, and Kristian Kersting. Bert has a moral compass: Improvements of ethical and moral values of machines.arXiv preprint arXiv:1912.05238,

    1912 arXiv

  14. [2019]

    Deep inside convolutional networks: Visualising image classification models and saliency maps

    Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. InarXiv preprint arXiv:1312.6034,

    arXiv

  15. [2020]

    In-context learning and induction heads.arXiv preprint arXiv:2209.11895,

    Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, et al. In-context learning and induction heads.arXiv preprint arXiv:2209.11895,

    arXiv

  16. [2021]

    Language models are few-shot learners.Advances in neural information processing systems, 33:1877–1901,

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners.Advances in neural information processing systems, 33:1877–1901,

    1901

  17. [2022]

    On the opportunities and risks of foundation models.arXiv preprint arXiv:2108.07258,

    Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportunities and risks of foundation models.arXiv preprint arXiv:2108.07258,

    arXiv

  18. [2023]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL-HLT), ...

    2019

  19. [2024]

    Language models represent space and time.arXiv preprint arXiv:2310.02207,

    Wes Gurnee and Max Tegmark. Language models represent space and time.arXiv preprint arXiv:2310.02207,

    arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.