Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

A single-layer, two-head attention-only transformer reaches perfect accuracy on the symbolic IOI task.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 07:37 UTC pith:NGOFA72Y

load-bearing objection A clean toy result under-sold by an in-sample-only evaluation; the two-head additive/contrastive story is plausible but needs a held-out test and more runs. the 4 major comments →

arxiv 2510.25013 v2 pith:NGOFA72Y submitted 2025-10-28 cs.CL cs.AIcs.LG

Emergence of Minimal Circuits for Indirect Object Identification in Attention-Only Transformers

Rabin Adhikari This is my paper
classification cs.CL cs.AIcs.LG
keywords Indirect Object Identificationmechanistic interpretabilityattention-only transformersminimal circuitsQK/OV circuit analysisresidual stream decompositioncomposition analysis
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper trains tiny attention-only transformers from scratch on a symbolic version of the Indirect Object Identification (IOI) task, where the model must output the correct indirect object from a fixed six-token template. It claims that a one-layer model with exactly two attention heads achieves perfect accuracy, and that the heads specialise into an additive subcircuit (aggregating both candidate names) and a contrastive subcircuit (suppressing the incorrect candidate). The paper further shows that a two-layer, one-head model reaches comparable accuracy by composing the first layer's output into the second layer's queries and values. The significance, if the claims hold, is that even a minimal attention-only architecture can implement a clean, interpretable coreference mechanism, offering a tractable testbed for studying how transformers reason.

Core claim

The central discovery is a minimal, two-head circuit for indirect object identification. Residual stream decomposition shows that head 0's output aligns with the sum of the correct and incorrect name directions, while head 1's output aligns with their difference; adding the two contributions cancels the wrong name's logit and amplifies the right one. Eigenvalue analysis of the QK and OV matrices supports this reading: head 0's OV circuit has all positive eigenvalues (copying), head 1's QK circuit has a dominant negative eigenvalue (suppression), and its OV circuit mixes additive and rotational components. In a two-layer, one-head model, the paper finds that composition across layers occurs m

What carries the argument

The key machinery is the decomposition of the residual stream at the final token position into per-head contributions, combined with spectral analysis of the QK and OV matrices. The additive-contrastive mechanism lives in the two attention heads: head 0 attends to both names in the dependent clause and writes a sum signal; head 1 attends to the main-clause subject and the opposite name and writes a difference signal. The spectral asymmetry (quantified by the fraction of positive eigenvalues) is used as evidence that the heads have specialised, with head 1 showing strong inhibitory dynamics. For the two-layer model, composition ablations that zero out first-layer contributions to Q, K, or V s

Load-bearing premise

The load-bearing premise is that perfect accuracy on all 60 possible sequences from the fixed template indicates a genuine learned circuit, rather than template memorisation, since no held-out data are used.

What would settle it

Train the same two-head model on a random 80% of the 60 template sequences and test on the remaining 20%; if accuracy on the held-out sequences drops far below perfect, the claimed generalizable circuit is instead a memorisation of the training distribution.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • A two-head attention-only architecture (no MLPs, no normalization) is sufficient for perfect performance on the fixed-template symbolic IOI task.
  • The additive-contrastive division of labour is a concrete, minimal mechanism for coreference-like reasoning in transformers.
  • The two-layer, one-head model's reliance on Q and V composition, rather than K, indicates that layer-to-layer information flow for this task runs through queries and values.
  • The spectral eigenvalue signature gives an inspectable marker for identifying additive versus contrastive heads in other trained models.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the additive-contrastive motif is a general primitive, then other reference-resolution tasks (pronoun disambiguation, entity linking) might reduce to similarly structured two-head circuits; training on parallel templates would test this directly.
  • The exhaustive in-distribution evaluation leaves open whether the circuit generalises to new names or reordered templates; a held-out test would settle whether the model learned a reusable algorithm or memorised the 60 sequences.
  • The spectral asymmetry metric could be used as a cheap, task-agnostic diagnostic for head specialisation in larger transformers, independent of residual-stream projection analysis.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper trains small attention-only Transformers from scratch on a symbolic version of the Indirect Object Identification (IOI) task, using a vocabulary of six names and two fixed 6-token templates. It reports that a single-layer model with two attention heads reaches 100% accuracy, and claims that the two heads specialize into additive and contrastive subcircuits, as supported by residual-stream decomposition, QK/OV eigenvalue spectra, and attention-pattern analyses. It further studies a two-layer, one-head model and claims that it composes information across layers. The stated contributions are that minimal attention-only models can solve IOI perfectly and that task-constrained training yields interpretable circuits.

Significance. If the empirical claims are robust, this work offers a tractable minimal testbed for studying mechanistic interpretability of attention-only transformers, and the proposed additive/contrastive decomposition could inform hypotheses about how larger models solve coreference-like tasks. However, the current evidence is insufficient to establish that the model learns a generalizable rule rather than memorizing the exhaustive set of 60 sequences; the mechanistic interpretations are largely post hoc and correlational. The paper is transparent about some limitations, but the central claims require stronger experimental validation.

major comments (4)
  1. [§2.1–§2.2] No held-out evaluation is performed. The dataset consists of all 60 unique sequences (§2.1), and each training batch contains all 60 (§2.2). With a small two-head model and only 60 input-output pairs, perfect in-sample accuracy could result from memorizing the training set. To support the claim that the model 'solves IOI' and learns a reusable circuit, the authors must evaluate on sequences not seen during training (e.g., train on one template and test on the other, hold out name pairs, or vary the template structure). Without such a test, the central claim that a generalizable algorithm emerged is not supported.
  2. [§2.2] Training details are incomplete: no number of training steps or epochs, no random seeds, and no variance across runs are reported. The central empirical result is that the two-head model achieves perfect accuracy, so the paper should report at least the final accuracy across several seeds and, ideally, training curves. The use of a high maximum learning rate (0.1) and a OneCycle schedule makes sensitivity analysis particularly important.
  3. [Abstract vs §3.3] The abstract states that the two-layer, one-head model 'composes information from the previous layer primarily through query-key interactions,' but the Composition Ablation in §3.3 reports that Q composition causes ≈100% accuracy drop and V composition ≈93.33% drop, while K composition causes only ≈26.67% drop. This directly contradicts the abstract. The abstract should say 'query-value interactions' (or the ablation results should be re-evaluated).
  4. [§3.2.2–§3.2.3] The additive/contrastive interpretation is based on post hoc projections and eigenvalue statistics, not on causal interventions. The manuscript itself concedes in §3.2.2 that the residual-stream decomposition is 'not foolproof' and that Head 1 also has components in the correct-token and sum directions. To make the mechanistic claim load-bearing, the authors should perform targeted causal interventions (e.g., ablating the 'difference' direction from Head 1's output, or zeroing out Head 0/Head 1 contributions) and measure the effect on logits and accuracy. Without such evidence, the specialization narrative is a plausible interpretation, not a demonstrated mechanism.
minor comments (5)
  1. [§3.2.3] The quantity called 'fraction of positive eigenvalues' is actually the sum of eigenvalues divided by the sum of absolute values. This is not a fraction of positive eigenvalues (it can be negative, as in Figure 5). Please rename it to something like 'eigenvalue asymmetry' and define it correctly.
  2. [§3.3, Figure 9] The caption of Figure 9 appears garbled ('Layer 1 Head 0 Layer 1 Head 0 Layer 1 Head 0' etc.). The figure would be clearer if subplots were labeled individually.
  3. [§3.2.1, Figure 3] The notation S1/S2 in Figure 3 is not defined in the caption. Clarify that S1 and S2 are the two name tokens in the dependent clause (e.g., IO and S in the template).
  4. [§2.2] The paper never states the total number of parameters. A brief statement of model size would help contextualize the memorization concern.
  5. [§3.2.3, Reference [16]] The claim 'We expect an equal number of positive and negative eigenvalues for a random matrix' is not precisely what the cited theorem states. Please state the exact result from the reference and why it applies here.

Circularity Check

0 steps flagged

No significant circularity: the perfect-accuracy result is a direct training measurement, and the additive/contrastive interpretation is post-hoc and does not define or produce that result.

full rationale

The paper's central empirical claim is that a one-layer, two-head attention-only transformer reaches 100% accuracy on the 60-template symbolic IOI set (Sections 2.2 and 3.2). This is a training outcome, not a quantity derived from the interpretability analysis. The 'additive/contrastive' circuit description (Section 3.2.2) is obtained after training by projecting residual-stream component outputs onto embedding directions; it is used to explain the already-measured accuracy, not to predict it. Even if that projection is imperfect—the paper itself notes 'This analysis is also not foolproof'—no fitted parameter is renamed as a prediction and no equation is defined in terms of the target result. The composition ablation in Section 3.3 is also an empirical intervention. References to prior work (Elhage et al., Wang et al., Shlegeris) are external, not self-citations, and none is load-bearing in the sense of providing an unverified uniqueness or ansatz that forces the conclusion. The absence of a held-out split is a genuine generalization/memorization concern, but that is a validity limitation, not circularity: the paper's reported accuracy is exactly what was optimized and measured. No self-definitional, fitted-input-called-prediction, self-citation, uniqueness-import, ansatz-smuggling, or renaming step can be exhibited with a quoted reduction, so the circularity score is 0.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 0 invented entities

The paper is an empirical study: its central claims rest on the faithfulness of the symbolic dataset, the linear-circuit decomposition assumptions, and hand-selected training hyperparameters. No theoretical entities are invented; the two-head circuit is observed post hoc.

free parameters (5)
  • max learning rate = 0.1
    Hand-chosen OneCycle maximum learning rate; no sweep reported.
  • weight decay = 0.01
    Hand-chosen; no ablation.
  • residual stream dimension = 8
    Chosen for interpretability; the additive/contrastive decomposition depends on this low-dimensional space.
  • name vocabulary size = 6
    Defines the 60-sequence exhaustive dataset; no held-out names are tested.
  • training duration (epochs/steps)
    Not reported; convergence criteria absent, so the perfect-accuracy claim is not reproducible from the paper alone.
axioms (4)
  • domain assumption The 6-token symbolic two-name template faithfully captures the relational core of IOI.
    Introduced in Section 2.1; if the symbolic pattern is not a faithful IOI abstraction, the discovered circuit does not transfer to natural-language IOI.
  • domain assumption Residual stream contributions add linearly and final logits are sums of component outputs.
    Used in Section 3.2.2 to infer additive/contrastive roles; standard in transformer circuits but not causally verified here.
  • domain assumption Eigenvalue structure of QK/OV matrices reveals functional specialization (copy vs. suppress).
    Used in Section 3.2.3; spectral asymmetry is correlated with the hypothesized roles but no causal intervention establishes it.
  • standard math Random-matrix null expectation of balanced positive/negative eigenvalues is the right baseline.
    Cited as Tarnowski [16] in Section 3.2.3; this background result frames the spectral asymmetry.

pith-pipeline@v1.3.0-alltime-deepseek · 7889 in / 14615 out tokens · 150042 ms · 2026-08-04T07:37:06.613491+00:00 · methodology

0 comments
read the original abstract

Mechanistic interpretability aims to reverse-engineer large language models (LLMs) into human-understandable computational circuits. However, the complexity of pretrained models often obscures the minimal mechanisms required for specific reasoning tasks. In this work, we train small, attention-only transformers from scratch on a symbolic version of the Indirect Object Identification (IOI) task, a benchmark for studying coreference-like reasoning in transformers. Surprisingly, a single-layer model with only two attention heads achieves perfect IOI accuracy, despite lacking MLPs and normalization layers. Through residual stream decomposition, spectral analysis, and embedding interventions, we find that the two heads specialize into additive and contrastive subcircuits that jointly implement IOI resolution. Furthermore, we show that a two-layer, one-head model composes information from the previous layer primarily through query-key interactions. These results demonstrate that task-specific training induces highly interpretable, minimal circuits, offering a controlled testbed for probing the computational foundations of transformer reasoning.

Figures

Figures reproduced from arXiv: 2510.25013 by Rabin Adhikari.

Figure 1
Figure 1. Figure 1: Single-Head, One-Layer Model Fails to Learn IOI. (a) The attention heatmap showing the <MID> token attends uniformly to the two names. (b) The QK circuit reveals that the <MID> token attends uniformly to all tokens. (c) The OV circuit shows that each name token has a large positive contribution to its own logit and a small negative contribution to the other name’s logit. Following Elhage et al. [5], we ana… view at source ↗
Figure 2
Figure 2. Figure 2: shows distinct attention patterns of the two heads, averaged across all the possible inputs. <BOS> IO S1 S2 <MID> Key <BOS> IO S1 S2 <MID> Query Attention Pattern: Layer 0 Head 0 <BOS> IO S1 S2 <MID> Key <BOS> IO S1 S2 <MID> Query Attention Pattern: Layer 0 Head 1 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The attention map for the second head depends on the template. While the first head always attends to the two name tokens in the dependent clause, the second head attends to the second occurrence of the subject in the main clause and the other name in the dependent clause — “BA” in the “BAAB” template and similarly, “AB” in the “BABA” template. 3.2.2 Residual Stream Decomposition To understand how the mode… view at source ↗
Figure 4
Figure 4. Figure 4: Residual Stream Decomposition for Two-Head, One-Layer Model. The dot product of the output of each of the components of the residual stream. The first head’s output aligns most with the sum direction, while the second head’s output aligns most with the difference direction. Spectral Properties of QK Circuits Observing the eigendecomposition of the QK matrices (see figure 5), we note the following: • Head 0… view at source ↗
Figure 5
Figure 5. Figure 5: Eigenvalue Distribution of QK Circuits for Two-Head, One-Layer Model. Head 1 has a larger dominant negative eigenvalue (positive fraction of −0.65) compared to Head 0 (positive fraction of −0.06), indicating a stronger suppressive effect in Head 1’s attention dynamics. Spectral Properties of OV Circuits Eigendecomposition of the OV matrices (see figure 6) further reveals the asymmetry between the two heads… view at source ↗
Figure 6
Figure 6. Figure 6: Eigenvalue Distribution of OV Circuits for Two-Head, One-Layer Model. The first head has all positive eigenvalues (positive fraction of 1), indicating an additive contribution to the residual stream. In contrast, the second head has positive as well as imaginary eigenvalues with negative real parts (positive fraction of 0.55), suggesting a mix of additive and contrastive contributions. 3.2.4 Positional Foc… view at source ↗
Figure 7
Figure 7. Figure 7: Average Attention Heatmap for Two-Head, One-Layer Model with Averaged Name Embeddings. Head 0 focuses almost equally on the positional embeddings of the two name tokens from the dependent clause, while Head 1 attends primarily to the positional embedding of the subject token of the main clause. 6 [PITH_FULL_IMAGE:figures/full_fig_p006_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Average Attention Heatmap for Two-Head, One-Layer Model trained without Posi￾tional Embeddings. Both heads focus mostly on the correct output name present in the dependent clause, i.e., B in the “BAAB” template and A in the “BABA” template. 3.3 Two-Layer, One-Head Model and Composition Motivation We also train a one-head, two-layer attention-only transformer to observe how a model performs IOI in the avail… view at source ↗
Figure 9
Figure 9. Figure 9: Attention Heatmap for Two-Layer, One-Head Model. The heatmap of both heads changes depending on the template. <BOS> IO S1 S2 <MID> Key <BOS> IO S1 S2 <MID> Query Attention Pattern: Layer 0 Head 0 <BOS> IO S1 S2 <MID> Key <BOS> IO S1 S2 <MID> Query Attention Pattern: Layer 1 Head 0 0.0 0.2 0.4 0.6 0.8 1.0 [PITH_FULL_IMAGE:figures/full_fig_p008_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Attention Heatmap for Two-Layer, One-Head Model with Averaged Name Embed￾dings. We see a strong positional focus in both heads. of the Q, K, and V matrices. We observe a drop in accuracy in the following order: Q composition (≈ 100% drop), V composition (≈ 93.33% drop), and K composition (≈ 26.67% drop). This indicates that the model is heavily relying on the Q and V compositions to perform the task. So, … view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Mechanistic Interpretability for Neural Networks: Circuits, Sparse Features and Symbolic Reasoning

    cs.LG 2026-07 accept

    A scoping review surveying circuit analysis, sparse autoencoders, activation steering, and neurosymbolic frameworks for interpreting and controlling Transformer-based neural networks.

Reference graph

Works this paper leans on

19 extracted references · 3 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Mechanistic Interpretability for AI Safety - A Review

    Leonard Bereska and Stratis Gavves. “Mechanistic Interpretability for AI Safety - A Review”. In:Trans. Mach. Learn. Res.2024 (2024) (see page: 1)

    2024

  2. [2]

    A Mechanis- tic Analysis of a Transformer Trained on a Symbolic Multi-Step Reasoning Task

    Jannik Brinkmann, Abhay Sheshadri, Victor Levoso, Paul Swoboda, and Christian Bartelt. “A Mechanis- tic Analysis of a Transformer Trained on a Symbolic Multi-Step Reasoning Task”. In:ACL (Findings). Association for Computational Linguistics, 2024, pp. 4082–4102 (see page: 1)

    2024

  3. [3]

    Language Models are Few-Shot Learners

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- V oss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litw...

    2020

  4. [4]

    Towards Automated Circuit Discovery for Mechanistic Interpretability

    Arthur Conmy, Augustine N. Mavor-Parker, Aengus Lynch, Stefan Heimersheim, and Adrià Garriga- Alonso. “Towards Automated Circuit Discovery for Mechanistic Interpretability”. In:NeurIPS. 2023 (see page: 1)

    2023

  5. [5]

    A Mathematical Framework for Transformer Circuits

    Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield- Dodds, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. ...

    2021

  6. [6]

    Transformer Feed-Forward Layers Are Key-Value Memories

    Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. “Transformer Feed-Forward Layers Are Key-Value Memories”. In:EMNLP (1). Association for Computational Linguistics, 2021, pp. 5484–5495 (see page: 1)

    2021

  7. [7]

    Decoupled Weight Decay Regularization

    Ilya Loshchilov and Frank Hutter. “Decoupled Weight Decay Regularization”. In:ICLR (Poster). Open- Review.net, 2019 (see page: 2)

    2019

  8. [8]

    Version v2.16.1

    Neel Nanda and Joseph Bloom.TransformerLens. Version v2.16.1. Aug. 26, 2022.URL: https:// github.com/TransformerLensOrg/TransformerLens(see page: 2)

    2022

  9. [9]

    Progress Measures for Grokking via Mechanistic Interpretability

    Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, and Jacob Steinhardt. “Progress Measures for Grokking via Mechanistic Interpretability”. In:ICLR. OpenReview.net, 2023 (see page: 1)

    2023

  10. [10]

    LessWrong

    nostalgebraist.interpreting GPT: the logit lens. LessWrong. Aug. 31, 2020.URL: https : / / www . lesswrong . com / posts / AcKRB8wDpdaN6v6ru / interpreting - gpt - the - logit - lens (see page: 4)

    2020

  11. [11]

    Zoom In: An Introduction to Circuits

    Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. “Zoom In: An Introduction to Circuits”. In:Distill5.3 (2020), e00024–001 (see page: 1)

    2020

  12. [12]

    GPT-4 Technical Report

    OpenAI. “GPT-4 Technical Report”. In:CoRRabs/2303.08774 (2023) (see page: 1)

    Pith/arXiv arXiv 2023

  13. [13]

    Language Models are Unsupervised Multitask Learners

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. “Language Models are Unsupervised Multitask Learners”. In:OpenAI blog1.8 (2019), p. 9 (see page: 1)

    2019

  14. [14]

    AI Alignment Forum

    Buck Shlegeris.One-layer transformers aren’t equivalent to a set of skip-trigrams. AI Alignment Forum. Feb. 18, 2023.URL: https : / / www . alignmentforum . org / posts / b5HNYh9ne5vEkX5ag / one - layer-transformers-aren-t-equivalent-to-a-set-of-skip(see page: 2)

    2023

  15. [15]

    Super-Convergence: Very Fast Training of Residual Networks Using Large Learning Rates

    Leslie N. Smith and Nicholay Topin. “Super-Convergence: Very Fast Training of Residual Networks Using Large Learning Rates”. In:CoRRabs/1708.07120 (2017) (see page: 2)

    Pith/arXiv arXiv 2017

  16. [16]

    Real Spectra of Large Real Asymmetric Random Matrices

    Wojciech Tarnowski. “Real Spectra of Large Real Asymmetric Random Matrices”. In:Physical Review E105.1 (Jan. 2022), p. L012104 (see page: 4)

    2022

  17. [17]

    LLaMA: Open and Efficient Foundation Language Models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurélien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. “LLaMA: Open and Efficient Foundation Language Models”. In:CoRRabs/2302.13971 (2023) (see page: 1)

    Pith/arXiv arXiv 2023

  18. [18]

    Attention is All you Need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. “Attention is All you Need”. In:NIPS. 2017, pp. 5998–6008 (see pages: 1, 2)

    2017

  19. [19]

    In- terpretability in the Wild: A Circuit for Indirect Object Identification in GPT-2 Small

    Kevin Ro Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. “In- terpretability in the Wild: A Circuit for Indirect Object Identification in GPT-2 Small”. In:ICLR. OpenReview.net, 2023 (see page: 1). 9

    2023