Pith. sign in

REVIEW 5 minor 31 references

Hidden multi-step reasoning over filler tokens is readable from residual streams at 80–95% accuracy without labels.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-12 02:01 UTC pith:YNZPMH3T

load-bearing objection Solid open-weights evidence that filler-token reasoning is real, causal, and largely readable from the residual stream on clean multi-step tasks.

arxiv 2607.03502 v1 pith:YNZPMH3T submitted 2026-07-03 cs.CL cs.AIcs.LG

Reading Between the Dots: Decoding Hidden Computation across Filler Tokens

Kaley Brauer , Claudio Mayrink Verdun , Samuel Marks This is my paper
classification cs.CL cs.AIcs.LG
keywords filler tokenshidden computationchain-of-thought monitorabilityresidual streamlogit lensKV-cache transplantmechanistic interpretabilityLLM oversight
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Frontier language models can solve multi-step problems while writing only content-free filler (dots, counting sequences) between question and answer, so chain-of-thought monitoring sees nothing useful. This paper shows that the same computation is structured and legible inside the model. On four task families—single-fact addition, two-fact addition, letter-position lookups, and systems of equations—two open-weight frontier models route information through the filler region, encode intermediate values in the residual stream, and rely on those values causally. An unsupervised pipeline that only inspects hidden states recovers the intermediates at 80–95% accuracy with a strong LLM judge, without training or ground-truth labels. The central claim is therefore that monitorability is a property of the full computational trace, not merely of the surface tokens: hidden-from-output does not equal hidden-from-us on these decomposable tasks.

Core claim

On four task families with clear intermediate values, DeepSeek V3 and Kimi K2 perform structured multi-step computation over content-free filler tokens; an unsupervised residual-stream decoding pipeline recovers those intermediates at 80–95% accuracy (best LLM judge) without labels or training, showing that computation invisible to behavioral CoT monitoring remains readable from the residual stream.

What carries the argument

Unsupervised residual decoding pipeline: residual-stream states at every (layer, filler position) are projected via the logit lens, residualized by subtracting the cross-example mean, aggregated into a top-50 token list, and interpreted by an off-the-shelf LLM judge under a neutral prompt.

Load-bearing premise

The method assumes that the intermediate computation is expressible through the model’s unembedding matrix so that logit-lens projections of mid-to-late residual states plus a generic LLM judge can recover it.

What would settle it

Show that, on the same four task families, the residual-decoding pipeline (or a stronger residual-stream method) systematically fails to recover the true intermediates while the model still exhibits filler-token accuracy uplift and causal dependence on filler KV content.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Behavioral CoT monitoring alone is incomplete for filler-token regimes; residual-stream access supplies a complementary audit channel.
  • On tasks with discrete nameable intermediates, white-box auditing can recover both successful and failed intermediate steps without supervision.
  • Positional structure of encoding is itself causal: transplanting only the positions that decode a given addend recovers most of the answer-swap effect.
  • Hybrid behavioral-plus-interpretability monitoring becomes the relevant standard when surface tokens are content-free.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the same residual structure appears under other forms of opaque surface text (obfuscated CoTs, encrypted reasoning), the same unsupervised pipeline may generalize beyond literal filler.
  • Training models to push intermediate values into directions orthogonal to the unembedding would constitute a concrete adversarial test of residual-stream monitorability.
  • The pipeline’s success on both retrieved and in-context-computed values suggests it can diagnose composition failures separately from retrieval failures in multi-hop settings.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

0 major / 5 minor

Summary. The paper shows that two open-weights frontier MoE models (DeepSeek V3, Kimi K2) perform multi-step reasoning over content-free filler tokens (dots, counting, alphabet sequences) on four task families with clear intermediates: 1-fact addition, 2-fact addition, 2-hop letter position, and in-context systems of equations. Behavioral accuracy rises with non-disruptive filler length (Fig. 2). Mechanistically, attention redistributes into a question o filler o answer relay (§4.1), logit-lens heatmaps reveal intermediates (A1 early, A2 later, sum late; Fig. 3–4 and App. D), and KV-cache transplants confined to filler positions (or to the subset of positions that decode each addend) causally raise donor-answer rank by tens to hundreds of places and recover 85–93% of the whole-filler effect (Table 1 / App. E). An unsupervised pipeline (residual logit lens + cross-example mean subtraction + aggregation + neutral LLM judge) recovers the intermediates at 80–95% top-2 accuracy (best judge) across models and tasks without labels or training (Fig. 6, App. H–I). The authors conclude that computation invisible to surface CoT monitoring remains readable from the residual stream on these tasks.

Significance. The result is significant for CoT monitorability and white-box oversight. It converts a known failure mode of behavioral monitoring (filler-token reasoning) into a positive claim that the same computation is legible in the residual stream, supported by three independent lines of evidence plus a practical unsupervised decoder. Strengths include causal position-resolved transplants with CIs, McNemar tests, residualization and judge-prompt ablations, incorrect-example diagnostics that separate retrieval from composition failures (App. J), shuffled-token controls collapsing to ~1%, and public code plus aggregated outputs. The claim is carefully scoped to discrete nameable intermediates, which is appropriate and does not over-reach. If the pattern generalizes, hybrid behavioral+interpretability auditing becomes a concrete path forward for safety-relevant monitoring.

minor comments (5)
  1. [§4.2 / Figure 4] Figure 4 caption and surrounding text: the qualitative claim that filler residual streams “resemble the model’s ordinary single-pass computation” more than written CoT is visually clear but would benefit from a simple quantitative summary (e.g., layer-order correlation of peak decode strength across the five chain quantities) so readers can gauge the strength of the resemblance without relying solely on the aggregate heatmap.
  2. [§6.5] §6.5 and Appendix K: residual subtraction helps on dots filler but can hurt on counting filler because digits become part of the baseline. A short decision rule or default recommendation for practitioners (when to residualize) would make the ablation more actionable.
  3. [Appendix A] Appendix A example prompts: the system prompt explicitly tells the model there will be filler tokens. A brief note on whether uplift and encoding persist under a fully silent filler (no mention of filler in the system prompt) would close a small ecological-validity gap.
  4. [§6 / Figure 6] Figure 6 and §6: main-text accuracies are pooled over correct examples only. While Appendix J supplies the incorrect-example numbers and the diagnostic interpretation, a single sentence in the main text reminding the reader of this conditioning would prevent misreading the 80–95% figures as unconditional.
  5. [Appendix A / Introduction] Typos / polish: “DeepSeek V2” appears once in Appendix A (system-of-equations easier variation) where V3 is intended; “compose facts” repository link is given but the exact commit or snapshot date is not; a few long sentences in the introduction could be split for readability.

Circularity Check

0 steps flagged

No significant circularity: empirical measurement with unsupervised readout, causal transplants, and shuffled controls; nothing reduces to its inputs by construction.

full rationale

This is an empirical interpretability paper, not a first-principles derivation. Behavioral uplift (Fig. 2), attention redistribution (§4.1), logit-lens heatmaps (§4.2), and KV-cache transplants (§4.3 / App. E) are independent measurements; the unsupervised pipeline (Fig. 5) takes only residual states, subtracts a cross-example mean (not ground-truth targets), aggregates top tokens, and hands them to an off-the-shelf LLM judge under a neutral prompt with no labels or training. Accuracy is scored post hoc against known intermediates; shuffled-token controls collapse to ~1% (App. H), showing the judge is not exploiting task priors. Position-resolved transplants recover 85–93% of the whole-filler causal effect from only the positions where the lens decodes each addend, establishing that the positional structure is causal rather than a lens artifact. Citations (Lanham, Pfau, Greenblatt, Belrose, etc.) supply background and do not load-bear uniqueness or force the result. No fitted parameter is renamed a prediction, no self-definitional loop, no uniqueness theorem imported from the authors. The work is self-contained measurement against external ground truth; circularity score is zero.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The paper is empirical; it inherits standard transformer residual-stream and attention assumptions, the reliability of the logit lens in late layers, and the causal interpretation of KV-cache transplants. Free parameters are analysis choices (layer band, residualization, top-K, position-labeling threshold θ=0.15) rather than fitted physical constants. No new physical or mathematical entities are postulated.

free parameters (4)
  • layer band for logit lens = 30–60
    Analysis restricted to layers 30–60 of both models; earlier layers discarded as unreliable. Choice affects which intermediates surface.
  • position-labeling threshold θ = 0.15
    A filler position is labeled A1 (or A2) if top numeric token matches in >15% of examples; used for selective KV transplants. Ablated at 0.20/0.30.
  • top-T / top-50 aggregation = T=30, keep 50
    Per-cell top-30 residual tokens aggregated to global top-50 before judge; T and final K are free analysis choices.
  • filler length k and type = k=5–1000 (task-dependent)
    Behavioral and decoding results pooled over multiple k and filler types (dots/counting/alphabet); exact lengths chosen by experimenters.
axioms (4)
  • domain assumption Late-layer residual states projected through RMSNorm + unembedding (logit lens) are a usable, if noisy, readout of intermediate values.
    Standard in the interpretability literature (nostalgebraist 2020, Belrose et al. 2023); paper restricts to second half of layers and aggregates to mitigate known early-layer unreliability.
  • domain assumption KV-cache transplants at selected positions constitute a valid causal intervention for testing whether those positions carry task-relevant information.
    Standard activation-patching methodology (Vig, Meng, Wang, Heimersheim & Nanda).
  • domain assumption 4-bit quantized checkpoints preserve the qualitative filler-uplift and residual encoding patterns of the full-precision models.
    Authors verify behavioral uplift under quantization (Appendix B) but note possible residual-geometry effects (Limitations).
  • ad hoc to paper An off-the-shelf LLM judge given only top residual tokens can name the intermediate without being given the original prompt.
    Core of the unsupervised pipeline; validated by shuffled-token controls and by comparison to direct top-2 token match.

pith-pipeline@v1.1.0-grok45 · 40963 in / 2996 out tokens · 26263 ms · 2026-07-12T02:01:50.530352+00:00 · methodology

0 comments
read the original abstract

Frontier LLMs can perform multi-step reasoning over content-free filler tokens like dots or counting sequences, producing correct answers with no visible chain-of-thought (CoT). This is a limit case for behavioral oversight, where surface tokens carry no information about the underlying reasoning. But hidden from the output is not the same as hidden from us. On four task families (fact retrieval, parallel numeric composition, string manipulation, and in-context computation), two open-weights frontier models (DeepSeek V3, Kimi K2) compute over filler tokens in a structured, legible way: attention routes the question through the filler region to the answer, logit-lens readouts show retrieved facts emerging early and their composition crystallizing in late layers, and KV-cache transplants at filler positions causally swap outputs between examples. We introduce an unsupervised decoding pipeline that takes only hidden states as input and recovers intermediate values with 80-95% accuracy (best LLM judge) across both models and all four tasks, without ground-truth labels or training. Hidden computation that defeats behavioral CoT monitoring is, on these tasks, directly readable from the residual stream, suggesting monitorability is a property of the model's full computational trace, not just its surface tokens.

Figures

Figures reproduced from arXiv: 2607.03502 by Claudio Mayrink Verdun, Kaley Brauer, Samuel Marks.

Figure 1
Figure 1. Figure 1: Frontier-scale LLMs perform decodable hidden multi-step reasoning over meaningless filler tokens. Without filler, the model fails the question (left); with filler appended to the prompt, it succeeds (middle). Our unsupervised pipeline can decode the intermediate values (A1 = 77, A2 = 35) and their composition (112) from the residual stream at filler positions (right). 72% on DeepSeek V3), 2-fact addition (… view at source ↗
Figure 2
Figure 2. Figure 2: Filler tokens improve accuracy across tasks, models, and filler types. Uplift typically quickly increases with filler length until, at very long filler lengths, it asymptotes and even degrades. Error bars show ±1 SE under a binomial model (same fixed test set across all k values). Top left: 1-fact addition on DeepSeek V3 (API; 800 examples). All filler types produce comparable uplift while scrambled varian… view at source ↗
Figure 3
Figure 3. Figure 3: Logit-lens heatmaps show intermediate values of hidden computation are encoded in the residual stream over filler positions. For the 2-fact addition task, we apply the logit lens at each (layer, filler position). Heatmap color shows the fraction of examples where the top number token exactly matches the ground-truth target (A1, A2, or A1+A2). Top (correct examples): A1 is encoded most strongly early in the… view at source ↗
Figure 4
Figure 4. Figure 4: Logit-lens heatmaps show that filler computation is not the same as chain-of-thought when solving systems of equations. (DeepSeek V3; each panel aggregates 500 examples). Each of the five chain quantities x → c1x → y → c2y → answer has a fixed color; its opacity at a given (layer, token position) is its decode strength averaged over the 500 examples. Left (baseline, no filler): the chain resolves in a sing… view at source ↗
Figure 5
Figure 5. Figure 5: The unsupervised decoding pipeline. (1) Extract residual-stream activations at every layer and filler position. (2) Apply the logit lens at each cell and subtract a cross-example mean to remove question-independent baselines. (3) Aggregate the resulting residual scores across cells; keep the top 50 tokens per example. (4) Pass the top tokens to a judge LLM under a neutral prompt asking what the original mo… view at source ↗
Figure 6
Figure 6. Figure 6: Decoding accuracy across tasks, models, and judges. Error bars show the min/max across all filler conditions. For each task–model condition, three bars compare: (grey) the fraction of examples for which the hidden intermediate appears among the top-2 recovered tokens (no LLM judge); (light rust) the fraction for which a Haiku judge, given the top-50 decoded tokens, includes the target among its top-2 guess… view at source ↗
Figure 7
Figure 7. Figure 7: No-filler baseline, 2-fact addition (DeepSeek V3). We apply the logit lens at each (layer, position) for the end of the question and the answer forming position; color shows the fraction of examples whose top numeric token exactly matches the ground-truth target (A1, A2, or A1+A2), the same measure as [PITH_FULL_IMAGE:figures/full_fig_p018_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Kimi K2, 2-fact addition with dot-10 filler. We apply the logit lens at each (layer, filler position). Heatmap color shows the fraction of examples where the top number token exactly matches the ground-truth target (A1, A2, or A1+A2). Rows split examples by whether the model answered correctly. The same general pattern is observed as in [PITH_FULL_IMAGE:figures/full_fig_p018_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: DeepSeek V3, 2-fact addition with counting-10 filler. We apply the logit lens at each (layer, filler position). Heatmap color shows the fraction of examples where the top number token exactly matches the ground-truth target (A1, A2, or A1+A2). Rows split examples by whether the model answered correctly. The same general pattern is observed as in [PITH_FULL_IMAGE:figures/full_fig_p019_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: DeepSeek V3, 2-fact addition with dots-50 filler. We apply the logit lens at each (layer, filler position). Heatmap color shows the fraction of examples where the top number token exactly matches the ground-truth target (A1, A2, or A1+A2). Rows split examples by whether the model answered correctly. The same general pattern is observed as in [PITH_FULL_IMAGE:figures/full_fig_p019_10.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

31 extracted references · 6 linked inside Pith

  1. [1]

    Bowman and Ethan Perez , title=

    Tamera Lanham and Anna Chen and Ansh Radhakrishnan and Benoit Steiner and Carson Denison and Danny Hernandez and Dustin Li and Esin Durmus and Evan Hubinger and Jackson Kernion and Kamile Lukosiute and Karina Nguyen and Newton Cheng and Nicholas Joseph and Nicholas Schiefer and Oliver Rausch and Robin Larson and Sam McCandlish and Sandipan Kundu and Saura...

    2023

  2. [2]

    Advances in Neural Information Processing Systems , volume=

    Language models don't always say what they think: Unfaithful explanations in chain-of-thought prompting , author=. Advances in Neural Information Processing Systems , volume=

  3. [3]

    arXiv preprint arXiv:2503.08679 , year=

    Chain-of-thought reasoning in the wild is not always faithful , author=. arXiv preprint arXiv:2503.08679 , year=

    Pith/arXiv arXiv

  4. [4]

    Nature , volume=

    DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning , author=. Nature , volume=. 2025 , publisher=

    2025

  5. [5]

    Advances in neural information processing systems , volume=

    Investigating gender bias in language models using causal mediation analysis , author=. Advances in neural information processing systems , volume=

  6. [6]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    The impact of language mixing on bilingual llm reasoning , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    2025

  7. [7]

    arXiv preprint arXiv:2404.15255 , year=

    How to use and interpret activation patching , author=. arXiv preprint arXiv:2404.15255 , year=

    Pith/arXiv arXiv

  8. [8]

    The Eleventh International Conference on Learning Representations , year=

    Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 Small , author=. The Eleventh International Conference on Learning Representations , year=

  9. [9]

    Advances in neural information processing systems , volume=

    Locating and editing factual associations in gpt , author=. Advances in neural information processing systems , volume=

  10. [10]

    Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    Do Large Language Models Perform Latent Multi-Hop Reasoning without Exploiting Shortcuts? , author=. Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    2025

  11. [11]

    arXiv preprint arXiv:2507.05246 , year=

    When chain of thought is necessary, language models struggle to evade monitors , author=. arXiv preprint arXiv:2507.05246 , year=

    Pith/arXiv arXiv

  12. [12]

    arXiv preprint arXiv:2503.11926 , year=

    Monitoring reasoning models for misbehavior and the risks of promoting obfuscation , author=. arXiv preprint arXiv:2503.11926 , year=

    Pith/arXiv arXiv

  13. [13]

    arXiv preprint arXiv:2505.05410 , year=

    Reasoning models don't always say what they think , author=. arXiv preprint arXiv:2505.05410 , year=

    Pith/arXiv arXiv

  14. [14]

    Bowman , booktitle=

    Jacob Pfau and William Merrill and Samuel R. Bowman , booktitle=. Let. 2024 , url=

    2024

  15. [15]

    The Twelfth International Conference on Learning Representations , year=

    Think before you speak: Training Language Models With Pause Tokens , author=. The Twelfth International Conference on Learning Representations , year=

  16. [16]

    2025 , howpublished=

    Recent LLMs can use filler tokens or problem repeats to improve (no-CoT) math performance , author=. 2025 , howpublished=

    2025

  17. [17]

    Greenblatt, Ryan , year=. Recent

  18. [18]

    Tomek Korbak and Mikita Balesni and Elizabeth Barnes and Yoshua Bengio and Joe Benton and Joseph Bloom and Mark Chen and Alan Cooney and Allan Dafoe and Anca D. Dragan and Scott Emmons and Owain Evans and David Farhi and Ryan Greenblatt and Dan Hendrycks and Marius Hobbhahn and Evan Hubinger and Geoffrey Irving and Erik Jenner and Daniel Kokotajlo and Vic...

    2025

  19. [19]

    Interpreting

    nostalgebraist , year=. Interpreting

  20. [20]

    CoRR , volume=

    Nora Belrose and Zach Furman and Logan Smith and Danny Halawi and Igor Ostrovsky and Lev McKinney and Stella Biderman and Jacob Steinhardt , title=. CoRR , volume=. 2023 , cdate=

    2023

  21. [21]

    2021 , howpublished=

    A Mathematical Framework for Transformer Circuits , author=. 2021 , howpublished=

    2021

  22. [22]

    Forty-first International Conference on Machine Learning , year=

    Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models , author=. Forty-first International Conference on Machine Learning , year=

  23. [23]

    Do Large Language Models Latently Perform Multi-Hop Reasoning?

    Yang, Sohee and Gribovskaya, Elena and Kassner, Nora and Geva, Mor and Riedel, Sebastian. Do Large Language Models Latently Perform Multi-Hop Reasoning?. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024. doi:10.18653/v1/2024.acl-long.550

    doi:10.18653/v1/2024.acl-long.550 2024

  24. [24]

    CoRR , volume=

    Eden Biran and Daniela Gottesman and Sohee Yang and Mor Geva and Amir Globerson , title=. CoRR , volume=. 2024 , cdate=

    2024

  25. [25]

    AWQ: Activation-aware Weight Quantization for On-Device LLM Compression and Acceleration , url =

    Lin, Ji and Tang, Jiaming and Tang, Haotian and Yang, Shang and Chen, Wei-Ming and Wang, Wei-Chen and Xiao, Guangxuan and Dang, Xingyu and Gan, Chuang and Han, Song , booktitle =. AWQ: Activation-aware Weight Quantization for On-Device LLM Compression and Acceleration , url =

  26. [26]

    Zhang and Han Bao and Hanwei Xu and Haocheng Wang and Haowei Zhang and Honghui Ding and Huajian Xin and Huazuo Gao and Hui Li and Hui Qu and J

    DeepSeek-AI and Aixin Liu and Bei Feng and Bing Xue and Bingxuan Wang and Bochao Wu and Chengda Lu and Chenggang Zhao and Chengqi Deng and Chenyu Zhang and Chong Ruan and Damai Dai and Daya Guo and Dejian Yang and Deli Chen and Dongjie Ji and Erhang Li and Fangyun Lin and Fucong Dai and Fuli Luo and Guangbo Hao and Guanting Chen and Guowei Li and H. Zhang...

    2024

  27. [27]

    arXiv e-prints , keywords =

    Kimi K2: Open Agentic Intelligence. arXiv e-prints , keywords =. doi:10.48550/arXiv.2507.20534 , archivePrefix =. 2507.20534 , primaryClass =

    Pith/arXiv arXiv doi:10.48550/arxiv.2507.20534

  28. [28]

    Guo, Daya and Yang, Dejian and Zhang, Haowei and Song, Junxiao and Wang, Peiyi and Zhu, Qihao and Xu, Runxin and Zhang, Ruoyu and Ma, Shirong and Bi, Xiao and Zhang, Xiaokang and Yu, Xingkai and Wu, Yu and Wu, Z. F. and Gou, Zhibin and Shao, Zhihong and Li, Zhuoshu and Gao, Ziyi and Liu, Aixin and Xue, Bing and Wang, Bingxuan and Wu, Bochao and Feng, Bei ...

    doi:10.1038/s41586-025-09422-z

  29. [29]

    The Twelfth International Conference on Learning Representations , year=

    Efficient Streaming Language Models with Attention Sinks , author=. The Twelfth International Conference on Learning Representations , year=

  30. [30]

    Future Lens: Anticipating Subsequent Tokens from a Single Hidden State , url=

    Pal, Koyena and Sun, Jiuding and Yuan, Andrew and Wallace, Byron and Bau, David , year=. Future Lens: Anticipating Subsequent Tokens from a Single Hidden State , url=. doi:10.18653/v1/2023.conll-1.37 , booktitle=

    doi:10.18653/v1/2023.conll-1.37 2023

  31. [31]

    Psychometrika , author=

    Note on the Sampling Error of the Difference Between Correlated Proportions or Percentages , volume=. Psychometrika , author=. 1947 , pages=. doi:10.1007/BF02295996 , number=

    doi:10.1007/bf02295996 1947