Pith. sign in

REVIEW 3 major objections 5 minor 38 references

Do All Autoregressive Transformers Remember Facts the Same Way? A Cross-Architecture Analysis of Recall Mechanisms

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Qwen-family transformers store factual recall in early attention layers, unlike GPT and LLaMA, which use early MLP layers.

desk verdict First cross-architecture evidence that Qwen-family models store facts in early attention layers; the knockout inference is plausible but needs a non-factual control before you'd edit Qwen on it. read the letter →

arxiv 2509.08778 v1 pith:KSNI7JP5 submitted 2025-09-10 cs.CL

Minyeong Choe , Haehyun Cho , Changho Seo , Hyunil Kim This is my paper · ORCID
classification cs.CL
keywords factualassociationrecallcausaltracingknowledgeeditingautoregressivetransformersattentionvsMLPlocalizationQwenarchitecturesemanticsimilarityevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether the familiar finding that early MLP layers store factual associations, established mostly on GPT-family models, holds across other autoregressive transformers. Using causal-tracing restoration, severing, and knockout experiments on GPT, LLaMA, Qwen, and DeepSeek models, it claims that Qwen-based models place the weight of factual recall at the last subject token in the early attention layers, not the MLP. If correct, factual-recall localization is architecture-dependent, and interpretability and knowledge editing for Qwen-family models should target early attention rather than early MLP. The paper also argues that standard severing-based causal tracing underestimates attention contributions, and that knockout plus semantic-similarity evaluation gives a more reliable comparison.

What carries the argument

The machinery is causal tracing over module outputs, implemented as three interventions. Restoration effects restore a clean module activation inside a corrupted run and measure the Average Indirect Effect (AIE) on the object-token probability; severing effects replace the target module's activation with its corrupted value; knockout effects zero out both attention and MLP updates at the last subject token over five consecutive layers. The paper adds a Gini coefficient over the AIE distribution to measure how concentrated each module's contribution is, and a semantic-similarity objects rate computed with Sentence-BERT cosine similarity at a 0.7 threshold to score predictions beyond exact string matches. The knockout intervention is the load-bearing tool: it directly blocks module output and is what reveals early attention in Qwen, where severing fails.

What would settle it

Run the identical five-layer knockout on a non-subject token position, such as the relation token or the first word of the prompt, in Qwen-2.5-1.5B; if the drop in objects rate is comparable to the drop seen at the last subject token, then the intervention measures generic disruption, not where facts are stored.

Watch

Extended reading notes

Core claim

The central claim is that in Qwen-based autoregressive transformers, the early layers of the attention module contribute more to factual association recall at the last subject token position than the MLP module does. The paper reports restoration-effect maps showing high Average Indirect Effect concentrated in early attention layers for Qwen-2.5 and DeepSeek-R1-Distill-Qwen models, while GPT and LLaMA models show the canonical early-MLP pattern. Gini-coefficient analysis shows the attention contribution is concentrated in a single early layer, yet severing that layer barely lowers AIE; the paper attributes this to attention's alternative residual paths and uses a five-layer knockout intervention to show early attention actually carries the recall. A semantic-similarity-based objects rate, rather than string matching, confirms the same architecture-specific split when outputs vary in wording. The paper concludes that storage location shifts with architecture, and practical tools should be chosen accordingly.

Load-bearing premise

The conclusion rests on the assumption that zeroing a block of five consecutive layers at the last subject token removes the stored fact itself rather than simply disrupting the model's general information flow; if the same knockout anywhere in the network produces a comparable object-rate drop, the localization story collapses.

Editorial extensions

If this is right

  • For GPT and LLaMA models, the early-MLP localization of factual recall survives replication, so prior causal-tracing conclusions hold for those families.
  • For Qwen-based models, factual recall at the last subject token is concentrated in early attention layers, so factual-recall localization is not a universal Transformer property.
  • Severing-based attribution can mask attention's role because attention information flows through alternative residual paths; knockout interventions give a more direct measure.
  • Knowledge editing and interpretability tools deployed on Qwen-family models should target early attention layers rather than early MLP layers.
  • Semantic-similarity evaluation changes measured objects rate and should be used in cross-architecture comparisons to avoid lexical-variation artifacts.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural testable extension is to run a rank-one knowledge-edit that writes into early attention layers of a Qwen model; success at editing the target fact while leaving others intact would confirm the localization functionally, not just causally.
  • Because the paper found severing attention nearly ineffective but knockout effective, finer-grained attribution such as per-head path patching might reveal that a small number of early attention heads carry the factual object information; the paper does not test this.
  • The single-token subject control shows tokenization is not responsible, but the architectural cause remains open; training small transformers that differ only in attention mechanism or head count could isolate which design choice produces attention-centered recall.
  • Editing pipelines should be validated per model family rather than per individual checkpoint, since a localization result from one Transformer family does not transfer to another.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper compares where factual associations are stored in autoregressive Transformers across GPT-2, LLaMA-3.2, Qwen-2.5, and DeepSeek-R1 distilled models. Using COUNTERFACT prompts, it runs three intervention-based evaluations: restoration effects (corrupting subject embeddings and restoring individual hidden states), severing effects (corrupting a module's output within a restored hidden state), and knockout effects (zeroing five-layer windows of Attention or MLP updates at the last subject token). The main claim is that in Qwen-based and Qwen-distilled DeepSeek models, early Attention layers contribute more to factual recall than early MLP layers, contrary to the GPT/LLaMA pattern where early MLP layers dominate. The paper also introduces a semantic-similarity-based objects rate metric and a Gini-coefficient concentration analysis.

Significance. If substantiated, the central claim would show that factual recall localization is architecture-dependent, with direct consequences for knowledge-editing and interpretability tools that currently target early MLP layers. The paper makes a good-faith effort to resolve an internal inconsistency between restoration and severing results, releases code and data, and extends the analysis to 17 model configurations in the appendix. However, the decisive knockout experiment lacks the controls needed to attribute the objects-rate drop to stored factual associations rather than generic sensitivity to ablating early attention, and small-sample, threshold-dependent, no-error-bar results currently leave the claim conditional.

major comments (3)
  1. [Section 4.3, Figure 5, Eqs. (7)-(8)] The knockout experiment zeroes the Attention and MLP updates at the last subject token over five consecutive layers, but no control intervention is reported at an unrelated token position, at randomly selected layers, or with a corrupted-replacement (e.g., mean activation) rather than zero. Without such controls, the substantial objects-rate drop observed in Qwen-2.5-1.5B and DeepSeek-R1-Distill-Qwen-1.5B could reflect a generic sensitivity of these models to early-attention ablation or the magnitude of the removed update, rather than the location of the stored fact. Since this experiment is the basis for the central claim in Section 5.3, the authors should add at least one control condition and show that the Qwen-specific drop disappears for control tokens/layers.
  2. [Section 5.2, Table 2] The paper reports that in Qwen and DeepSeek, severing the high-AIE early Attention layers reduces AIE by only 8.25% and -0.14%, respectively, while knockout of the same modules produces a large objects-rate drop (Section 5.3, Figure 5). The reconciliation offered in Section 5.2—that attention has alternative paths—undercuts the knockout interpretation: if information can bypass a severed attention layer, then zeroing the attention output does not cleanly remove a stored fact; it removes a full component of the residual stream whose contribution may be generic. The authors should test this directly, for example by comparing the knockout effect on factual prompts with the knockout effect on matched non-factual prompts or on prompts with the subject replaced by a different entity.
  3. [Section 4.3 and Appendix B] Only 100 successfully predicted facts are used per model, the figures contain no error bars, and the semantic threshold tau=0.7 (Eq. (8)) is justified only by example pairs in Table 4, not by a sensitivity analysis. Because the cross-architecture ordering (Qwen early-attention vs. LLaMA/GPT early-MLP) rests on objects-rate differences in Figure 5, the authors should report confidence intervals or bootstrap ranges and show that the ordering is stable over a range of tau (e.g., 0.6 to 0.8) and over different random samples of prompts.
minor comments (5)
  1. [Section 4.2, Eq. (6)] The Average Indirect Effect (AIE) is used throughout but is never formally defined; the authors should provide the aggregation formula for AIE over prompts to match Eq. (5).
  2. [Table 2] The negative drop rate of -0.14% for DeepSeek-R1-Distill-Qwen-1.5B Attention severing is never discussed; the authors should explain whether this reflects measurement noise or a genuine increase in AIE after severing.
  3. [Appendix D.1] The sentence 'We hypothesized that multi-token subjects which consists may contribute...' contains a grammatical error; also the model name 'DeepSeek-R1-Distill-Qwen-2.5-1.5B' in Figure 20 does not match the name 'DeepSeek-R1 Distill-Qwen-1.5B' used elsewhere.
  4. [Figures 4 and 15] The text in Figure 4 and Figure 15 appears to contain unencoded glyph strings (e.g., the long /uni... sequence in the Figure 4 caption), making the plots unreadable in the submitted PDF; these need to be regenerated with a proper font.
  5. [Section 1] The phrase 'as showed in Figure 1' should be 'as shown in Figure 1'.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the central claim is an external empirical comparison using standard causal-tracing and knockout methods; no fitted parameter or self-citation chain forces the result.

full rationale

The paper's central claim—that Qwen-based models rely more on early Attention layers than MLPs for factual recall—is an empirical, cross-architecture comparison. The methods are adopted from external prior work (ROME/Meng et al. 2022a and Geva et al. 2023), and the dataset (COUNTERFACT) is external. The metrics IE, AIE, Gini, and ObjectsRate are defined operationally from model outputs and standard formulas; none is derived from a fitted parameter that is then renamed as a prediction. The semantic-similarity threshold tau=0.7 is a fixed evaluation criterion justified by illustrative examples in Appendix B, not optimized to produce the Qwen-attention conclusion. The knockout experiment is an additional intervention, not a restatement of the restoration or severing results. No self-citations are load-bearing, and no uniqueness theorem or ansatz is imported from the authors' own prior work. Possible weaknesses, such as the generic-disruption confound in the knockout intervention or the success-only sampling of 100 sentences, are validity threats rather than circularity: the conclusion would not be true by construction even if those threats were real. Therefore no significant circularity is present.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim depends on several empirical assumptions: the COUNTERFACT success-filtered sample is representative, knockout interventions isolate storage rather than generic disruption, the tau=0.7 semantic threshold is valid across families, and family differences can be attributed to architecture rather than training data. No new entities are postulated.

free parameters (4)
  • semantic similarity threshold tau = 0.7
    Set in Section 4.3 and defended in Appendix B with word-pair examples; no sensitivity analysis shows whether the Attention vs MLP ordering for Qwen holds for other thresholds.
  • corruption noise scale = 3 * sigma_sub
    Section 4.1; inherited from ROME, but no robustness check is reported. It sets the corrupted-run baseline for AIE.
  • knockout window size = 5 consecutive layers
    Section 4.3; chosen for the intervention, with no analysis of sensitivity to window length.
  • number of sampled prompts per model = 100
    Section 4; only successfully predicted facts are sampled, which may vary per model and affect cross-model comparability.
assumptions (4)
  • domain assumption The COUNTERFACT dataset with 100 success-filtered prompts per model is representative of factual recall.
    Section 4 states this sampling without showing distribution or variance; all conclusions are averaged over these prompts.
  • domain assumption Knockout of module updates at the last subject token isolates factual storage rather than generic model disruption.
    Section 5.3 interprets objects-rate drops as evidence of where facts are stored; no control intervention is reported for unrelated tokens or random modules.
  • domain assumption Sentence-BERT cosine similarity >= 0.7 between a generated token and a BM25 candidate is a valid measure of semantic equivalence across all tested model families.
    Appendix B justifies the threshold with word pairs; no per-model calibration or ablation is provided.
  • domain assumption Observed differences between model families are attributable to architecture rather than pretraining data or training objectives.
    Appendix D rules out single factors but does not control training data; this premise underpins the abstract's 'architectural variations' wording.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Do All Autoregressive Transformers Remember Facts the Same Way? A Cross-Architecture Analysis of Recall Mechanisms." pith.science (2026). https://pith.science/paper/KSNI7JP5

@misc{pith2026250908778,
  author       = {Pith},
  title        = {Pith review of: Do All Autoregressive Transformers Remember Facts the Same Way? A Cross-Architecture Analysis of Recall Mechanisms},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KSNI7JP5}},
  note         = {Machine review of arXiv:2509.08778}
}
read the original abstract

Understanding how Transformer-based language models store and retrieve factual associations is critical for improving interpretability and enabling targeted model editing. Prior work, primarily on GPT-style models, has identified MLP modules in early layers as key contributors to factual recall. However, it remains unclear whether these findings generalize across different autoregressive architectures. To address this, we conduct a comprehensive evaluation of factual recall across several models -- including GPT, LLaMA, Qwen, and DeepSeek -- analyzing where and how factual information is encoded and accessed. Consequently, we find that Qwen-based models behave differently from previous patterns: attention modules in the earliest layers contribute more to factual recall than MLP modules. Our findings suggest that even within the autoregressive Transformer family, architectural variations can lead to fundamentally different mechanisms of factual recall.

Figures

Figures reproduced from arXiv: 2509.08778 by the authors.

Figure 1
Figure 1. Overview of Our Evaluation. Given a sentence represented as knowledge triples of the form t = (s, r, o), the pipeline proceeds in three stages: #1 restoration effects, #2 severing effects, and #3 factual prediction. #1 and #2 are conducted using the indirect effect, which is computed from clean-run and corrupted-run. In contrast, #3 is conducted on knockout-run. Each stage applies a distinct intervention strategy to… view at source ↗
Figure 2
Figure 2. Restoration effects across multiple autoregressive Transformer models. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Severing effects across multiple autoregressive Transformer models (restricted to layers 0–15 to verify [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (17 more)
Figure 4
Figure 4. Figure 4: Gini coefficient-based concentration analysis [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Factual prediction evaluation after knockout of Attention and MLP outputs across autoregressive Trans [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Restoration effects in GPT-family autoregressive Transformer models with varying parameter scales. [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: Restoration effects in LLaMA-family autoregressive Transformer models with varying parameter scales. [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: Restoration effects in Qwen-family autoregressive Transformer models with varying parameter scales. [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: Restoration effects in DeepSeek-family autoregressive Transformer models with varying parameter scales. [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]
Figure 10
Figure 10. Figure 10: Severing effects across multiple autoregressive Transformer models (all layers). [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]
Figure 11
Figure 11. Figure 11: Severing effects across GPT-family autoregressive Transformer models at varying parameter scales. [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]
Figure 12
Figure 12. Figure 12: Severing effects across LLaMA-family autoregressive Transformer models at varying parameter scales. [PITH_FULL_IMAGE:figures/full_fig_p017_12.png]
Figure 13
Figure 13. Figure 13: Severing effects across Qwen-family autoregressive Transformer models at varying parameter scales. [PITH_FULL_IMAGE:figures/full_fig_p017_13.png]
Figure 14
Figure 14. Figure 14: Severing effects across DeepSeek-family autoregressive Transformer models at varying parameter scales. [PITH_FULL_IMAGE:figures/full_fig_p018_14.png]
Figure 15
Figure 15. Figure 15: Gini coefficient-based concentration analysis of causal effects in Attention and MLP modules across [PITH_FULL_IMAGE:figures/full_fig_p018_15.png]
Figure 16
Figure 16. Figure 16: Factual prediction evaluation after knockout of Attention and MLP outputs across GPT-family autore [PITH_FULL_IMAGE:figures/full_fig_p018_16.png]
Figure 17
Figure 17. Figure 17: Factual prediction evaluation after knockout of Attention and MLP outputs across LLaMA-family [PITH_FULL_IMAGE:figures/full_fig_p019_17.png]
Figure 18
Figure 18. Figure 18: Factual prediction evaluation after knockout of Attention and MLP outputs across Qwen-family [PITH_FULL_IMAGE:figures/full_fig_p019_18.png]
Figure 19
Figure 19. Figure 19: Factual prediction evaluation after knockout of Attention and MLP outputs across DeepSeek-family [PITH_FULL_IMAGE:figures/full_fig_p019_19.png]
Figure 20
Figure 20. Figure 20: Restoration effects across multiple autoregressive Transformer models under the single-token subject [PITH_FULL_IMAGE:figures/full_fig_p020_20.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 1 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Yossi Adi, Einat Kermany, Yonatan Belinkov, Ofer Lavi, and Yoav Goldberg. 2016. Fine-grained analysis of sentence embeddings using auxiliary prediction tasks. arXiv preprint arXiv:1608.04207

    arXiv 2016

  4. [4]

    Yonatan Belinkov. 2022. https://doi.org/10.1162/coli_a_00422 Probing classifiers: Promises, shortcomings, and advances . Computational Linguistics, 48(1):207--219

    doi:10.1162/coli_a_00422 2022

  5. [5]

    Yonatan Belinkov, Nadir Durrani, Fahim Dalvi, Hassan Sajjad, and James Glass. 2017. What do neural machine translation models learn about morphology? arXiv preprint arXiv:1704.03471

    arXiv 2017

  6. [6]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, and 1 others. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877--1901

    2020

  7. [7]

    Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D. Manning. 2019. https://doi.org/10.18653/v1/W19-4828 What does BERT look at? an analysis of BERT `s attention . In Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 276--286, Florence, Italy. Association for Computational Linguistics

    doi:10.18653/v1/w19-4828 2019

  8. [8]

    Alexis Conneau, German Kruszewski, Guillaume Lample, Lo \"i c Barrault, and Marco Baroni. 2018. https://doi.org/10.18653/v1/P18-1198 What you can cram into a single \ & ! \# * vector: Probing sentence embeddings for linguistic properties . In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p...

    doi:10.18653/v1/p18-1198 2018

Show all 38 references
  1. [9]

    Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. 2022. https://doi.org/10.18653/v1/2022.acl-long.581 Knowledge neurons in pretrained transformers . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long ...

    2022 doi

  2. [10]

    Guy Dar, Mor Geva, Ankit Gupta, and Jonathan Berant. 2023. https://doi.org/10.18653/v1/2023.acl-long.893 Analyzing transformers in embedding space . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 16124-...

    2023 doi

  3. [11]

    Robert Dorfman. 1979. A formula for the gini coefficient. The review of economics and statistics, pages 146--149

    1979

  4. [12]

    Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, Eduard Hovy, Hinrich Sch \"u tze, and Yoav Goldberg. 2021. Measuring and improving consistency in pretrained language models. Transactions of the Association for Computational Linguistics, 9:1012--1031

    2021

  5. [13]

    Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, and 1 others. 2021. A mathematical framework for transformer circuits. Transformer Circuits Thread, 1(1):12

    2021

  6. [14]

    Allyson Ettinger, Ahmed Elgohary, and Philip Resnik. 2016. Probing for semantic evidence of composition by means of simple classification tasks. In Proceedings of the 1st workshop on evaluating vector-space representations for nlp, pages 134--139

    2016

  7. [15]

    Junfeng Fang, Houcheng Jiang, Kun Wang, Yunshan Ma, Shi Jie, Xiang Wang, Xiangnan He, and Tat-Seng Chua. 2024. Alphaedit: Null-space constrained knowledge editing for language models. arXiv preprint arXiv:2410.02355

    2024 arXiv

  8. [16]

    Mor Geva, Jasmijn Bastings, Katja Filippova, and Amir Globerson. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.751 Dissecting recall of factual associations in auto-regressive language models . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language ...

    2023 doi

  9. [17]

    Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.446 Transformer feed-forward layers are key-value memories . In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5484--5495, ...

    2021 doi

  10. [18]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  11. [19]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948

    2025 arXiv

  12. [20]

    Peter Hase, Mohit Bansal, Been Kim, and Asma Ghandeharioun. 2023. Does localization inform editing? surprising differences in causality-based localization vs. knowledge editing in language models. Advances in Neural Information Processing Systems, 36:17643--17668

    2023

  13. [21]

    Dieuwke Hupkes, Sara Veldhoen, and Willem Zuidema. 2018. Visualisation and'diagnostic classifiers' reveal how recurrent and recursive neural networks process hierarchical structure. Journal of Artificial Intelligence Research, 61:907--926

    2018

  14. [22]

    Zhengbao Jiang, Frank F Xu, Jun Araki, and Graham Neubig. 2020. How can we know what language models know? Transactions of the Association for Computational Linguistics, 8:423--438

    2020

  15. [23]

    Xiaopeng Li, Shasha Li, Shezheng Song, Jing Yang, Jun Ma, and Jie Yu. 2024. Pmet: Precise model editing in a transformer. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 18564--18572

    2024

  16. [24]

    Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022 a . Locating and editing factual associations in gpt. Advances in neural information processing systems, 35:17359--17372

    2022

  17. [25]

    Kevin Meng, Arnab Sen Sharma, Alex Andonian, Yonatan Belinkov, and David Bau. 2022 b . Mass-editing memory in a transformer. arXiv preprint arXiv:2210.07229

    2022 arXiv

  18. [26]

    Hosein Mohebbi, Willem Zuidema, Grzegorz Chrupa a, and Afra Alishahi. 2023. Quantifying context mixing in transformers. arXiv preprint arXiv:2301.12971

    2023 arXiv

  19. [27]

    Judea Pearl. 2022. Direct and indirect effects. Probabilistic and Causal Inference: The Works of Judea Pearl, page 373

    2022

  20. [28]

    Fabio Petroni, Tim Rockt \"a schel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019. https://doi.org/10.18653/v1/D19-1250 Language models as knowledge bases? In Proceedings of the 2019 Conference on Empirical Methods in Natural Language P...

    2019 doi

  21. [29]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, and 1 others. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9

    2019

  22. [30]

    Nils Reimers and Iryna Gurevych. 2019. https://arxiv.org/abs/1908.10084 Sentence-bert: Sentence embeddings using siamese bert-networks . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics

    2019 arXiv

  23. [31]

    Adam Roberts, Colin Raffel, and Noam Shazeer. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.437 How much knowledge can you pack into the parameters of a language model? In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages...

    2020 doi

  24. [32]

    Stephen E Robertson, Steve Walker, Susan Jones, Micheline M Hancock-Beaulieu, Mike Gatford, and 1 others. 1995. Okapi at trec-3. Nist Special Publication Sp, 109:109

    1995

  25. [33]

    Victor Sanh, Thomas Wolf, and Alexander Rush. 2020. Movement pruning: Adaptive sparsity by fine-tuning. Advances in neural information processing systems, 33:20378--20389

    2020

  26. [34]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30

    2017

  27. [35]

    Jesse Vig. 2019. https://doi.org/10.18653/v1/P19-3007 A multiscale visualization of attention in the transformer model . In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 37--42, Florence, Italy. Associatio...

    2019 doi

  28. [36]

    Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. 2020. Investigating gender bias in language models using causal mediation analysis. Advances in neural information processing systems, 33:12388--12401

    2020

  29. [37]

    Ben Wang and Aran Komatsuzaki. 2021. Gpt-j-6b: A 6 billion parameter autoregressive language model

    2021

  30. [38]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, and 1 others. 2024. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115

    2024 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.