{"id":"83249f2a-837c-486d-aba5-74ab254227cd","arxiv_id":"2509.08778","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Qwen-family language models recall facts mainly through early attention layers, while GPT and LLaMA models recall facts through early MLP layers.","lead":"The authors ran causal-tracing experiments on GPT, LLaMA, Qwen, and DeepSeek language models to map where factual knowledge is stored and retrieved. They report that Qwen-based models rely more on early attention layers for factual recall, while GPT and LLaMA models rely on early MLP layers.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Knockout attribution in Qwen lacks a non-factual control: zeroing early Attention at the last subject token may lower objects rate through generic disruption, so the central claim is not yet settled.","rationale":"Good-faith reading: the paper is a useful empirical extension of causal tracing to four model families. It ships code and data, uses established methods, and includes a single-token subject control in Appendix D. The restoration effect in Qwen at early attention is visually clear, and the fact that GPT/LLaMA reproduce earlier MLP results lends internal consistency. However, the central claim is specifically about Qwen attention storing facts, not merely being causally upstream. The strongest evidence for that claim is the knockout experiment, because the authors themselves show severing does not reduce AIE in Qwen attention (Section 5.2). Their explanation, that attention has alternative paths, means the knockout cannot be read as a localized deletion of a stored fact: zeroing a module output is a much coarser intervention than severing with corrupted values. The absence of control interventions at unrelated positions or random layers, and the absence of a zero-versus-corrupted comparison, leaves open the possibility that the objects-rate drop reflects generic architectural sensitivity of Qwen's early attention at any token, or the size of the ablated update. Appendix D's single-token subject analysis reduces one confound (token aggregation) but does not address this one. The reader's weakest assumption is therefore the right one. Since a single control experiment could settle it, the appropriate verdict remains CONDITIONAL; no change to the reader's verdict is needed. If the proposed controls fail, the paper should be reframed as reporting a position-specific causal upstreamness effect, not a storage localization result.","tokens_in":17534,"tokens_out":7164,"duration_ms":62977,"concrete_test":"Rerun the Section 4.3 knockout on Qwen-2.5-1.5B and DeepSeek-R1-Distill-Qwen-1.5B adding three arms: (1) apply the same zero-out window to a non-subject token position (e.g., a token in the relation phrase) and compare objects-rate drop; (2) at the last subject token, replace the target module output with its corrupted-run value (the Section 4.2 severing-style intervention) rather than zero; (3) perform the zero-out on a version of the prompts where the object is randomized so no factual association is present. If the unrelated-position or corrupted-replacement arms produce drops comparable to the last-subject-token zero-out in Qwen, then the early-attention localization is an artifact of generic disruption or output magnitude, and the central claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.3 conducts the knockout experiment by zeroing the Attention/MLP updates at the last subject token over five-layer windows and converts the resulting objects-rate drop into a claim about where facts are stored (Figure 5, Section 5.3). This is the decisive experiment because the restoration result in Qwen is contradicted by the severing result (Section 5.2: severing early Attention yields only minimal AIE reduction), and the paper explains that contradiction by saying Attention has alternative paths. But if alternative paths make severing ineffective, then zeroing the attention output is not a clean removal of the stored fact; it removes a whole component of the residual stream. Without controls at unrelated token positions or random layers, and without a corrupted-replacement (rather than zero) condition, the observed drop in Qwen could reflect the model's general sensitivity to early-attention ablation at any token, or the magnitude of the removed update, not factual association content. Appendix D's single-token subject control addresses tokenization, not this generic-disruption confound. Thus the inference from knockout drop to 'early Attention stores Qwen facts' is conditional.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper compares where factual associations are stored in autoregressive Transformers across GPT-2, LLaMA-3.2, Qwen-2.5, and DeepSeek-R1 distilled models. Using COUNTERFACT prompts, it runs three intervention-based evaluations: restoration effects (corrupting subject embeddings and restoring individual hidden states), severing effects (corrupting a module's output within a restored hidden state), and knockout effects (zeroing five-layer windows of Attention or MLP updates at the last subject token). The main claim is that in Qwen-based and Qwen-distilled DeepSeek models, early Attention layers contribute more to factual recall than early MLP layers, contrary to the GPT/LLaMA pattern where early MLP layers dominate. The paper also introduces a semantic-similarity-based objects rate metric and a Gini-coefficient concentration analysis.","tokens_in":17842,"tokens_out":3657,"duration_ms":28420,"significance":"If substantiated, the central claim would show that factual recall localization is architecture-dependent, with direct consequences for knowledge-editing and interpretability tools that currently target early MLP layers. The paper makes a good-faith effort to resolve an internal inconsistency between restoration and severing results, releases code and data, and extends the analysis to 17 model configurations in the appendix. However, the decisive knockout experiment lacks the controls needed to attribute the objects-rate drop to stored factual associations rather than generic sensitivity to ablating early attention, and small-sample, threshold-dependent, no-error-bar results currently leave the claim conditional.","major_comments":[{"comment":"The knockout experiment zeroes the Attention and MLP updates at the last subject token over five consecutive layers, but no control intervention is reported at an unrelated token position, at randomly selected layers, or with a corrupted-replacement (e.g., mean activation) rather than zero. Without such controls, the substantial objects-rate drop observed in Qwen-2.5-1.5B and DeepSeek-R1-Distill-Qwen-1.5B could reflect a generic sensitivity of these models to early-attention ablation or the magnitude of the removed update, rather than the location of the stored fact. Since this experiment is the basis for the central claim in Section 5.3, the authors should add at least one control condition and show that the Qwen-specific drop disappears for control tokens/layers.","section":"Section 4.3, Figure 5, Eqs. (7)-(8)"},{"comment":"The paper reports that in Qwen and DeepSeek, severing the high-AIE early Attention layers reduces AIE by only 8.25% and -0.14%, respectively, while knockout of the same modules produces a large objects-rate drop (Section 5.3, Figure 5). The reconciliation offered in Section 5.2—that attention has alternative paths—undercuts the knockout interpretation: if information can bypass a severed attention layer, then zeroing the attention output does not cleanly remove a stored fact; it removes a full component of the residual stream whose contribution may be generic. The authors should test this directly, for example by comparing the knockout effect on factual prompts with the knockout effect on matched non-factual prompts or on prompts with the subject replaced by a different entity.","section":"Section 5.2, Table 2"},{"comment":"Only 100 successfully predicted facts are used per model, the figures contain no error bars, and the semantic threshold tau=0.7 (Eq. (8)) is justified only by example pairs in Table 4, not by a sensitivity analysis. Because the cross-architecture ordering (Qwen early-attention vs. LLaMA/GPT early-MLP) rests on objects-rate differences in Figure 5, the authors should report confidence intervals or bootstrap ranges and show that the ordering is stable over a range of tau (e.g., 0.6 to 0.8) and over different random samples of prompts.","section":"Section 4.3 and Appendix B"}],"minor_comments":[{"comment":"The Average Indirect Effect (AIE) is used throughout but is never formally defined; the authors should provide the aggregation formula for AIE over prompts to match Eq. (5).","section":"Section 4.2, Eq. (6)"},{"comment":"The negative drop rate of -0.14% for DeepSeek-R1-Distill-Qwen-1.5B Attention severing is never discussed; the authors should explain whether this reflects measurement noise or a genuine increase in AIE after severing.","section":"Table 2"},{"comment":"The sentence 'We hypothesized that multi-token subjects which consists may contribute...' contains a grammatical error; also the model name 'DeepSeek-R1-Distill-Qwen-2.5-1.5B' in Figure 20 does not match the name 'DeepSeek-R1 Distill-Qwen-1.5B' used elsewhere.","section":"Appendix D.1"},{"comment":"The text in Figure 4 and Figure 15 appears to contain unencoded glyph strings (e.g., the long /uni... sequence in the Figure 4 caption), making the plots unreadable in the submitted PDF; these need to be regenerated with a proper font.","section":"Figures 4 and 15"},{"comment":"The phrase 'as showed in Figure 1' should be 'as shown in Figure 1'.","section":"Section 1"}],"recommendation":"major_revision","confidential_remarks":"The paper reports an interesting empirical divergence, but the central claim currently rests on an under-controlled knockout experiment. I encourage the authors to add control conditions and uncertainty quantification. The negative drop rate in Table 2 is a red flag that should be addressed. The paper is within scope for cs.CL; the main question is whether the empirical claim survives the additional controls."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"What you should know: this paper runs the standard causal-tracing, severing, and knockout toolkit over GPT, LLaMA, Qwen, and DeepSeek families and gets a clean split — GPT and LLaMA localize factual recall in early MLPs, while Qwen (and Qwen-distilled DeepSeek) localize it in early attention layers. That split holds across parameter scales and survives a single-token subject control, so it is not a tokenization artifact. This is the first time attention-centric recall has been mapped across a whole major open-source family, and if it holds, knowledge-editing tools for Qwen should target early attention, not early MLPs.\n\nCredit where it is earned: the paper is honest about its own messiness. Restoration and severing experiments disagree for Qwen, and the authors say so explicitly, then try to reconcile the discrepancy with a Gini-concentration analysis and a knockout experiment. Code and data are public. The appendix covers 17 models, not just four. The semantic-similarity objects-rate metric is a sensible fix for the brittleness of string matching.\n\nThe soft spot is the knockout experiment. The central claim rests on zeroing out attention or MLP updates over a five-layer window at the last subject token and watching the objects rate drop. The stress-test concern is legitimate: there is no non-factual control. Zeroing early attention at one token could hurt any prediction, not just factual ones, and the magnitude of the removed update is not controlled. The cross-family comparison partially reassures — LLaMA does not drop when attention is zeroed, only when MLP is, so this is not a universal sensitivity to attention ablation. But it could still be a Qwen-specific sensitivity to early-attention ablation in general. A control on non-factual prompts, or a corrupted-replacement intervention instead of zeroing, would settle it.\n\nOther limits are real but minor: 100 successfully predicted facts per model is a small sample; figures have no error bars; the tau=0.7 threshold gets no sensitivity analysis; and the models differ in training data and objectives, so the paper's framing that architectural variations cause the difference is stronger than the evidence. The authors themselves hedge in Appendix D and the limitations section, which I appreciate.\n\nWho this is for: interpretability and model-editing researchers, especially anyone working with Qwen. It deserves a serious referee — the cross-family comparison is novel and the claim is actionable — but I would not bet model-editing decisions on it until the knockout attribution gets the extra control. Send it to peer review, and ask for that control, error bars or per-prompt variance, and a sensitivity check on tau and window size.","headline":"First cross-architecture evidence that Qwen-family models store facts in early attention layers; the knockout inference is plausible but needs a non-factual control before you'd edit Qwen on it.","tokens_in":18272,"tokens_out":1967,"would_cite":true,"duration_ms":398325,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Qwen-family transformers store factual recall in early attention layers, unlike GPT and LLaMA, which use early MLP layers.","keywords":["factual association recall","causal tracing","knowledge editing","autoregressive transformers","attention vs MLP localization","Qwen architecture","semantic similarity evaluation"],"falsifier":"Run the identical five-layer knockout on a non-subject token position, such as the relation token or the first word of the prompt, in Qwen-2.5-1.5B; if the drop in objects rate is comparable to the drop seen at the last subject token, then the intervention measures generic disruption, not where facts are stored.","tokens_in":17307,"feed_emoji":"🧠","tokens_out":6061,"duration_ms":50079,"temperature":0.7,"pith_summary":"The paper asks whether the familiar finding that early MLP layers store factual associations, established mostly on GPT-family models, holds across other autoregressive transformers. Using causal-tracing restoration, severing, and knockout experiments on GPT, LLaMA, Qwen, and DeepSeek models, it claims that Qwen-based models place the weight of factual recall at the last subject token in the early attention layers, not the MLP. If correct, factual-recall localization is architecture-dependent, and interpretability and knowledge editing for Qwen-family models should target early attention rather than early MLP. The paper also argues that standard severing-based causal tracing underestimates attention contributions, and that knockout plus semantic-similarity evaluation gives a more reliable comparison.","feed_headline":"Qwen models recall facts in early attention layers","feed_subtitle":"Knockout tests across 17 GPT, LLaMA, Qwen, and DeepSeek models show factual recall site depends on architecture.","key_machinery":"The machinery is causal tracing over module outputs, implemented as three interventions. Restoration effects restore a clean module activation inside a corrupted run and measure the Average Indirect Effect (AIE) on the object-token probability; severing effects replace the target module's activation with its corrupted value; knockout effects zero out both attention and MLP updates at the last subject token over five consecutive layers. The paper adds a Gini coefficient over the AIE distribution to measure how concentrated each module's contribution is, and a semantic-similarity objects rate computed with Sentence-BERT cosine similarity at a 0.7 threshold to score predictions beyond exact string matches. The knockout intervention is the load-bearing tool: it directly blocks module output and is what reveals early attention in Qwen, where severing fails.","core_discovery":"The central claim is that in Qwen-based autoregressive transformers, the early layers of the attention module contribute more to factual association recall at the last subject token position than the MLP module does. The paper reports restoration-effect maps showing high Average Indirect Effect concentrated in early attention layers for Qwen-2.5 and DeepSeek-R1-Distill-Qwen models, while GPT and LLaMA models show the canonical early-MLP pattern. Gini-coefficient analysis shows the attention contribution is concentrated in a single early layer, yet severing that layer barely lowers AIE; the paper attributes this to attention's alternative residual paths and uses a five-layer knockout intervention to show early attention actually carries the recall. A semantic-similarity-based objects rate, rather than string matching, confirms the same architecture-specific split when outputs vary in wording. The paper concludes that storage location shifts with architecture, and practical tools should be chosen accordingly.","pith_inferences":["A natural testable extension is to run a rank-one knowledge-edit that writes into early attention layers of a Qwen model; success at editing the target fact while leaving others intact would confirm the localization functionally, not just causally.","Because the paper found severing attention nearly ineffective but knockout effective, finer-grained attribution such as per-head path patching might reveal that a small number of early attention heads carry the factual object information; the paper does not test this.","The single-token subject control shows tokenization is not responsible, but the architectural cause remains open; training small transformers that differ only in attention mechanism or head count could isolate which design choice produces attention-centered recall.","Editing pipelines should be validated per model family rather than per individual checkpoint, since a localization result from one Transformer family does not transfer to another."],"forward_implications":["For GPT and LLaMA models, the early-MLP localization of factual recall survives replication, so prior causal-tracing conclusions hold for those families.","For Qwen-based models, factual recall at the last subject token is concentrated in early attention layers, so factual-recall localization is not a universal Transformer property.","Severing-based attribution can mask attention's role because attention information flows through alternative residual paths; knockout interventions give a more direct measure.","Knowledge editing and interpretability tools deployed on Qwen-family models should target early attention layers rather than early MLP layers.","Semantic-similarity evaluation changes measured objects rate and should be used in cross-architecture comparisons to avoid lexical-variation artifacts."],"supporting_citations":[{"why":"Defines the autoregressive Transformer architecture whose attention and MLP modules are the objects of the evaluation.","marker":"(Vaswani et al., 2017)"},{"why":"Supplies the causal-tracing restoration and severing protocol, the COUNTERFACT dataset, and the GPT-based early-MLP result this paper tests across architectures.","marker":"(Meng et al., 2022a)"},{"why":"Supplies the knockout method, BM25 candidate-object construction, and the prior MLP-localization baseline that the paper extends with semantic evaluation.","marker":"(Geva et al., 2023)"},{"why":"Explains why severing a single attention layer fails to suppress its contribution because information re-enters through residual pathways.","marker":"(Elhage et al., 2021)"},{"why":"Provides the Sentence-BERT embeddings used to compute semantic similarity for the objects-rate metric.","marker":"(Reimers and Gurevych, 2019)"},{"why":"Documents the Qwen-2.5 architecture and tokenizer whose attention-centered recall is the paper's new finding.","marker":"(Yang et al., 2024)"},{"why":"Documents the DeepSeek-R1-Distill models used to show that distillation inherits the base model's recall localization.","marker":"(Guo et al., 2025)"}],"fun_headline_variants":["Qwen recalls facts via early attention, not MLP","Fact recall site shifts across transformer architectures","Qwen stores facts in attention; GPT and LLaMA in MLP","Knockout tests reveal Qwen's early attention recall"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The conclusion rests on the assumption that zeroing a block of five consecutive layers at the last subject token removes the stored fact itself rather than simply disrupting the model's general information flow; if the same knockout anywhere in the network produces a comparable object-rate drop, the localization story collapses.","fun_headline_variants_meta":{"raw":{"variants":["Qwen recalls facts via early attention, not MLP","Fact recall site shifts across transformer architectures","Qwen stores facts in attention; GPT and LLaMA in MLP","Knockout tests reveal Qwen's early attention recall"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000178,"raw_usage":{"total_tokens":1259,"prompt_tokens":871,"completion_tokens":388,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":487,"completion_tokens_details":{"reasoning_tokens":321}},"tokens_in":487,"tokens_out":388,"duration_ms":4230,"temperature":1.0,"reasoning_tokens":321,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T15:59:02.413359+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the identical five-layer knockout on a non-subject token position, such as the relation token or the first word of the prompt, in Qwen-2.5-1.5B; if the drop in objects rate is comparable to the drop seen at the last subject token, then the intervention measures generic disruption, not where facts are stored.","supporting_citations":[],"review_version":2}