{"id":"433ae3e9-4cc7-4293-aa0a-29b402a786a1","arxiv_id":"2507.11809","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Attention heads in GPT-2 and Pythia-6.9B that promote factual output act by general copy suppression rather than selective counterfactual suppression, with domain-dependent effects that sharpen in larger models.","lead":"This paper re-runs and extends three prior studies on how LLM attention heads mediate the competition between model-memorized facts and false statements in the prompt. It finds the relevant heads suppress copying in general, not counterfactuals specifically, and that their influence depends on knowledge domain and scales with model size.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Missing random-head control on the fact-replaced prompts leaves the copy-suppression conclusion vulnerable to a generic perturbation artifact.","rationale":"The paper's new and central claim is that factual-recall heads operate through general copy suppression rather than selective counterfactual suppression. The decisive evidence is Section 4.3.2 / Figure 5, where replacing the counterfactual token with the factual token makes strengthening the target heads reduce factual predictions. For this inference to be valid, the intervention must specifically activate the copy-suppression function of these heads rather than simply perturbing the model. The paper itself demonstrates awareness of this confound by including a random-head baseline (Appendix B.1, Figure 9), but that baseline is only applied to the original counterfactual prompts, not to the fact-replaced prompts used for the central claim. The reported Pythia-6.9B behavior at alpha = 10, where the model outputs degenerate tokens such as 'the' or 'also', makes generic destabilization a real alternative explanation. A random-head control on the fact-replaced dataset would directly settle whether the observed drop is specific to the target heads. The reader's weakest-assumption analysis identifies the same issue, and the existing CONDITIONAL verdict remains appropriate pending this check. I do not find an internal inconsistency in the main logic; the concern is an uneliminated confound, not a formal error.","tokens_in":11407,"tokens_out":4609,"duration_ms":59963,"concrete_test":"Run the same attention-modification protocol (Section 3.1.2, alpha in {1, 2, 5, 10, 100}) on the fact-replaced datasets from Section 4.3.1, but with the random heads from Appendix B.1 (several seeds) instead of the target heads (L10H7/L11H10 for GPT-2 and the Pythia heads), and compare factual prediction ratios to Figure 5. If random heads produce a comparable drop at alpha <= 10, the effect is a generic perturbation artifact and the copy-suppression conclusion is unsupported. If target heads drop significantly more than random heads, the concern is resolved. As a secondary check, compute logit attribution in the fact-replaced prompts to verify the target heads still promote the factual token before modification.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central mechanistic claim (Section 5 and the abstract) rests on Figure 5 in Section 4.3.2: when the counterfactual token in the prompt is replaced by the factual token, strengthening the target heads (alpha >= 5) lowers factual predictions, which is interpreted as general copy suppression. The load-bearing assumption is that this drop specifically reflects the heads' copy-suppression function rather than a generic destabilizing effect of attention modification. The paper's own random-head baseline (Appendix B.1, Figure 9) is run only on the original counterfactual prompts, not on the fact-replaced prompts. Notably, for Pythia-6.9B the authors report that at alpha = 10 the model 'mostly return[s] \"the\" or \"also\"', which is exactly the kind of non-specific degradation that a control would detect. Without a random-head control on the fact-replaced dataset, the observed decrease in factual predictions could be explained by perturbation-induced instability rather than by copy suppression, and the headline conclusion would lose support.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper is a reproducibility study of prior mechanistic-interpretability work on factual and counterfactual competition in GPT-2 and Pythia-6.9B. The authors replicate the attention-head modification experiments of Ortu et al., probe the effect of prompt wording and sentence structure, and test two competing mechanisms: selective suppression of counterfactual tokens versus general copy suppression (anti-induction). Using prompts in which the counterfactual token is replaced by the factual token, they find that strengthening tfact-supporting heads lowers the proportion of factual predictions, which they interpret as evidence for general copy suppression. They also split the data by subject and answer categories and report that head contributions are category-dependent, with stronger category selectivity in Pythia-6.9B than in GPT-2. The paper makes its code available and largely reproduces the prior results it builds on.","tokens_in":11622,"tokens_out":5742,"duration_ms":61108,"significance":"If the copy-suppression conclusion holds, it changes the interpretation of heads labeled 'factual recall heads': rather than selectively suppressing counterfactual continuations, these heads suppress copying from context indiscriminately, which would explain why strengthening them can hurt factual outputs when the fact is present in the prompt. This is a valuable, falsifiable result with direct consequences for attention-editing interventions and for the robustness of mechanistic explanations. The study is strengthened by its use of held-out manipulations (fact-replaced prompts, category splits) rather than parameter fitting, by its reliance on the original code and datasets, and by its open-source release. The main interpretive risk is that the key intervention lacks a same-dataset random-head control, so the conclusion is conditional on that control.","major_comments":[{"comment":"The central conclusion that tfact-supporting heads perform general copy suppression rather than counterfactual-specific suppression rests on the fact-replaced-prompt intervention in Figure 5, but that experiment has no random-head control on the fact-replaced dataset. The random-head baseline in Appendix B.1 (Figure 9) is run only on the original counterfactual prompts, and the paper itself reports that Pythia-6.9B at alpha=10 'mostly return[s] \"the\" or \"also\"' (Section 4.3.2), which is exactly the kind of non-specific degradation that a control is meant to detect. Without a random-head control on the same fact-replaced prompts, the observed drop in factual predictions could be a generic perturbation artifact rather than copy suppression, so Hypothesis 2.2.2 is not yet decisively preferred over Hypothesis 2.2.1.","section":"§4.3.1–4.3.2, Figure 5"},{"comment":"The claim that larger models exhibit more specialized and category-sensitive attention heads is based on comparing GPT-2 (117M parameters, 12 layers, 12 heads) with Pythia-6.9B (32 layers, 32 heads). This comparison confounds model size with model family, vocabulary, training data, and architecture, so Figures 7, 10, and 11 cannot by themselves support Hypothesis 2.3.2. Section 5 acknowledges that experiments across multiple sizes within one model family are needed; this should be treated as a required experiment, not a future-work remark.","section":"§4.4.2, §5 (Hypothesis 2.3.2)"},{"comment":"Most quantitative results are reported as single-run counts with no error bars or seed variation (for example, Table 1 and Figures 2 and 5), even though the random-head baseline in Appendix B.1 uses four seeds. Without variance estimates, statements such as 'α = 10 yields better results than α = 5' for same-structure sentences (Section 4.2.2) and the numerical size of the drop in Figure 5 cannot be distinguished from run-to-run noise. The authors should report repeated runs or bootstrap intervals for at least the main intervention experiments.","section":"§4.1–4.4 (Tables 1, Figures 2–5)"},{"comment":"The domain-specialization analysis depends entirely on GPT-4o-generated category labels, but the paper only says the classification results were 'manually checked' without specifying the prompting scheme, the number or definition of categories, or the checking and agreement procedure. Since Figures 6–8 and the head-selection analysis in Appendix B.2 are defined by these labels, this omission is a reproducibility gap for the domain claims.","section":"§4.4.1"}],"minor_comments":[{"comment":"The typo 'answe,r' in the first section should be corrected.","section":"Introduction"},{"comment":"The caption lists values as counts but does not state the model or dataset; the text implies GPT-2 and the 10,000-entry CounterFact-derived set, but the caption should say so explicitly.","section":"Table 1 caption"},{"comment":"Phrases such as 'This may be due to that' and 'the results become less pronounced' should be copy-edited for clarity.","section":"§4.2.2"},{"comment":"Figure 9 would be easier to compare with Figure 5 if both plots used the same axes and response categories.","section":"Appendix B.1, Figure 9"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid reproducibility contribution and the requested random-head control on the fact-replaced prompts is a cheap, well-scoped addition that would substantially strengthen the central claim. The editor may also want to ask the authors to document the GPT-4o category-label audit in enough detail to be independently repeatable. No concerns about novelty disclosure: the paper transparently builds on prior code and datasets."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is worth your time. It reproduces Ortu et al.'s core results, then runs the decisive experiment they didn't: swap the counterfactual token in the prompt for the factual one and strengthen the supposed 'factual recall' heads. Factual predictions drop. That is a real, new piece of evidence, and it reframes prior head-strengthening interventions as generally dangerous rather than selectively helpful. The category-sliced logit attribution and SVD analysis also give the domain-specialization claim more support than I expected. The authors are honest about limits: they note the scaling comparison is confounded across architectures, they flag that hint/no-hint dataset composition differs between models, and they openly say more model sizes are needed for hypothesis 2.3.2. That honesty earns credit.\n\nThe soft spots are real but not fatal. The main one is exactly what the stress-test note says: the random-head baseline (Appendix B.1, Figure 9) runs only on the original counterfactual prompts, not on the fact-replaced prompts used in Figure 5. So we don't know whether the drop in factual predictions reflects copy suppression or just generic perturbation-induced instability. The authors' own observation that Pythia-6.9B at alpha=10 'mostly returns the or also' is a red flag—that looks like degradation, not mechanism. A random-head control on the fact-replaced dataset would have cleanly answered this, and its absence leaves the headline claim one experiment short of convincing. Also, most results are single-run with no error bars; the GPT-4o category labels rest on an unspecified manual check. These are minor compared to the missing control, but they're worth knowing.\n\nWho is this for? Anyone working on mechanistic interpretability of factual recall, and anyone tempted to boost 'factual heads' as an intervention. It's a solid reproduction with a corrective hypothesis. I'd bring it to reading group and cite it. If this crossed my desk as a referee, I'd send it out—it's exactly the kind of empirical check the field needs—but I'd ask for the missing control before accepting.\n\nBottom line: engage with it, but treat the copy-suppression conclusion as a strong hypothesis rather than a settled result until that control exists.","headline":"A genuinely useful reproducibility study whose central claim—that 'factual recall' heads do general copy suppression—is plausible but lacks the one control that would nail it.","tokens_in":662,"tokens_out":759,"would_cite":true,"duration_ms":29846,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Attention heads that promote factual output do so by general copy suppression, not by selectively suppressing counterfactual tokens.","keywords":["Mechanistic interpretability","Attention heads","Counterfactual reasoning","Copy suppression","Factual recall","Domain specialization","Reproducibility study","Large language models"],"falsifier":"A direct falsifier would be a matched-pair experiment in which the context states either a falsehood or the corresponding fact with the same surface structure, and some head raises factual output only in the falsehood case; if such a head exists, the general copy-suppression account fails. Simpler still, on the fact-replaced prompts one could measure whether the suppression is specific to tokens that match an earlier context position or instead hits all high-probability tokens, since copy suppression predicts the former.","tokens_in":11235,"feed_emoji":"🧠","tokens_out":8902,"duration_ms":84497,"temperature":0.7,"pith_summary":"The paper tries to settle a dispute about why certain attention heads in generative language models shift the output away from a counterfactual statement in the prompt and toward a fact stored in the model's parameters. Prior studies called these 'factual recall heads' and proposed that they selectively suppress the copied counterfactual token. Working with GPT-2 and Pythia-6.9B on CounterFact-derived prompts, this study reproduces the factual-versus-counterfactual competition and then tests the two candidate mechanisms directly. The decisive result is that strengthening the heads also suppresses correct facts when the prompt itself states the fact, indicating general copy suppression rather than targeted counterfactual vetoing. The paper also shows that the heads' contributions are domain-dependent, with larger models displaying more specialized and category-sensitive patterns.","feed_headline":"Heads that boost facts also block correct facts","feed_subtitle":"A new study shows 'factual recall' heads work by general copy suppression, so boosting them hurts when the fact is already in the prompt.","key_machinery":"The central object is the transformer attention head, specifically the heads L10H7 and L11H10 in GPT-2 and L17H28, L20H18, L21H8 in Pythia-6.9B, which earlier work flagged as 'factual recall heads.' The paper carries the argument with two tools: attention modification, which scales (by a factor $\\alpha$) the attention paid by the final-token position to an earlier position, and the logit-difference measure $\\Delta_{\\mathrm{cofa}} = \\mathrm{logit}(t_{\\mathrm{cofa}}) - \\mathrm{logit}(t_{\\mathrm{fact}})$, which says which side a head supports. The discriminating experiment is the counterfactual-to-factual replacement in the prompt, which cleanly separates selective counterfactual suppression (hypothesis 2.2.1) from general induction/copy suppression (hypothesis 2.2.2). Domain specialization is probed by splitting prompts by answer category and by decomposing each head's OV matrix with SVD to inspect which tokens a head encodes.","core_discovery":"The paper's central claim is that the attention heads previously labeled 'factual recall heads' operate as general copy-suppression (anti-induction) heads: they inhibit the copying of tokens from earlier context positions, regardless of whether the token is a counterfactual or the true fact. The decisive experiment (Section 4.3.1) replaces each counterfactual token in the prompt with the factual token; if the heads were selective counterfactual suppressors, boosting them should leave factual output largely intact, but instead the factual prediction rate falls steadily as the boosting factor $\\alpha$ increases, falling below 70% at $\\alpha=10$ in Pythia-6.9B and below 90%–70% in GPT-2. A secondary claim is that these heads are domain-specific: their effect on the counterfactual-factual logit difference varies strongly with the answer category, and in the larger Pythia model some heads even flip which side they support across categories, while heads near zero mean influence can still be highly influential on individual samples.","pith_inferences":["For retrieval-augmented generation, this suggests that interventions tuned to suppress hallucinated or counterfactual context will also suppress genuinely retrieved facts when the same copying pathway is engaged, so faithfulness scores should be monitored when these heads are modified.","The near-zero-mean, high-variance heads found in Pythia-6.9B imply that full-dataset summary statistics can hide strong per-category behavior; future mechanistic studies should report per-category effects before assigning a head a global role.","A testable extension would check whether the general copy-suppression account transfers to instruction-tuned or encoder-decoder models, where the copied span and the final-token position are separated differently in the attention pattern."],"forward_implications":["Despite the earlier framing, strengthening these heads is not a reliable lever for increasing factual output: when the correct answer is already present in the context, boosting them suppresses it too.","The mechanism of these heads aligns with the anti-induction/copy-suppression motif previously described for L10H7 in GPT-2, so the three prior studies can be reconciled under one account.","The competition between factual and counterfactual tokens is itself reproducible across different premises and sentence structures, though the optimal boosting strength shifts.","Any intervention on these heads must be applied with knowledge of the answer domain, because a head that supports the factual side in one category can support the counterfactual side in another, especially in larger models."],"supporting_citations":[{"why":"Supplies the 'factual recall heads' (L10H7, L11H10, etc.), the CounterFact-derived 10k-prompt datasets, and the attention-modification protocol that this study reproduces and extends.","marker":"[13]"},{"why":"Describes the copy-suppression role of head L10H7 in GPT-2 and provides the anti-induction evidence that motivates hypothesis 2.2.2.","marker":"[7]"},{"why":"Demonstrates factual/counterfactual head competition in Pythia-1.4B on capital-city data and the observation that factual ratios never exceeded 20%, motivating the search for a more general mechanism.","marker":"[17]"},{"why":"Defines induction heads and the anti-induction effect that the paper uses to formulate the general copy-suppression hypothesis.","marker":"[12]"},{"why":"Provides the CounterFact base prompts and factual/counterfactual target pairs from which the paper's datasets and prompt patterns are built.","marker":"[8]"},{"why":"Introduces the SVD decomposition of OV matrices that the paper uses to test whether a head's encoded tokens match its observed domain contributions.","marker":"[9]"}],"fun_headline_variants":["Fact-boosting heads actually suppress copying","Boosting fact heads hurts when facts are in prompt","Copy suppression, not fact recall: LLM attention heads","Fact heads are domain-specific, not universal suppressors","LLM 'fact recall' heads are copy suppressors"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central conclusion hinges on the assumption that replacing a counterfactual token with a factual token in the prompt isolates the heads' copy-suppression function, rather than changing the attention pattern in some other way or destabilizing the model's behavior.","fun_headline_variants_meta":{"raw":{"variants":["Fact-boosting heads actually suppress copying","Boosting fact heads hurts when facts are in prompt","Copy suppression, not fact recall: LLM attention heads","Fact heads are domain-specific, not universal suppressors","LLM 'fact recall' heads are copy suppressors"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000352,"raw_usage":{"total_tokens":1903,"prompt_tokens":917,"completion_tokens":986,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":533,"completion_tokens_details":{"reasoning_tokens":911}},"tokens_in":533,"tokens_out":986,"duration_ms":10818,"temperature":1.0,"reasoning_tokens":911,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T17:00:47.145207+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct falsifier would be a matched-pair experiment in which the context states either a falsehood or the corresponding fact with the same surface structure, and some head raises factual output only in the falsehood case; if such a head exists, the general copy-suppression account fails. Simpler still, on the fact-replaced prompts one could measure whether the suppression is specific to tokens that match an earlier context position or instead hits all high-probability tokens, since copy suppression predicts the former.","supporting_citations":[{"cited_title":"Characterizing Mechanisms for Factual Recall in Language Models","cited_arxiv_id":null,"evidence_quote":"Demonstrates factual/counterfactual head competition in Pythia-1.4B on capital-city data and the observation that factual ratios never exceeded 20%, motivating the search for a more general mechanism."},{"cited_title":"In-context Learning and Induction Heads","cited_arxiv_id":null,"evidence_quote":"Defines induction heads and the anti-induction effect that the paper uses to formulate the general copy-suppression hypothesis."},{"cited_title":"Locating and Editing Factual Associations in GPT","cited_arxiv_id":null,"evidence_quote":"Provides the CounterFact base prompts and factual/counterfactual target pairs from which the paper's datasets and prompt patterns are built."},{"cited_title":"The singular value decompositions of transformer weight matrices are highly interpretable","cited_arxiv_id":null,"evidence_quote":"Introduces the SVD decomposition of OV matrices that the paper uses to test whether a head's encoded tokens match its observed domain contributions."}],"review_version":1}