{"id":"ec540576-f2b4-4115-8eb8-47e8d7bd7abb","arxiv_id":"2412.12841","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"GAR, a new synthetic benchmark for compositional relational reasoning, reveals poor LLM performance and yields the discovery of 'True/False' attention heads that encode truthfulness in Vicuna-33B.","lead":"Researchers built a synthetic benchmark, Generalized Associative Recall (GAR), that combines several known reasoning tasks into a unified framework and shows that large language models, including GPT-4, perform far below perfect on it. They then used mechanistic interpretability to locate attention heads in Vicuna-33B that appear to encode abstract true/false signals, and showed these heads matter for performance across model sizes and tasks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"True/False heads' 'fundamental role across models and tasks' is not causally validated; cross-model uses different heads, cross-dataset is a probe only.","rationale":"The paper makes two contributions: the GAR benchmark and the mechanistic discovery of True/False heads. The benchmark evaluation is solid, covering multiple models and including error analysis. The mechanistic story is intriguing but the jump from 'these heads matter on the specific tasks where they were found' to 'they encode abstract truth and play a fundamental role across models and tasks' is not fully supported. The reader correctly flagged the manual circuit discovery as a weakness. My concern is more specific: even if the circuit discovery is accurate for Vicuna-33B, the generalization claims rest on (a) interventions on the same distribution used for discovery, (b) a cross-model analysis that changes the head identities, and (c) a probe on natural language data that is only correlational. The inverse intervention (Table 10) drops accuracy to 60%, which is well above chance, suggesting other mechanisms can partially compensate. This is consistent with the heads being one of several parallel routes rather than a fundamental, non-redundant mechanism. The proposed concrete test would settle whether these heads are truly necessary for truthfulness outside the easy templated GAR tasks. If they are not, the conclusion should be tempered to describe the heads as important for GAR-like classification in Vicuna, not as a general truthfulness circuit. Since the reader already recommended a conditional verdict, I do not change the verdict; the concern reinforces the need for the stated conditions.","tokens_in":20321,"tokens_out":9693,"duration_ms":84714,"concrete_test":"Perform a causal knockout of Vicuna-33B's True/False heads (14.18, 14.46, 15.51, 14.0) on held-out hard GAR classification tasks (nr=2, negate) and on SNLI/GoT examples, comparing accuracy against unmodified runs and against a control set of four random heads from the same layers. If accuracy does not drop substantially on these held-out tasks, the heads are not necessary for truthfulness judgments beyond the easy discovery tasks, weakening the 'fundamental role' claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires not just that four specific attention heads in Vicuna-33B (14.18, 14.46, 15.51, 14.0) are important on the eight GAR tasks used for circuit discovery, but that they encode an abstract truth value and are fundamental across models and tasks. The cross-model evidence (Figure 5a and Table 9) covers only Vicuna 7B/13B/33B, and the named heads differ per model (e.g., Vicuna-7B True heads are 11.31, 14.23; False heads are 11.9, 9.12, 7.12). Thus the universality statement applies to a functional class, not to the specific heads named in the abstract. More importantly, the only evidence on non-GAR datasets (SNLI and GoT, Table 3) is a supervised MLP trained on the activation values of these four heads; this is a correlational probe, not a causal intervention. The paper does not show that knocking out or manipulating these heads changes truth judgments on SNLI/GoT, nor on harder GAR tasks (e.g., nr=2 with negate, where Vicuna-33B accuracy is below 40% per Figure 2b). Without a causal necessity test across diverse tasks and models, the 'fundamental role' claim remains vulnerable to the alternative that these heads are artifacts of the specific synthetic templates used for discovery, and the abstract 'truth' encoding is a byproduct of the relational-loop structure rather than a general mechanism.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Generalized Associative Recall (GAR), a synthetic benchmark that unifies associative recall, knowledge recall, and indirect object identification into a relational-loop framework with semantic and syntactic variations (negate, g2c, swapQA, swapKV). It evaluates 10 LLMs on 384 tasks and reports that even GPT-4 reaches only 71.5% average accuracy, with the compositionality gap increasing as models scale. Using attribution patching on Vicuna-33B, the authors identify reusable circuits for classification and generation tasks, including 'True' heads (14.18, 14.46) and 'False' heads (15.51, 14.0) whose activations are claimed to encode abstract truth values. Interventions on these heads improve accuracy on GAR classification tasks, and a supervised MLP trained on their activations separates true/false statements on SNLI and four GoT subsets. The paper concludes that these heads play a fundamental role in compositional relational reasoning across models and tasks.","tokens_in":20655,"tokens_out":6954,"duration_ms":67869,"significance":"If the causal role of True/False heads were established, this would be an important step toward showing that LLMs reuse composable truthfulness abstractions, and GAR could become a useful testbed for mechanistic interpretability. The paper's strengths include a public dataset and code release, a principled schema-based task generator, weak and strong intervention protocols, an inverse-intervention control, and analysis across multiple model sizes. The benchmark itself is likely to be useful even if the circuit-interpretation claims are later refined. However, the strongest conclusions about a 'fundamental role across various models and tasks' are currently supported by a narrower set of interventions than the text claims: cross-model evidence relies on newly identified heads per model, and cross-dataset evidence is a supervised probe rather than a causal manipulation.","major_comments":[{"comment":"The claim that True/False heads 'play an important role' on SNLI and GoT is supported only by a supervised MLP classifier trained on the activations of the four GAR-discovered heads. This demonstrates that these activations contain task-relevant information, but it is not a causal or mechanistic test: nothing in the experiment shows that Vicuna-33B actually uses these heads to produce its SNLI/GoT judgments, and the probe is fit to the labels of those datasets. A complementary causal test (e.g., activation patching or knockout on held-out natural-language statements) or a carefully qualified claim is needed before the conclusion that the activation patterns are 'robust across other datasets' is justified.","section":"§The efficacy of True/False Heads in Other Datasets (Table 3, Appendix E)"},{"comment":"The specific heads named in the abstract (14.18, 14.46, 15.51, 14.0) are Vicuna-33B heads; across models, Table 9 lists different heads (e.g., Vicuna-7B True heads 11.31 and 14.23). The universality claim is therefore about a functional class, not about the specific heads advertised in the abstract. Moreover, the cross-model evidence is limited to three Vicuna models on three GAR classification tasks, with no test on non-Vicuna architectures and no test on harder GAR settings such as nr=2 with negate, where Figure 2b reports Vicuna-33B accuracy below 40%. The scope of the 'across various models and tasks' claim should be restricted accordingly or supported by additional experiments.","section":"§Validating True/False Heads across Models (Table 9, Figure 5a)"},{"comment":"The main benchmark conclusions (e.g., GPT-4 at 71.5%, compositionality gap increasing with scale) rest on a lenient accuracy criterion: an answer is counted correct if its probability exceeds the average probability of the alternatives. For generation tasks with three alternatives, a model assigning only 0.34 probability to the correct answer is scored as correct. In addition, each task contains only 8/16 examples, and Figures 2, 4, and 5 report no confidence intervals or significance tests. Given these choices, the precise accuracy numbers and the strength of the 'fundamental deficiency' claim should be treated with caution; reporting error bars and, ideally, top-1/argmax accuracy as a secondary metric would strengthen the benchmark analysis.","section":"§Evaluating LLMs on GAR and Appendix C"},{"comment":"The circuit discovery relies on manual step-wise attribution patching, with the query/key/value attribution steps chosen manually, and the authors state that automatic circuit discovery methods were not used because of model size and circuit complexity. Since the identification of True/False heads and the subsequent 'fundamental role' claims depend on the completeness of this attribution, the manual procedure needs additional validation—for example, by comparing against activation patching on held-out tasks or by ablating all other heads on a sample to check for equally important alternative circuits. Without such a check, the central mechanism claim remains vulnerable to attribution error; the paper's own limitation statement in this section should be weighed explicitly in the conclusions.","section":"§Discovering and Analyzing the Circuits"}],"minor_comments":[{"comment":"The figure contains a typo: 'Jonh' should be 'John'.","section":"Figure 1"},{"comment":"The column header 'GoT Acc(%)' is based on four selected subsets, not the full Geometry of Truth dataset; the main text should say 'GoT (4 subsets)' to avoid overgeneralization.","section":"Table 3"},{"comment":"The exclusion of GoT subsets (greater-than, less-than, conjunctions, disjunctions, negations, common claim, likely) is a scope restriction that should appear in the main text next to the 'robust across other datasets' claim, because the excluded subsets are exactly those requiring logical operations.","section":"Appendix E"},{"comment":"The accuracy criterion is described only in the appendix; please define 'average probability of the alternatives' in the main text and discuss its effect on the reported accuracy numbers.","section":"Appendix C"},{"comment":"The sentence 'the existence of the loop ensures predictability' is stated as a first-principles guarantee, but no formal argument is provided; as written, it is more a design intuition than a testable claim.","section":"§Generalized Associative Recall Benchmark"},{"comment":"Please clarify how 'average activation values' are computed across multiple tokens in the hypothesis or statement, since the current definition of activation value refers to a single attending position.","section":"Figure 5(b)"}],"recommendation":"major_revision","confidential_remarks":"The manuscript reports a useful benchmark and a concrete set of interventions, and I believe the empirical core is worth publishing after the claims are calibrated. My main concern is not the benchmark or the individual interventions, but the gap between the evidence—correlational probes, manual attribution, and tiny per-task samples—and the abstract's strong 'fundamental role across various models and tasks' claim. I recommend major revision rather than rejection, since the missing causal checks appear to be within the authors' reach."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Start with the punchline: the GAR benchmark is a real contribution, and the paper's evaluation shows an interesting and reproducible performance gap in LLMs on compositional relational reasoning. The head-level finding—True/False heads in Vicuna-33B that respond to statement truth—is intriguing but the paper's central claim, that these heads play a fundamental role in CRR across various models and tasks, is not supported by the evidence actually presented.\n\nWhat's new and good. GAR integrates AR, KR, and IOI into one framework, and the task design is thoughtful: varying the number of non-same relations and adding negation gives a controlled difficulty axis. The evaluation across 10 models is broad, and the finding that the compositionality gap grows with model size is worth having. The circuit discovery on Vicuna-33B is detailed, and the strong intervention experiments on the eight discovery tasks are convincing: forcing the identified attention patterns to form improves accuracy, sometimes above Llama-3-70B. The authors also deserve credit for shipping code and data.\n\nWhere it gets soft. The leap from 'these heads matter on the tasks where they were discovered' to 'they play a fundamental role across models and tasks' is too big. The cross-model evidence uses different head IDs per model, so at best it shows a functional class exists, not that the specific heads are universal. The cross-dataset evidence on SNLI and GoT is an MLP probe trained on four heads' activations—that is a correlational probe, not a causal intervention, and the paper never shows that manipulating these heads changes truth judgments on those datasets. The manual attribution method is acknowledged as a limitation, but it matters here: without automatic circuit discovery or at least a causal test on an out-of-distribution task, the abstract 'truth encoding' could be an artifact of the synthetic template structure. The evaluation also uses a lenient accuracy metric and only 8/16 examples per task, so the performance numbers have more noise than the figures suggest.\n\nStill, the paper is coherent and honest about its own constraints. I would send it to peer review, but I would ask for tempered claims, error bars, and at least one causal intervention outside the GAR tasks. The GAR benchmark I would cite; the universal head claim, not yet. This is a good paper for a reading group—there's plenty to discuss about what counts as evidence in mechanistic interpretability.","headline":"GAR is a genuinely useful benchmark, but the True/False heads claim overreaches from a causal result on one model to a universal mechanism.","tokens_in":21146,"tokens_out":2918,"would_cite":true,"duration_ms":28811,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that two classes of higher-order relating attention heads in Vicuna-33B encode abstract truth values and are reused across models and tasks in compositional relational reasoning.","keywords":["compositional relational reasoning","mechanistic interpretability","attention heads","truthfulness","benchmark","associative recall","in-context learning","large language models"],"falsifier":"A concrete check: run the same GAR tasks on a smaller model (for example, a 1-2 billion parameter model) with an automated circuit-discovery method; if the automatically found circuit for correct answers does not include heads whose activations separate true from false GAR statements, the claim that True/False heads play a fundamental role would be falsified. Alternatively, on correctly answered GAR classification examples, if randomly permuting the attention of heads 14.18, 14.46, 15.51, and 14.0 leaves accuracy unchanged, the causal attribution fails.","tokens_in":20133,"feed_emoji":"🧠","tokens_out":9526,"duration_ms":79142,"temperature":0.7,"pith_summary":"The paper builds a synthetic benchmark, Generalized Associative Recall (GAR), that turns associative recall, knowledge recall, and indirect-object-identification-style tasks into one two-hop relational-reasoning framework, and uses it to argue that current large language models have a fundamental deficiency in compositional relational reasoning (CRR). Even GPT-4 reaches only 71.5% average accuracy on GAR, and the compositionality gap, the gap between solving subproblems and composing them, widens as models get larger. The paper then traces how Vicuna-33B actually solves these tasks with attribution patching and identifies reusable higher-order relating attention heads, including True heads and False heads, whose activations encode whether a statement is true or false. Intervention experiments suggest these heads are causally important for truthfulness judgments and transfer to natural-language inference and truth datasets beyond GAR. If the account holds, LLMs do not reason compositionally from scratch but reuse a small set of truth-encoding attention heads, and GAR supplies a controllable testbed for studying that reuse.","feed_headline":"Two attention-head classes encode true and false in LLM reasoning","feed_subtitle":"New GAR benchmark shows GPT-4 reaches only 71.5% on two-hop reasoning; the same heads steer truth judgments across models","key_machinery":"The load-bearing object is the relational loop, the interleaving of a long-range semantic relation (for example, same or kindOf) with a local syntactic relation (for example, subject-object or adjacent position) between the elements K, V, Q, and A of a statement; GAR tasks are generated by choosing two relational schemas and optionally adding negation, so every task contains such a loop. The paper's circuit analysis then centers on higher-order relating attention heads, which are relating heads whose own attention pattern is formed by composing the outputs of other relating, local, and induction heads; the True and False heads are the two classes of higher-order relating heads that compare the query-side K-V pair with the key-side pair and write a match or mismatch signal. The method that carries the discovery is step-wise attribution patching with integrated gradients, tracing logit contributions back through MLPs and attention heads, and then validating candidates with weak and strong interventions that replace or repattern the heads' attention weights.","core_discovery":"The central claim is that a handful of attention heads in Vicuna-33B implement the abstract truth value of a GAR statement, and that these heads are reused across tasks and model sizes. In classification tasks, two higher-order relating heads (14.18, 14.46) act as True heads: their query side gathers the information of the attended key-value pair and the key side gathers the candidate answer's pair, so a match writes a true signal into the residual stream. Two other heads (15.51, 14.0) act as False heads, activating when the compared pairs do not match and the statement is false. The paper shows that these heads' activation values separate true from false examples, that knocking out the wrong head class or enforcing the right attention pattern changes accuracy by 6 to 17 percentage points across Vicuna-7B/13B/33B, and that the same four heads' activations, used as features, classify SNLI and Geometry-of-Truth items near or above Vicuna-7B's accuracy. The broader discovery claim is that GAR's relational-loop motif, interleaved semantic and syntactic relations, is the pattern these circuits detect, and that LLMs' failure on hard GAR tasks reflects weak composition of lookup and retrieval relations rather than missing knowledge.","pith_inferences":["One testable extension: if True/False heads really encode abstract truth, then similar pairs of heads should be discoverable in other decoder-only families on the same GAR tasks, and their homologous activations should separate true from false statements; the paper only demonstrates universality across Vicuna sizes.","The paper's own error analysis shows most failures come from applying the retrieval relation to the wrong distractor candidate; a natural next experiment is to measure whether the False heads fire on those erroneous candidate generations, which would make the truth-value heads a monitor for reasoning errors rather than just a classification feature.","GAR's relational-loop view of composition suggests a continuum from associative recall to multi-hop reasoning; extending the benchmark to chains of three or more relations with intermediate variables would test whether the same higher-order relating heads scale to deeper compositions, something the paper does not claim.","Because weak intervention works but less than strong intervention, the model's weights contain the right attention patterns in other heads but cannot compose them into a circuit; this suggests fine-tuning the connectivity between heads, not just their individual attention, might improve compositional generalization."],"forward_implications":["GAR provides a controllable difficulty dial: accuracy drops monotonically as the number of non-same semantic relations nr goes from 0 to 2 and when negation is added, so future work can test CRR improvements at fixed difficulty.","Because the compositionality gap grows with model size on generation tasks, scaling alone is unlikely to close the CRR deficiency; targeted data or architecture changes would be needed.","Truthfulness judgments in at least one LLM family are implemented by reusable attention heads rather than by task-specific circuitry, and the same heads transfer to SNLI and Geometry-of-Truth classification.","Intervention on the identified heads can push Vicuna-33B's accuracy toward or above Llama-3-70B on several GAR task types, suggesting a concrete path for steering or improving compositional reasoning without retraining.","The circuit-reuse picture implies that higher-order induction heads play a general role in bridging demonstrations to current-token computations, extending the standard induction-head account of in-context learning."],"supporting_citations":[{"why":"Supplies the step-wise patching methodology and the indirect-object-identification circuit that GAR's negation variant builds on.","marker":"(Wang et al. 2023)"},{"why":"Provides the integrated-gradients attribution-patching technique used to locate the vital heads.","marker":"(Hanna, Pezzelle, and Belinkov 2024)"},{"why":"Defines induction heads, which GAR's higher-order induction heads generalize for in-context learning.","marker":"(Olsson et al. 2022)"},{"why":"Supplies the factual relational schemas (countries, landmarks) used in the GAR task generator.","marker":"(Hernandez et al. 2024)"},{"why":"Provides the Geometry of Truth dataset whose true/false statements test transfer of the True/False heads.","marker":"(Marks and Tegmark 2023)"},{"why":"Provides the SNLI dataset used as the second transfer benchmark for the True/False heads.","marker":"(Bowman et al. 2015)"},{"why":"Establishes the attention-sink behavior that defines when a head counts as activated.","marker":"(Xiao et al. 2024)"},{"why":"Defines the compositionality gap that GAR evaluation uses to measure reasoning deficiency.","marker":"(Press et al. 2023)"},{"why":"Shows attention-head circuit reuse across tasks and supplies the strong-intervention style used in validation.","marker":"(Merullo, Eickhoff, and Pavlick 2024a)"}],"fun_headline_variants":["Truth and falsehood encoded in two LLM attention head classes","Four heads, two classes: LLMs' abstract truth indicators","LLM relational reasoning hinges on truth and false head classes","GAR benchmark reveals LLMs' core relational reasoning deficiency","Intervening on one head class shifts LLM accuracy by up to 17 points"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's central conclusion assumes that the heads it found by manually tracing which parts of Vicuna-33B most influence the answer are really the ones doing the reasoning; no automatic circuit finder was used because existing tools could not handle the model's size and complexity, so if the tracing pointed to the wrong heads, the True/False-head claim would collapse.","fun_headline_variants_meta":{"raw":{"variants":["Truth and falsehood encoded in two LLM attention head classes","Four heads, two classes: LLMs' abstract truth indicators","LLM relational reasoning hinges on truth and false head classes","GAR benchmark reveals LLMs' core relational reasoning deficiency","Intervening on one head class shifts LLM accuracy by up to 17 points"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000854,"raw_usage":{"total_tokens":3750,"prompt_tokens":1023,"completion_tokens":2727,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":639,"completion_tokens_details":{"reasoning_tokens":2638}},"tokens_in":639,"tokens_out":2727,"duration_ms":20790,"temperature":1.0,"reasoning_tokens":2638,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T13:40:10.419019+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete check: run the same GAR tasks on a smaller model (for example, a 1-2 billion parameter model) with an automated circuit-discovery method; if the automatically found circuit for correct answers does not include heads whose activations separate true from false GAR statements, the claim that True/False heads play a fundamental role would be falsified. Alternatively, on correctly answered GAR classification examples, if randomly permuting the attention of heads 14.18, 14.46, 15.51, and 14.0 leaves accuracy unchanged, the causal attribution fails.","supporting_citations":[],"review_version":1}