{"id":"66929182-a63b-4abc-aacd-015e32473990","arxiv_id":"2504.20938","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Lorsa replaces an MHSA layer with thousands of sparsely activated rank-1 attention heads and shows these heads recover known behaviors like induction heads plus new arithmetic and thematic units.","lead":"The paper introduces Lorsa, a sparse replacement model that decomposes Transformer attention layers into thousands of interpretable low-rank attention heads, and uses it to rediscover known attention behaviors and find new arithmetic and topic heads. It matters because it offers a new tool for mechanistic interpretability of large language models, potentially making attention circuits easier to trace and understand.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"MSE reconstruction alone cannot identify atomic attention units; without a ground-truth superposition test, Lorsa's 'atomic units' may be a basis artifact.","rationale":"The reader's weakest assumption is exactly the one I would flag. The paper's Section 3.1 objective is a reconstruction loss on MHSA output, and the architecture restricts each head to a 1D OV circuit with Top-K selection. These constraints do not identify a unique decomposition: the output of any rank-d_h MHSA head is invariant under invertible transformations of the value/output basis, so infinitely many 1D-OV dictionaries are exactly output-equivalent. Learned sparsity may break ties, but it is an inductive bias, not evidence about the model's internal units. The validations in Sections 4 and 5—rediscovery of coarse head types, agreement with SAEs, and GPT-4o interpretability—are all consistent with a useful but arbitrary basis; SAE correlation is especially weak because SAEs are themselves sparse dictionary learners with the same non-identifiability. The clearest missing experiment is a ground-truth superposition recovery test; the paper cites Jermyn et al. (2024) for exactly such a construction but never applies Lorsa to it. I would keep the conditional verdict: the method is well-specified, code is released, and known-head rediscovery is a meaningful sanity check, but the central ontological claim needs the synthetic test before acceptance of 'atomic attention units' or 'superior circuit discovery.'","tokens_in":20572,"tokens_out":7471,"duration_ms":85946,"concrete_test":"Run the Jermyn et al. (2024) constructed superposition case: plant 5 ground-truth attention units into 2 MHSA heads by mixing their OV directions, sample a corpus from the synthetic model, train Lorsa exactly as in Section 3.2, and check whether the learned 1D OV circuits recover the planted units (e.g., cosine similarity > 0.9 after greedy matching, or matching behavioral signatures under path patching). In addition, retrain with 5 different random seeds and report median pairwise overlap of the learned dictionaries. If the recovered units do not match the planted units, or if the dictionaries vary substantially across seeds, the 'true atomic units' claim is unsupported and the paper's findings should be reframed as properties of one admissible sparse factorization.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing premise is that the sparse Top-K, 1D-OV decomposition learned by minimizing MSE against MHSA output (Section 3.1) recovers the true atomic attention units, not merely one admissible sparse factorization. This premise is not supported by the objective: for any MHSA head, the output is A X W_V W_O, and W_V W_O has rank at most d_h, so any invertible linear transformation of the value/output basis gives the same head output but completely different 1D OV 'units'. The same non-identifiability holds for sums of heads and shared-QK groups. The training loss fixes only the sum over Top-K heads, so infinitely many dictionaries are MSE-equivalent. The paper's validations—rediscovery of coarse head types, correlation with SAE features, and GPT-4o autointerp—are consistent with a useful but arbitrary basis; SAE agreement is especially weak because SAEs are themselves sparse dictionary learners with the same non-identifiability. Crucially, the paper cites Jermyn et al. (2024), who constructed a case with 5 ground-truth units in 2 heads, but does not run Lorsa on that or any ground-truth superposition. Appendix F.2's oblique-projection attributions are also non-unique for overcomplete dictionaries. Until identifiability is tested, claims about 'atomic attention units' and 'superior circuit discovery' are about the replacement model's factorization, not necessarily the original MHSA.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper introduces Low-Rank Sparse Attention (Lorsa), a sparse replacement model for multi-head self-attention. Each Lorsa head has a 1D OV circuit, and the output is a sum over the top-K active heads per token, with QK parameters shared across groups of heads. The model is trained to minimize squared error against the original MHSA output on Pythia-160M and Llama-3.1-8B. The paper reports that Lorsa rediscovers known attention behaviors (induction heads, successor heads, attention sinks, copy suppression), identifies a family of arithmetic-specific heads in Llama-3.1-8B, reaches comparable automated-interpretability scores to SAEs, and presents scaling-law and error analyses. The central claim is that Lorsa decomposes 'attention superposition' into atomic, individually comprehensible attention units, with better circuit-discovery properties than prior methods.","tokens_in":20867,"tokens_out":8478,"duration_ms":82741,"significance":"If the atomicity claim is established, Lorsa would be a significant new tool for mechanistic interpretability, as it is an open-sourced, tractable replacement model that turns attention output into sparse linear components and enables direct QK and OV attribution. The paper's strengths include the clarity of the architecture, the breadth of the empirical study, the open code and weights, and the concrete rediscovery of known head behaviors as an external check. However, the significance is conditional: the identification of atomic units is not currently separated from the choice of sparse factorization, and the 'superior circuit discovery' claim lacks a quantitative baseline.","major_comments":[{"comment":"The scalar least-squares objective L = E||Lorsa(x) - MHSA(x)||^2 does not identify the proposed 'atomic attention units.' For any MHSA head, W_V W_O has rank at most d_h, so replacing (W_V, W_O) by (W_V M^{-1}, M W_O) with any invertible M leaves the head output unchanged while producing a completely different set of 1D OV units. The same non-identifiability applies to sums of heads and to the parameter-shared QK groups used in the paper. Because the paper trains only against MHSA output, infinitely many sparse dictionaries are equivalent under the training loss, and the specific units found by Lorsa need not correspond to ground-truth attention units. The paper cites Jermyn et al. (2024), who construct a case with five ground-truth units in two heads, but it never runs Lorsa on that or any synthetic superposition with known units. Without such a test, or an identifiability condition proved for the Top-K sparse objective, the central claim about recovering atomic units is not supported.","section":"Section 3.1"},{"comment":"The evidence that units are spread across MHSA heads is derived from oblique projections of one overcomplete basis onto another (Appendix F.1). For overcomplete dictionaries these coefficients are non-unique, so the 90%-contribution threshold in Appendix F.2 does not establish that a given Lorsa head is a 'true' unit distributed across MHSA heads. This quantity is a property of the chosen factorization, and the claim in Section 2 that about 25% of attention units are spread across multiple heads needs a ground-truth anchor or a uniqueness guarantee.","section":"Appendix F.1/F.2"},{"comment":"The abstract claims that 'Lorsa exhibits superior circuit discovery properties, especially for features computed collectively by multiple MHSA heads,' but no quantitative circuit-discovery comparison against SAE is provided. Section 5 uses path patching to locate task-relevant Lorsa heads, and Appendix I shows one induction circuit in a 2-layer transformer, yet there is no metric for circuit quality and no SAE-based control. Either provide a controlled comparison or restate the claim as qualitative.","section":"Abstract and Section 5"},{"comment":"The claim of interpretability parity with SAE rests on t-tests at alpha = 0.05 over 100 samples, yielding 6 wins, 3 losses, and 15 ties across 24 layers. A non-significant difference is not evidence of parity; the report would be stronger with equivalence tests or effect sizes, and the figure should display the score distributions rather than only means and significance indicators.","section":"Section 4.3 and Figure 3"}],"minor_comments":[{"comment":"There are typos 'relect' and 'privleged' that should be corrected; additionally, the first piece of evidence (compound-word neurons) refers to MLP neurons, so the text should clarify how it supports polysemy of attention heads.","section":"Section 2"},{"comment":"The notation in Algorithm 1 labels w_v^h as a column vector in R^{d x 1} but refers to it as '1-Dim Value weights'; this is correct but should be explained as a map from the residual stream to a scalar.","section":"Section 3.1"},{"comment":"'with till convergence' should be 'until convergence.'","section":"Section 3.2"},{"comment":"The scaling law is fitted to the method's own training data; the text should state explicitly that this is a descriptive fit and not a predictive scaling law, since the relationship between N, K, and loss is only shown at one layer.","section":"Appendix C"},{"comment":"The autointerp prompts contain typos ('indentifying', 'Don´t') that should be fixed; also many code blocks are not typeset as code, making the prompts hard to read.","section":"Appendix D"}],"recommendation":"major_revision","confidential_remarks":"The non-identifiability issue is the main technical risk. The omission of the Jermyn et al. ground-truth test is conspicuous because the paper cites it; I would ask the authors to add such an experiment. The paper is otherwise a careful empirical study and the fit with the journal is reasonable. I recommend major revision, not rejection, because the central idea is promising and the missing identifiability validation is within the scope of an added experiment."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Hi — quick take on 2504.20938. The genuinely new thing here is the replacement architecture: Lorsa replaces an entire MHSA layer with thousands of 1D-OV heads, shared QK groups, and TopK selection, trained by MSE to match MHSA output. It ships code and weights, does careful ablations on QK dimension and binding ratio, and the rediscovery of induction, successor, and attention-sink heads is a legitimate external sanity check. The arithmetic head family is a real finding, and the dashboards make those heads easy to inspect. Credit where due: this is a useful tool and a solid empirical paper.\n\nThe soft spot is the one you flagged, and the stress-test note has it right. The central claim is that Lorsa recovers atomic attention units from attention superposition, but MSE reconstruction plus sparsity cannot identify those units. For any MHSA head, W_V W_O has rank at most d_h; changing the value/output basis gives the same head output with completely different 1D OV units. The training objective only fixes the sum over TopK heads, so infinitely many dictionaries are equally valid. The paper even cites Jermyn et al.'s ground-truth superposition example but does not run Lorsa on it. Agreement with SAE features is not strong validation either, because SAEs are sparse dictionary learners with the same non-identifiability. So the stronger claims — 'atomic units' and 'superior circuit discovery' — are about the factorization, not necessarily about the original MHSA. The path-patching results measure the replacement model, not the original.\n\nOther soft spots, in proportion: the autointerp metric is a GPT-4o proxy on partly manual examples; 'superior circuit discovery' has no quantitative benchmark against baselines; and the scaling law is a fit to the method's own data, presented as a scaling law rather than a prediction. None of these are fatal for the tool. The limitation section is honestly written about shared QK circuits and dark matter, but it does not engage with the identifiability problem.\n\nOverall: this paper deserves a serious referee. It will shape how people think about sparse attention decomposition, and the engineering is reproducible. But the authors should soften the 'atomic' language and add a ground-truth superposition test before claiming to have understood the nature of attention. If a student of yours is working on circuit tracing, this is a good starting point.","headline":"Lorsa is a genuinely useful sparse attention replacement with solid engineering and a nice rediscovery sanity check, but its 'atomic units' claim needs a ground-truth identifiability test before the strong conclusions are taken at face value.","tokens_in":21383,"tokens_out":2243,"would_cite":true,"duration_ms":25677,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that attention in Transformers is a superposition of atomic units that a low-rank sparse decomposition can disentangle into individually comprehensible heads.","keywords":["attention superposition","sparse dictionary learning","mechanistic interpretability","low-rank sparse attention","induction heads","circuit discovery","large language models","sparse autoencoders"],"falsifier":"Train a synthetic MHSA layer with, say, five ground-truth attention units superposed over two heads, then train Lorsa on its outputs and check whether the learned heads recover the implanted units one-to-one in both function and attribution. If the recovered units are mixtures, vary across seeds, or depend on hyperparameters, the claim that Lorsa finds the true atomic units would be refuted.","tokens_in":20394,"feed_emoji":"🧠","tokens_out":6616,"duration_ms":60612,"temperature":0.7,"pith_summary":"The paper proposes Low-Rank Sparse Attention (Lorsa), a replacement module that learns to reproduce a Transformer's multi-head self-attention output using thousands of sparsely activated heads, each reading and writing only a single residual-stream direction. Its central claim is that this decomposition disentangles 'attention superposition': what looks like a polysemantic or uninterpretable attention head is really several atomic attention units packed together, and Lorsa recovers those units as individually comprehensible heads. The paper reports that Lorsa heads rediscover known behaviors such as induction heads, name movers, successor heads, copy suppression, and attention sinks at finer granularity, and uncovers a family of arithmetic-specific heads in Llama-3.1-8B. Automated interpretability scores place Lorsa at parity with sparse autoencoders, and path-patching experiments suggest that Lorsa is better than attention-output SAEs at exposing circuits where several original heads cooperate. If the claim holds, attention-mediated feature interactions in language models become traceable at the level of individual atomic operations.","feed_headline":"Sparse heads split LLM attention into readable atomic units","feed_subtitle":"A replacement attention module finds cleaner induction, successor, and arithmetic heads in Pythia-160M and Llama-3.1-8B.","key_machinery":"The central object is the sparse, low-rank Lorsa head: a one-dimensional OV circuit (one input direction $w_v$, one output direction $w_o$) with a scalar activation $z$, many such heads sharing a single QK circuit, and only the top-K heads active per token. This turns the attention layer into a sparse dictionary over attention directions: the output is a sum of sparsely selected scalar coefficients times output directions. The z pattern, $z_i = A_i v = \\sum_j A_{i,j} v_j$, makes each head's activation linearly attributable to previous tokens, which is what lets a single head be interpreted as one atomic 'attend from X to Y and read Z' operation. The QK-sharing groups keep the query-key side high-dimensional because the authors find that lowering QK dimension degrades reconstruction; the sparse top-K selection over thousands of heads is what isolates individual units.","core_discovery":"On the paper's own terms, the discovery is that attention in real Transformers is organized as a sparse superposition of atomic attention units, and that a particular dictionary-learning architecture can expose them. A Lorsa head is an attention head with a one-dimensional OV circuit and a scalar activation; a group of Lorsa heads shares one high-dimensional QK circuit, and at each token only the top-K heads by activation contribute to the output. Trained by squared error against the frozen MHSA output, this model produces heads that are cleaner and finer-grained versions of previously reported MHSA behaviors, and about a quarter of the learned units are spread across multiple original MHSA heads, matching the superposition prediction. The same units also align with SAE features: arithmetic in Llama-3.1-8B decomposes into operand-specific heads such as 'op1 in 27-43' or 'op1 % 10 in [4,5,6]', and automated interpretability gives Lorsa scores comparable to those of SAE features, with better circuit-discovery performance when features are computed collectively.","pith_inferences":["A direct test of the central claim would implant synthetic attention units in a controlled MHSA whose ground truth is known and check whether Lorsa recovers exactly those units; the paper's evidence is limited to rediscovering known heads in real models.","If the units are genuine, they should be stable across training seeds and across model sizes; testing whether the same Lorsa heads appear in differently initialized or differently sized models would separate decomposition artifacts from model biology.","The 'thematic anchor' heads suggest a mechanism for topical persistence: a small set of heads broadcast a topic representation to all later tokens; patching these heads in long-document tasks would test whether they actually steer vocabulary choice.","Since Lorsa error correlates with SAE error, the next step may be a joint model that learns sparse attentional and residual-structure bases together, or a cross-layer version of Lorsa that captures interactions spanning multiple layers."],"forward_implications":["If Lorsa heads are the atomic units, interpretability analysis shifts from whole MHSA heads to finer-grained units, so uninterpretable polysemantic heads become collections of comprehensible behaviors.","Circuits in which several MHSA heads cooperate to compute one feature become traceable, because each Lorsa head carries its own scalar attribution and can be path-patched independently.","The arithmetic-head family implies that arithmetic in LLMs is implemented as a set of per-operand heuristics (range and digit checks) rather than a single algorithm, matching neuron-level findings.","Because Lorsa and SAEs trained on the same layers make correlated errors, improvements in one sparse dictionary method may transfer to the other, or the residual error may be a shared 'dark matter' not captured by any sparse linear basis.","The architectural constraints identified (QK dimension no smaller than MHSA head dimension, and QK-pair count no fewer than MHSA head count) give concrete design rules for future sparse attention replacement models."],"supporting_citations":[{"why":"Supplies the OV/QK circuit vocabulary used to define a Lorsa head's one-dimensional read/write.","marker":"[Elhage et al., 2021]"},{"why":"Proposes attention superposition and gives a constructed example with five ground-truth units over two heads, the hypothesis Lorsa targets.","marker":"[Jermyn et al., 2024]"},{"why":"Documents induction heads, one of the behaviors Lorsa must rediscover to validate its decomposition.","marker":"[Olsson et al., 2022]"},{"why":"Documents successor heads, another validation target and an instance of polysemantic heads.","marker":"[Gould et al., 2024]"},{"why":"Describes attention sinks, which Lorsa reproduces as dedicated first-token-attending heads.","marker":"[Xiao et al., 2024]"},{"why":"Establishes SAE-style sparse dictionary learning and top-activation interpretation, the methodology Lorsa borrows and compares against.","marker":"[Bricken et al., 2023]"},{"why":"Provides top-K sparsity and the L(N,K) scaling-law setup that Lorsa adopts.","marker":"[Gao et al., 2024]"},{"why":"Introduces attention-output SAEs and direct feature attribution, the baseline Lorsa must beat on circuit discovery.","marker":"[Kissane et al., 2024]"},{"why":"Supplies the automated interpretability protocol used to score Lorsa heads and SAE features.","marker":"[Bills et al., 2023]"}],"fun_headline_variants":["Low-rank sparse exposes atomic attention units","Sparse decomposition turns LLM attention into atomic parts","Disentangling attention via low-rank sparse: atomic heads","Atomic attention heads unmasked by sparse low-rank model","Sparse low-rank peeling reveals attention's atomic units"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument rests on the premise that the sparse, top-K, single-direction decomposition that best reproduces the original attention output also isolates the model's true atomic attention units; the paper never checks against an example whose true units are known.","fun_headline_variants_meta":{"raw":{"variants":["Low-rank sparse exposes atomic attention units","Sparse decomposition turns LLM attention into atomic parts","Disentangling attention via low-rank sparse: atomic heads","Atomic attention heads unmasked by sparse low-rank model","Sparse low-rank peeling reveals attention's atomic units"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001165,"raw_usage":{"total_tokens":4830,"prompt_tokens":960,"completion_tokens":3870,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":576,"completion_tokens_details":{"reasoning_tokens":3796}},"tokens_in":576,"tokens_out":3870,"duration_ms":27359,"temperature":1.0,"reasoning_tokens":3796,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:15:15.555129+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a synthetic MHSA layer with, say, five ground-truth attention units superposed over two heads, then train Lorsa on its outputs and check whether the learned heads recover the implanted units one-to-one in both function and attribution. If the recovered units are mixtures, vary across seeds, or depend on hyperparameters, the claim that Lorsa finds the true atomic units would be refuted.","supporting_citations":[{"cited_title":"Successor heads: Recurring, interpretable attention heads in the wild","cited_arxiv_id":null,"evidence_quote":"Documents successor heads, another validation target and an instance of polysemantic heads."}],"review_version":1}