{"id":"f8c912b0-6ef8-4b8f-b3ba-19b72172e5a2","arxiv_id":"2608.03913","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Sparse Weight Decomposition reparameterizes transformer weight matrices into sparse factors whose bottleneck units support efficient circuit extraction with less data and sparser circuits than learned sparse baselines.","lead":"A new method, Sparse Weight Decomposition, splits each dense weight matrix of a pretrained AI model into two sparse parts, creating small 'bottleneck' units that can be individually switched off. This lets researchers find the few units responsible for a behavior using far less data than current methods, on models up to 27 billion parameters.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Task-specific fidelity of the approximate factorization AB is the load-bearing unverified step; matched average CE does not control whether SWD circuits describe the dense model.","rationale":"The reader's weakest_assumption precisely identifies the approximate-replacement faithfulness issue, and my reading agrees that this is the most load-bearing point. The paper's own limitations acknowledge the gap, which strengthens the credibility of the report, but the acknowledgment does not remove the need for a concrete test. The central empirical comparison controls average replacement CE, yet circuit sufficiency and necessity are computed relative to each method's replacement, so task-specific drift in AB could invalidate the claim that SWD circuits describe the original dense model. I do not see a stronger internal inconsistency: the experiments are carefully controlled within their chosen protocol, and the paper honestly scopes full-model and scaling results. The appropriate verdict remains CONDITIONAL because the conditional should explicitly require a task-fidelity check before the central claim is accepted at face value. My concrete test would settle whether the concern lands: if SWD's task-margin deviation from the dense model is no worse than the baselines, the matched-CE protocol is adequate and the concern is resolved; if it is worse, the circuit-cost comparisons need to be re-evaluated on the dense model.","tokens_in":35981,"tokens_out":4627,"duration_ms":48884,"concrete_test":"For each fixed replacement used in the Section 3.3 comparison, compute the dense-model task margin Q_dense and the replacement task margin Q_rep on the held-out circuit test split for all four tasks, along with the 95th-percentile absolute per-example logit-margin deviation between dense and replacement. If SWD's |Q_dense - Q_rep| or its per-example margin deviation is substantially larger than the Transcoder and VPD-Recon-CI baselines at the same matched CE, then the CE-matching protocol does not control the circuit comparison and SWD's active-edge advantage may be an artifact of AB rather than a property of the dense model.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that SWD exposes task circuits from dense pretrained projections with fewer active edges at matched replacement fidelity. However, every circuit intervention in Sections 3.3-3.6 is performed on the approximate replacement AB, never on the original dense weight W. Section 5 concedes that matched cross-entropy and reconstruction control average drift but cannot guarantee agreement with the original dense model on every input. The matching rule in Section 3.2 only requires pre-pruning CE differences of at most 0.001, and circuit scores are computed relative to each method's own unpruned replacement. If AB's per-example error is concentrated on the task-relevant inputs (public circuit datasets, not the FineWeb-Edu calibration distribution), then SWD's low-cost sufficiency and necessity curves could reflect properties of the factorization rather than of the dense model. This is not a completeness caveat: the abstract's claim that SWD 'reparameterizes' pretrained linear projections asserts that the factorization is the object of analysis. The decisive unverified step is that task-margin behavior of AB matches W closely enough that selecting or ablating bottleneck units in AB predicts what would happen in W. No experiment in the paper checks this directly.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Sparse Weight Decomposition (SWD), which factorizes a dense weight matrix as W ≈ A B with sparse factors A and B, and treats the shared intermediate coordinates as addressable bottleneck units for circuit extraction. The authors evaluate replacement fidelity (CE delta, KL, activation relative MSE) and circuit cost-quality tradeoffs (sufficiency and necessity as functions of selected units and active edges) on GPT-2 Small, Qwen2.5 0.5B-3B, and Qwen3.5-27B, comparing against Transcoder, VPD, sparse pretraining, and exact dense SVD/Random-B controls. They report that SWD reaches matched replacement CE with less than 1% of the data used by trained baselines, reaches circuit targets with fewer active edges, remains competitive after fixed-support fine-tuning for full-model replacement on GPT-2 Small, and has a zero-data variant.","tokens_in":36108,"tokens_out":10484,"duration_ms":99143,"significance":"The paper is a carefully executed empirical study of a practical idea: reusing double sparse factorization as a cheap parameter-space source of circuit units. The experimental protocol has real strengths: pre-pruning CE is matched within 0.001, exact dense controls (full-rank SVD and Random-B) are used to separate the effect of sparse edges from factorization alone, zero-ablation robustness is checked, non-matched baselines are reported honestly, and the code and model links are provided. If the central claim holds, SWD would substantially lower the cost of obtaining causally testable units from dense checkpoints, and the method's applicability to attention and MLP projections is broader than activation-space replacements. The main risk is that the causal claims are made about the dense model while all interventions are performed on the approximate factorization, and the active-edge metric is partly built into the comparison with activation-space baselines.","major_comments":[{"comment":"Section 5 states that matched cross-entropy and reconstruction control average drift but cannot guarantee agreement with the original dense model on every input; yet all circuit interventions in Sections 3.3-3.6 are performed on the approximate replacement AB, never on the dense weight W. Because W = AB + R, the same top-k keep/ablate protocols can be applied to the dense model by holding the residual term R fixed and intervening only on the AB term's unit activations. I request a direct experiment comparing task margins and sufficiency/necessity curves of SWD circuits in AB versus in this dense-model implementation, evaluated on the circuit train/test splits. Without such a check, the abstract's claim that SWD 'reparameterizes' pretrained projections and yields circuits of the dense model is not established; the circuits may be properties of the factorization.","section":"Sections 3.2-3.6, Section 5"},{"comment":"The matching rule in Section 3.2 only constrains pre-pruning CE delta (at most 0.001) on FineWeb-Edu, while Suff and NecDrop are defined relative to each method's own unpruned replacement Munpruned. This does not imply equal task margins Qτ(Munpruned; D) on the circuit splits; for example, Table 8 lists Qwen3.5-27B checkpoints with CE deltas of 0.001847 (VPD-Recon-CI) and 0.001121 (TC48k), which are outside the stated tolerance but are still used in circuit comparisons as 'references.' If Q_unpruned differs across methods, the same relative sufficiency or necessity drop does not correspond to the same absolute task behavior. Please report absolute held-out task margins for every method and either match Q_unpruned, normalize appropriately, or explicitly exclude outside-tolerance baselines from the 'matched-fidelity' language.","section":"Section 3.2, Table 8"},{"comment":"The active-edge cost C_edge(S) counts nonzero read/write weights of selected units, but Transcoder and VPD units have dense parameter-side read/write vectors by construction: Table 1 classifies them as activation-sparse or input-dependent-selection methods, not weight-sparse methods. Comparing C_edge between SWD and these baselines therefore partly measures the representation choice rather than circuit quality; the exact dense controls in Section 3.5 confirm that dense factorizations need more edges, but they do not make Transcoder edges commensurable with SWD edges. The central abstract claim of 'fewer active read/write edges... than learned sparse baselines' should be supported by a normalized cost meaningful across parameter-space and activation-space methods, or the selected-unit axis should be made primary for cross-paradigm comparisons.","section":"Section 3.2, Section 3.5, Table 1"}],"minor_comments":[{"comment":"The footnote 'This greater data use does not consistently reduce circuit cost' appears to say the opposite of what is meant; it should say 'lower data use' or 'data advantage.'","section":"Table 1 footnote"},{"comment":"The sentence 'zero-data SWD remains closer to W throughout the all sparsity levels' contains a typo and should read 'at all sparsity levels.'","section":"Section 3.6"},{"comment":"TC48k and VPD-Recon-CI are outside the 0.001 CE-matching tolerance on Qwen3.5-27B; the main-text Figure 6 caption and legend should mark them as non-matched references rather than leaving this information only in the appendix.","section":"Appendix C.5, Figure 6"},{"comment":"The main-text takeaway states a broad SWD advantage, but Appendix E reports mixed results on the full-MLP surface, with the standard MLP Transcoder competitive in several selected-unit and sufficiency comparisons; the main-text claims should be qualified to the single-matrix surface.","section":"Appendix E, Section 3.3 takeaway"},{"comment":"The targeted editing study uses a factorization fitted on WikiText-2 calibration data, whereas the main circuit protocol uses FineWeb-Edu; please specify whether unit c205 is drawn from the same factorization as the main experiments and whether the different calibration distribution affects unit selection.","section":"Section 3.7.2, Appendix I"}],"recommendation":"major_revision","confidential_remarks":"The paper is methodologically careful and the empirical claims are plausible, but the main revision should focus on the dense-model transfer question; the concern is addressable within the paper's scope, so I do not recommend rejection. The active-edge comparison with Transcoder and VPD is partially an artifact of representation, and the authors should either reframe the claim or add normalized cost metrics. If the task-specific fidelity and dense-transfer tests are added, the paper could be a solid contribution to the mechanistic-interpretability literature."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis is a solid, genuinely useful paper. It takes Double Sparse Factorization, previously used for compression, and shows it provides addressable bottleneck units for circuit extraction. The main result—that at matched pre-pruning cross-entropy, SWD reaches the same sufficiency and necessity targets with fewer active edges than Transcoder and VPD, using less than 1% of the training data—looks credible and is supported by careful controls: exact SVD and Random-B dense factorizations, zero-ablation robustness, and a candid limitations section. The fixed-support fine-tuning extension to full-model replacement is a nice addition.\n\nThe stress-test concern is real: every circuit intervention is performed on the approximate replacement AB, not on the dense weight W. Matching average CE on FineWeb-Edu does not guarantee that AB is faithful on the circuit-task inputs, and the paper never directly checks this. If AB's error is concentrated there, the extracted circuits describe the factorization, not the dense model. The authors acknowledge this in Section 5, but the abstract's claim that SWD 'reparameterizes' the dense projection is too strong. This is a moderate limitation, not a fatal one—the comparison against trained baselines is fair because all methods are replacements—but it should be fixed by testing per-example agreement on task data.\n\nOther soft spots are minor. Most circuit figures lack error bars, so the stability of the active-edge advantage is unknown. Two Qwen3.5-27B baselines (TC48k, VPD-Recon-CI) exceed the declared 0.001 CE-matching tolerance; the authors flag them, but this loosens the 'matched-fidelity' claim. Full-model replacement is only demonstrated on GPT-2 Small, which is stated.\n\nThe paper deserves a serious referee. The exciting part is the low data cost and sparse circuits; the weak part is the unverified bridge from factorization to dense-model behavior. With error bars and a task-level fidelity check, this would be a strong contribution to mechanistic interpretability.","headline":"Honest, well-controlled empirical paper that repurposes DSF for circuit extraction; the main caveat is that all circuit conclusions rest on an approximate factorization whose task-level fidelity is never directly tested.","tokens_in":36767,"tokens_out":3719,"would_cite":true,"duration_ms":33807,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Sparse Weight Decomposition rewrites a dense transformer's weight matrices as products of two sparse factors, turning each shared coordinate into an individually addressable, ablatable circuit unit without training any auxiliary…","keywords":["sparse weight decomposition","circuit extraction","mechanistic interpretability","bottleneck units","double sparse factorization","circuit sufficiency and necessity","weight sparsity","large language models"],"falsifier":"Replay a circuit extracted from $AB$ on the original dense model by materializing each selected unit's rank-one update $\\alpha A_{:,i}B_{i,:}$ directly in $W$: if a held-out task input changes the dense model's task margin against the direction predicted from $AB$ while average cross-entropy is matched, the extracted circuit is an artifact of the factorization rather than a property of the pretrained model.","tokens_in":35686,"feed_emoji":"🧠","tokens_out":10766,"duration_ms":81255,"temperature":0.7,"pith_summary":"SWD factorizes each dense weight matrix $W$ of a pretrained transformer into two sparse factors $A$ and $B$ with $W \\approx AB$, so every shared coordinate becomes a bottleneck unit: a rank-one path that reads a few inputs through a sparse column of $A$ and writes a few outputs through a sparse row of $B$. The paper's central claim is that these units can be scored, selected, and ablated exactly like learned sparse features — with no auxiliary replacement network trained — and that at matched replacement fidelity they reach the same circuit sufficiency and necessity targets with fewer active read/write edges than Transcoder, VPD, and sparse-pretrained baselines, while using less than 1% of the data those baselines consume. This matters because dense transformers hide task behavior in distributed, polysemantic weights, and the conventional route to circuit analysis — training sparse dictionaries or replacement modules — is costly and can end up analyzing a representation that has drifted from the original model. SWD instead puts the circuit surface directly in the pretrained weights and even offers a zero-data variant that needs no calibration activations, so the claim, if right, makes mechanistic circuit testing on dense checkpoints much cheaper.","feed_headline":"Sparse weight factors expose LLM circuits at 1% of the data","feed_subtitle":"Rewriting dense weights as sparse read-write paths makes circuit units addressable with no replacement network.","key_machinery":"The load-bearing object is the sparse read–write bottleneck unit: the rank-one path $A_{:,i}B_{i,:}$ formed by one column of the sparse factor $A$ and one row of the sparse factor $B$, so unit $i$ reads inputs only through the support of $A_{:,i}$ and writes outputs only through the support of $B_{i,:}$. The two factors are fit under a total nonzero budget $K$ by alternating $\\ell_0$-constrained regressions — the Double Sparse Factorization heuristic with ADMM inner updates, optionally weighted by the calibration Gram matrix $G = \\mathbb{E}[h^\\top h]$ — so the unit surface lives directly in the pretrained weights rather than in learned activation features. This machinery does three jobs at once: it partitions a dense map into independently ablatable additive paths, it makes circuit cost countable (each selected unit's active edges are simply the nonzeros of its read and write vectors), and it supplies the control that isolates sparsity from factorization, since dense SVD or random-orthogonal reparameterizations of the same matrix lack the edge savings.","core_discovery":"The central claim is that a dense pretrained linear projection can be reparameterized as $W \\approx AB$ with both factors sparse, and that the intermediate coordinates of this factorization form a usable causal unit surface for circuit extraction. Each bottleneck unit $i$ is the rank-one path $A_{:,i}B_{i,:}$ with sparse read and write edges, so its scalar activation $z_i = h A_{:,i}$ can be scored by positive first-order task-margin attribution, and keeping or mean-ablating it changes only that unit's term in the sum $\\sum_i z_i B_{i,:}$. Because the factors are fit directly to the checkpoint — minimizing the activation-weighted reconstruction error $\\mathbb{E}\\|hW - hAB\\|_2^2$ subject to an $\\ell_0$ budget on factor entries — no separate replacement network is trained. The paper reports that at matched pre-pruning cross-entropy, SWD reaches the same held-out sufficiency and necessity thresholds with fewer active read/write edges and selected units than Transcoder and VPD baselines on GPT-2, Qwen2.5, and Qwen3.5-27B; that fixed-support fine-tuning extends the approach to replacing all 48 attention and MLP matrices of GPT-2 Small at competitive loss; that exact dense factorizations (SVD, random-orthogonal) do not reproduce the edge savings, isolating sparse read/write structure as the cause; and that a zero-data variant with $G = I$ still produces useful task circuits.","pith_inferences":["If $AB$ is faithful on the inputs where the task actually lives, SWD units become a cheap circuit-hypothesis generator: circuits found on $AB$ could be re-verified on the dense model by materializing the same sparse rank-one updates in $W$, a two-stage validation the paper does not run.","The sparse-connectivity result suggests post-hoc factorization may serve as a fast proxy for sparse pretraining's interpretability benefits, letting researchers test circuit hypotheses on existing checkpoints before committing to training a weight-sparse model.","A testable extension is trajectory analysis: run the zero-data factorization at successive checkpoints of a single training run and ask whether the same bottleneck units appear, consolidate, or vanish as task skill emerges; the paper leaves this explicitly to future work.","Because matched cross-entropy only controls average drift, a natural next experiment is cross-method circuit agreement on the same task — whether SWD and Transcoder select the same computational paths when both are faithful — which would separate circuits that reflect the model from circuits that reflect the representation."],"forward_implications":["Circuit analysis of dense transformers no longer requires training sparse autoencoders, Transcoders, or replacement modules; the same scoring–selection–ablation workflow applies directly to factorized weights, using under 1% of the data.","Sparse read/write connectivity, not factorization alone, is what buys compact circuits: exact SVD and random-orthogonal reparameterizations of the same matrix need more active edges to hit the same sufficiency and necessity targets.","The method scales: single-matrix replacement reaches matched fidelity on Qwen3.5-27B, and fixed-support fine-tuning extends SWD to replacing all 48 attention and MLP matrices of GPT-2 Small at loss competitive with sparse pretraining.","The zero-data variant (Frobenius objective, no calibration activations) still yields units that support task circuits, enabling per-step mechanistic analysis of circuit structure during training.","SWD bottleneck units support targeted edits and attention diagnostics: a single unit's read direction can implement a more local weight edit than rank-4 LoRA, and an ablated query unit can sharply redirect attention in a head where reconstruction is faithful."],"supporting_citations":[{"why":"Supplies the Double Sparse Factorization solver and the fixed-budget two-sparse-matrix optimization that SWD adopts for fitting A and B.","marker":"Boza & Macko, 2025"},{"why":"Weight-sparse transformers: motivates the sparse-connectivity hypothesis and provides the full-model replacement target that SWD-FT must match.","marker":"Gao et al., 2025"},{"why":"Transcoders: the learned activation-space replacement baseline whose fidelity and circuit cost SWD is compared against.","marker":"Dunefsky et al., 2024"},{"why":"NaNA: the exact-SVD decomposition that motivates rank-one weight units and serves as the dense reparameterization control.","marker":"Xue & Andrzejak, 2026"},{"why":"ACDC: the automated circuit-discovery workflow whose scoring–selection–ablation protocol SWD plugs into.","marker":"Conmy et al., 2023"},{"why":"Attribution patching: supplies the positive first-order task-margin score used to rank bottleneck units.","marker":"Syed et al., 2023"},{"why":"IOI circuit analysis: establishes the mean-ablation convention used for the sufficiency and necessity interventions.","marker":"Wang et al., 2022"},{"why":"Edge pruning: contributes the task datasets, the held-out evaluation setup, and the active-edge cost accounting.","marker":"Bhaskar et al., 2024"}],"fun_headline_variants":["Sparse weight factors expose LLM circuits at 1% data","Sparse decomposition yields addressable LLM circuits","No retraining: sparse factors uncover LLM circuits","Dense weights become sparse circuits with 1% data"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Circuit conclusions are drawn from interventions on the approximate product $AB$, not on the original dense weights $W$, and matching average cross-entropy does not guarantee that $AB$ agrees with $W$ on the particular inputs where the task behavior lives.","fun_headline_variants_meta":{"raw":{"variants":["Sparse weight factors expose LLM circuits at 1% data","Sparse decomposition yields addressable LLM circuits","No retraining: sparse factors uncover LLM circuits","Dense weights become sparse circuits with 1% data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000758,"raw_usage":{"total_tokens":3443,"prompt_tokens":1098,"completion_tokens":2345,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":714,"completion_tokens_details":{"reasoning_tokens":2279}},"tokens_in":714,"tokens_out":2345,"duration_ms":15048,"temperature":1.0,"reasoning_tokens":2279,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T14:45:29.150791+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Replay a circuit extracted from $AB$ on the original dense model by materializing each selected unit's rank-one update $\\alpha A_{:,i}B_{i,:}$ directly in $W$: if a held-out task input changes the dense model's task margin against the direction predicted from $AB$ while average cross-entropy is matched, the extracted circuit is an artifact of the factorization rather than a property of the pretrained model.","supporting_citations":[{"cited_title":"These task examples are separate from the FineWeb- Edu data above and are not counted as replacement fitting or training data","cited_arxiv_id":null,"evidence_quote":"ACDC: the automated circuit-discovery workflow whose scoring–selection–ablation protocol SWD plugs into."}],"review_version":2}