{"id":"a3b3d1de-f0bd-484f-b2e1-b0f9a10501e2","arxiv_id":"2504.13756","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Task vectors for in-context learning decompose into sparse SAE features that detect and execute tasks, linked by an attention/MLP circuit in Gemma-1 2B.","lead":"This paper maps how a 2-billion-parameter language model performs in-context learning by finding sparse features that detect tasks and features that execute them, causally connected through attention and MLP layers. It also introduces a task vector cleaning algorithm that compresses task vectors into a few interpretable features, advancing SAE-based circuit analysis to larger models.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Attribution-patching IE quality is mediocre in the layers used (10–17), and no edge-level ablation verifies the claimed detection→attention/MLP→execution pathway.","rationale":"The reader's weakest assumption is exactly the attribution-patching IE approximation quality, and the paper's own Appendix E.2 and Section 4.1.3 confirm that this quality is limited, forcing exclusion of 10 of 18 layers. My stress-test sharpens this into the most load-bearing part of the central claim: the new mechanistic assertion is not merely that detection and execution features exist, but that detection features causally drive execution features through attention and MLP sublayers. That pathway is inferred from attribution-based edge IEs, and the faithfulness metric only validates node sets, not individual edges. The direct ablation in Figure 11 is suggestive but does not isolate the named sublayers, because SAE features are non-orthogonal residual-stream directions and a detection-direction ablation can reduce execution-feature activation through residual-stream projection overlap. Because the reader's CONDITIONAL verdict already accounts for this risk, and because a concrete edge-level patching check could resolve it, no change to the verdict is needed. The recommendation remains CONDITIONAL: the claims are plausible and partially supported by direct ablations, but the central pathway lacks edge-level causal verification and the SAE/IE artifacts are not yet released for independent re-running.","tokens_in":28594,"tokens_out":6741,"duration_ms":65958,"concrete_test":"Perform exact (non-attribution) activation patching on the claimed detector→executor edges: for each edge from a layer-11 output-token detection node to a layer-12 arrow-token execution node via its identified attention-output or transcoder node, zero/mean-ablate only the source node, keep all other nodes clean, and measure the change in target execution activation and task loss. Compute the correlation between these directly measured edge effects and the attribution-predicted IEs across all edges in layers 10–17. If the correlation is below 0.7 in the layers containing detector–executor edges (as Figures 20–21 suggest), the specific pathway is not supported; if it is high, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central causal claim is that task-detection features (layer 11, output tokens) drive task-execution features (layer 12, arrow tokens) through specific attention-output and transcoder nodes. The evidence for these edges is the SFC attribution-patching pipeline, but the paper's own validation (Appendix E.2, Figures 20–21) shows predicted-vs-actual IE correlations below 0.7 in most layers and below 0.5 before layer 6; the authors exclude layers 1–9 for this reason (Section 4.1.3). The remaining layers 10–17 still show only moderate correlations, and no layer- or edge-specific breakdown is reported for the exact detector–executor edges. Compounding this, Appendix E.1 states that edge pruning is performed but 'we do not perform full ablation of circuit edges'; faithfulness (Figure 7) is measured on node sets, not on the claimed edges. Figure 11 ablates detection directions and observes reduced execution activations, but because SAE features are non-orthogonal directions in the same residual stream, this ablation can lower execution SAE activations via direct projection overlap rather than via the named attention/MLP sublayers, even with attention patterns fixed. Thus the specific pathway, the paper's main new mechanistic claim, rests on approximate IEs of unverified edge-level accuracy.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper scales sparse feature circuit (SFC) discovery, previously applied to GPT-2-small, to Gemma-1 2B for the study of in-context learning (ICL) on a set of 23 tasks. The authors introduce Task Vector Cleaning (TVC), an optimization algorithm that decomposes task vectors into a small number of SAE latents, which they call task-execution features. Steering experiments show that these features improve zero-shot task performance. They then adapt SFC to the larger model, introducing token-type aggregation and a modified loss, and claim to discover task-detection features that activate on output tokens and causally drive task-execution features through attention and MLP sublayers. The paper includes multiple validation experiments: held-out steering evaluations, cross-task ablation specificity (Figure 8), and a direct ablation of detection directions (Figure 11).","tokens_in":28850,"tokens_out":3815,"duration_ms":38011,"significance":"If the central causal circuit claim holds, this would be a significant step forward: it extends automated SAE circuit discovery to a 2B-parameter model, unifies the task-vector literature with SAE interpretability, and identifies two concrete, editable feature families (detection and execution) behind ICL. The paper's strengths include careful held-out evaluation of steering, extensive cross-model sweeps for TVC (Gemma-2 2B/9B and Phi-3), qualitative max-activating examples that support feature interpretability, and a commitment to release SAEs and code. However, the claimed detection-to-execution pathway is not verified at the edge level, and the steering evidence carries a selection-on-metric circularity. These issues are fixable with additional experiments, so the work is promising but not yet conclusive as a mechanistic account.","major_comments":[{"comment":"The claim that task-execution features \"causally induce the task zero-shot\" is weakened by a selection-on-metric circularity: TVC optimizes weights to minimize the zero-shot negative log-likelihood loss (L = LNLL + λ·L1), and the same loss is then used in Section 3.2 to measure the steering effect of those features. Held-out prompt splits reduce overfitting but do not break the loop of selecting features specifically to optimize the evaluation metric. I recommend providing a non-circular validation, for example by testing features selected purely from max-activating examples or by comparing against a random-subspace baseline with matched L0.","section":"Section 3.1, Algorithm 1; Section 3.2"},{"comment":"The specific claim that task-detection features drive task-execution features through attention-output and transcoder nodes rests entirely on attribution-patching indirect effects, yet the paper's own validation (Appendix E.2, Figures 20-21) shows predicted-vs-actual IE correlations below 0.7 in most layers, and the authors exclude layers 1-9 for this reason. No layer-specific or edge-specific accuracy breakdown is given for the exact detector-to-executor edges. Moreover, Appendix E.1 states that the authors \"do not perform full ablation of circuit edges,\" and Figure 7 measures faithfulness on node sets, not edges. Therefore the main new mechanistic contribution (detection → attention/MLP → execution) is not verified at the claimed granularity. Please provide edge-level validation (e.g., path patching or ablation on a sample of top edges) or temper the causal-connectivity claim to what node-level evidence supports.","section":"Section 4.1.3, Appendix E.2, Figure 7"},{"comment":"The direct ablation experiment in Figure 11 may not isolate the claimed pathway. SAE feature directions are generally not orthogonal in the residual stream, so ablating a detector direction could reduce executor SAE activations through direct projection overlap rather than through the named attention/MLP sublayers, even when attention patterns are fixed. The paper does not report cosine similarities or a control ablation of a random equal-norm direction. I request such a control, or an orthogonalized ablation, to support the pathway-specific interpretation.","section":"Section 4.2, Figure 11"},{"comment":"The negative steering results in Figure 23 show substantially lower task specificity than the positive steering heatmaps, and the authors themselves note that \"steering experiments alone may not suffice for a comprehensive analysis of the ICL mechanism.\" This is in tension with the strength of the causal language used for positive steering in Sections 1 and 3.2. Please clarify how the positive steering results should be interpreted in light of these negative results, and specify what evidential weight each experiment family carries for the central claim.","section":"Appendix F.1, Figure 23"}],"minor_comments":[{"comment":"The pseudocode in Algorithm 1 omits the L1 coefficient λ, the early-stopping patience n, and the learning rate α; the text in Appendix D mentions some values (e.g., lr=0.15) but not all. Please report the exact hyperparameters used for each model.","section":"Algorithm 1"},{"comment":"The caption of Figure 24 says \"Full version of the heatmap in Figure 10\" but it appears to describe the unfiltered executor-feature heatmap, which should reference Figure 6.","section":"Figure 24 caption"},{"comment":"The max-activating examples are presented qualitatively without quantitative interpretability or specificity scores; consider using automated interpretability metrics or at least reporting activation statistics to support the claim that these features are task-specific.","section":"Appendix I"},{"comment":"The definition of indirect effect in Eq. (3) uses do(a=a′) but the paper does not define a′; since the implementation uses zero ablation, this should be stated explicitly to avoid ambiguity.","section":"Section 2.2, Eq. (3)"},{"comment":"The conclusion says the authors \"will also share SAE training codebase\" but does not explicitly promise release of the TVC and SFC implementation; please clarify the intended code release scope.","section":"Section 6"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"I read this paper as a serious advance in scaling SAE-based circuit analysis. The Task Vector Cleaning algorithm is genuinely new and works well: it decomposes task vectors into a sparse set of interpretable SAE features that retain most of the task vector's effect, across multiple models. The application of SFC to ICL on Gemma-1 2B is also new, and the identification of task-detection features on output tokens, causally linked to task-execution features on arrow tokens, is the main mechanistic finding.\n\nThe central claim holds up better than the stress-test suggests. The direct ablation in Figure 11 — ablating detection directions reduces execution activations with attention patterns fixed — is independent evidence for the detection-to-execution connection, not just the approximate IE pipeline. Combined with positive/negative steering and held-out splits, the evidence is meaningful.\n\nBut the soft spots are real. The most important is the attribution-patching approximation quality: correlations between predicted and actual IEs are below 0.7 in most layers and below 0.5 before layer 6, and the authors exclude 10 of 18 layers because of it. The specific detection-to-execution edges are not edge-ablated; faithfulness is measured on node sets. Figure 11 partially covers this, but the exact attention/MLP pathway still rests on IE estimates whose layer-level accuracy is only moderate. That is a genuine limitation, though not a fatal one.\n\nThe circularity concern is moderate. TVC optimizes the same steering loss used to evaluate the features. Held-out splits reduce overfitting, but the feature selection is still fitting to the metric. The independent support — negative steering, cross-task specificity, max-activating examples — helps, but it doesn't fully remove the worry.\n\nThe paper is honest about these issues, which I appreciate. It also clearly cites related work, including the overlap with Wang et al.'s 'label words' findings. Code and SAE weights are not yet released; the paper says they will be after acceptance. That's a reproducibility hold, not a fatal flaw.\n\nThis is a paper for mechanistic interpretability researchers and SAE practitioners. It deserves to go to peer review. I'd like to see the authors address the IE quality in layers 10–17 explicitly, add edge-level ablation if feasible, and release the SAE suite. With those revisions, it becomes a solid contribution.","headline":"A serious scaling of SAE-based circuit analysis to a 2B model with a new decomposition algorithm; the ICL detection-to-execution claim is real but rests on attribution-patching quality that is only moderate.","tokens_in":29450,"tokens_out":2281,"would_cite":true,"duration_ms":21374,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"In a 2-billion-parameter language model, in-context learning is carried by two causally linked families of sparse features: detectors that recognize the task from completed examples and executors that apply it at the arrow.","keywords":["in-context learning","sparse autoencoders","mechanistic interpretability","task vectors","sparse feature circuits","causal ablation","attribution patching","large language models"],"falsifier":"Recompute the indirect effects of the top-scoring task-detection features in layers 10-17 with exact activation patching, zero-ablating one node at a time, instead of gradient approximations, and compare the rankings; the paper's own Appendix E.2 reports predicted-versus-actual correlations below 0.7 in most middle layers, so if the correlation on the specific detection-to-execution edges is also below that level, the claimed causal chain is not supported by the evidence.","tokens_in":28361,"feed_emoji":"🧠","tokens_out":13106,"duration_ms":107197,"temperature":0.7,"pith_summary":"The paper tries to establish that in-context learning in a large language model is a localized, inspectable computation: a small set of sparse autoencoder features arranged in a detection-then-execution chain carries the demonstrated task through the model. It identifies two feature families: task-execution features that fire on arrow tokens and drive the production of correct answers, and task-detection features that fire on the output tokens of completed demonstrations and encode which task is being performed. Ablating a detection feature measurably suppresses its matching execution feature, showing the families are causally linked through attention and MLP layers. The paper also claims that task vectors, directions extracted from the residual stream that were already known to encode whole tasks, are well approximated by a sparse sum of these SAE latents, via a new cleaning algorithm that cuts active features by 50-80% without losing the vector's effect on task loss. A sympathetic reader would care because this turns an opaque capability into a short, editable list of feature directions, and it demonstrates that circuit-level analysis can reach a 2-billion-parameter model.","feed_headline":"In-context learning reduces to two linked feature families","feed_subtitle":"Task detectors fire on answer tokens, executors on arrows; ablating one silences the other.","key_machinery":"The central object is the sparse feature circuit: a causal graph whose nodes are SAE latents and whose edges are estimated indirect effects, built from a suite of sparse autoencoders trained on residual streams, attention outputs, and MLP inputs. Two pieces of new machinery carry the argument. Task Vector Cleaning (TVC) starts from the SAE encoding of a task vector and optimizes the reconstruction weights to minimize zero-shot task loss plus an $\\ell^1$ sparsity penalty, a procedure that keeps most of the vector's effect on loss while cutting the number of active features by 50-80% and exposing the task-execution features. The adapted sparse-feature-circuit search computes each node's indirect effect, $IE(m;a) = m(x \\mid do(a=a')) - m(x)$, via attribution patching in the SAE basis, aggregates effects per token position type (prompt, input, arrow, output, newline), and restricts itself to layers 10-17 because indirect-effect approximations before layer 10 are too inaccurate. The object it produces is a graph in which detection features sit upstream of execution features, passing through attention-output SAE nodes and transcoder (MLP) nodes, with faithfulness of about 0.6 reached at roughly 500 nodes.","core_discovery":"On the paper's own terms, the discovery is that in-context learning in Gemma-1 2B is mediated by two causally connected families of SAE features, ordered as detect-then-execute. Task-detection features activate almost exclusively on the output tokens of earlier demonstrations (about 97% of their activation mass in the paper's measurements), and steering with them induces the corresponding task zero-shot; task-execution features activate overwhelmingly on arrow tokens (about 90% of their mass), and steering with them also induces the task. When the strongest detection feature for a task is ablated, the activation of the matching execution feature drops substantially, establishing the causal chain through attention-output and transcoder (MLP) nodes. The paper further claims that the task vectors of prior work are well approximated by a sparse sum of SAE latents, including these task-execution features, and that the extracted circuits are task-specific, since ablating the top nodes of one task's circuit degrades other tasks' faithfulness only in closely related cases.","pith_inferences":["The two-family dichotomy is shown for discrete token-to-token tasks; for graded ICL settings such as in-context regression or classification with unseen labels, the analog of a detection feature may be a continuous axis rather than a discrete SAE latent, and looking for one would test whether detect-then-execute is an architectural constant.","The paper notes that translation-to-English tasks share a generic executor while language identity lives in a separate feature; taken further, this suggests ICL circuits factor into a reusable task kernel plus a per-instance modifier, which predicts that perturbing the modifier alone should switch tasks within a family.","TVC is validated on other architectures, but the full detection-to-execution circuit is only recovered on Gemma-1 2B; re-running the adapted circuit search on a second architecture is the direct test of whether the discovered structure is a general transformer property rather than a quirk of one model."],"forward_implications":["If the two-family circuit is right, then a handful of SAE feature directions, not diffuse activity, carries in-context learning, so switching a task on or off reduces to steering or ablating those few directions without retraining.","Task vectors become sparse, interpretable objects rather than opaque residual-stream directions: a few execution features, not the usual dozen or more noisy latents, preserve most of a task vector's effect on loss.","Because detection features sit upstream of execution features, a model should fail to perform a task it cannot detect, so audits and interventions on ICL behavior should target the detection stage first.","The pipeline ran end-to-end on a 2-billion-parameter model with 23 tasks, which makes full circuit analysis, rather than single-feature inspection, accessible for models of this scale."],"supporting_citations":[{"why":"Supplies the sparse feature circuit method being scaled: indirect-effect thresholds, faithfulness metric, and circuit construction.","marker":"Marks et al. (2024)"},{"why":"Provides the task-vector setting, the steering protocol, and the suite of 23 token-to-token tasks the analysis is built on.","marker":"Todd et al. (2024)"},{"why":"Independent source of the task-vector concept that Task Vector Cleaning decomposes into execution features.","marker":"Hendel et al. (2023)"},{"why":"Attribution patching, the gradient-based approximation used to estimate every indirect effect in the circuit search.","marker":"Syed et al. (2023)"},{"why":"The Gated SAE architecture on which all residual-stream, attention-output, and transcoder dictionaries are trained.","marker":"Rajamanoharan et al. (2024a)"},{"why":"Defines the do-calculus indirect-effect quantity that the circuit metric is based on.","marker":"Pearl (2009)"},{"why":"Closest prior finding of anchor-like tokens in ICL circuits; the paper compares its task-detection features against this work.","marker":"Wang et al. (2023)"}],"fun_headline_variants":["ICL circuit: detect on answers, execute on arrows","Sparse SAE latents causally drive in-context learning","Two feature families govern ICL: detect then execute","Ablating task detectors silences executors in Gemma-2B","Scaled circuit finding reveals ICL's two-step mechanism"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the gradient-based estimates of each feature's causal effect are accurate enough to pick out the true circuit edges, since the paper's own checks find these estimates only moderately correlated with actual effects in the middle layers and poor before layer 6, and the same inaccuracy in layers 10-17 would make the detection-to-execution links artifacts.","fun_headline_variants_meta":{"raw":{"variants":["ICL circuit: detect on answers, execute on arrows","Sparse SAE latents causally drive in-context learning","Two feature families govern ICL: detect then execute","Ablating task detectors silences executors in Gemma-2B","Scaled circuit finding reveals ICL's two-step mechanism"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000262,"raw_usage":{"total_tokens":1605,"prompt_tokens":963,"completion_tokens":642,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":579,"completion_tokens_details":{"reasoning_tokens":556}},"tokens_in":579,"tokens_out":642,"duration_ms":5630,"temperature":1.0,"reasoning_tokens":556,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T12:00:50.310136+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute the indirect effects of the top-scoring task-detection features in layers 10-17 with exact activation patching, zero-ablating one node at a time, instead of gradient approximations, and compare the rankings; the paper's own Appendix E.2 reports predicted-versus-actual correlations below 0.7 in most middle layers, so if the correlation on the specific detection-to-execution edges is also below that level, the claimed causal chain is not supported by the evidence.","supporting_citations":[{"cited_title":"L., Sen Sharma, A., et al","cited_arxiv_id":null,"evidence_quote":"Provides the task-vector setting, the steering protocol, and the suite of 23 token-to-token tasks the analysis is built on."}],"review_version":1}