{"id":"caa9cc37-43f8-4a81-a753-3f8e13a54e62","arxiv_id":"2411.08790","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Direct SAE decompositions of steering vectors are dominated by encoder bias and cannot capture negative feature projections, making them unreliable for interpretation.","lead":"Steering vectors control how large language models behave, but a popular tool for reading their inner meaning, sparse autoencoders, misreads them. This paper pinpoints two reasons: steering vectors sit outside the data SAEs are trained on, and SAEs cannot represent negative contributions, so their decompositions are misleading.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Negative-coefficient prevalence rests on a prompt-difference proxy that is not validated against direct projections of the steering vector onto SAE features; the second headline reason is empirically under-supported.","rationale":"The reader's CONDITIONAL verdict with high confidence is appropriate. The OOD claim is robust: the zero-vector decomposition control (Table 1) and the norm distribution (Figure 1) directly demonstrate encoder-bias dominance, and the default-component story explains why scaling alone cannot fix it. The negative-coefficient claim is the weaker link. Appendix C.2 explicitly says the prompt-difference method is not ground truth, but the method is then used to quantify prevalence. The JumpReLU threshold and the overcomplete SAE dictionary mean activation differences on prompt pairs are not guaranteed to have the same sign as projections of v; this is a concrete, testable gap. The proposed test settles it by comparing the proxy to direct projections and to an unconstrained sparse decomposition. If the proxy fails, the paper's second reason remains mathematically plausible but empirically unquantified, which supports keeping CONDITIONAL rather than full ACCEPT. If the proxy passes, the paper's case is strengthened. There is no reason to reject: the core phenomenon—direct SAE decomposition of steering vectors is unreliable—is well demonstrated by the zero-vector control alone.","tokens_in":9240,"tokens_out":12258,"duration_ms":257946,"concrete_test":"Recompute the Appendix C.2 analysis at layer 14 for the corrigibility and refusal steering vectors: (i) form v from the CAA mean difference; (ii) for the top-100 features by |mean(f_i(a_+)-f_i(a_-))|, compute the direct projection <v, d_i/||d_i||> and also the signed coefficient of v under an unconstrained sparse solve (e.g. gradient pursuit) in the SAE dictionary; (iii) measure sign agreement between the proxy and each direct estimate. If agreement is below ~80%, Table 3's prevalence claim should be re-quantified; if agreement is high, the proxy concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.2 and Appendix C.2 estimate how common negative coefficients are by comparing SAE activations on positive vs negative contrastive prompts, then counting features whose mean activation is higher on negative prompts. This is not a decomposition of the steering vector v itself. Because the JumpReLU encoder is nonlinear, sign(f_i(a_+) - f_i(a_-)) need not equal sign(<d_i, v>) or sign of a sparse coefficient of v in the SAE dictionary; thresholding can flip signs whenever one of the paired activations is suppressed. The authors explicitly concede the method 'does not provide a ground truth decomposition,' yet the 51.2% figure and Table 3's ~50/100 negative features are presented as evidence that meaningful negative projections are widespread. If the proxy is biased by JumpReLU thresholds, the prevalence of negative coefficients could be substantially over- or underestimated. This matters because the paper's second reason—SAEs cannot accommodate negative coefficients—depends on negative coefficients being a real, prevalent feature of steering vectors. The OOD argument (Section 3.1) is independently well supported by the zero-vector control, so the central claim survives, but the negative-coefficient reason is currently backed by an unvalidated proxy rather than by direct measurement.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper asks whether sparse autoencoders (SAEs) can be used to directly decompose steering vectors, and answers that they cannot be trusted for this purpose. Using a corrigibility steering vector extracted from Gemma 2 2B via contrastive activation addition and a Gemma Scope layer-14 SAE, the authors identify two reasons: (1) steering vectors are out-of-distribution for SAEs, so the encoder bias dominates the decomposition; and (2) steering vectors can have meaningful negative projections in feature directions, which non-negative SAE encoders cannot represent and which can also cause spurious positive activations through negative cosine similarity between features. The evidence includes a zero-vector control showing nearly identical top features, per-behavior top-feature tables, and comparisons of SAE activations on positive versus negative prompts. The paper concludes that direct SAE decomposition should not be used and sketches a future approach that decomposes contrastive activations before subtraction.","tokens_in":9420,"tokens_out":9120,"duration_ms":83247,"significance":"If the paper's diagnostic claims are correct, they provide a useful caveat for the interpretability community and help explain why prior attempts to interpret steering vectors with SAEs produced mixed results. The zero-vector control and the near-identical top-feature tables in Tables 1 and 2 are a clean, convincing demonstration that the direct encoding of a small-norm steering vector is dominated by the SAE encoder bias. The use of external, openly available SAEs, datasets, and code supports reproducibility. The logical argument that SAEs cannot output negative coefficients is sound, but the empirical claim that negative projections are widespread (51.2% in Section 3.2 and Table 3) currently rests on an acknowledged proxy that has not been validated against direct measurements. If that proxy is biased by JumpReLU thresholding, the prevalence estimate could be off, although the qualitative message that direct SAE decomposition is unreliable is independently supported by the out-of-distribution result.","major_comments":[{"comment":"The headline statistic that 51.2% of active features are 'negative' is computed by comparing mean SAE activations on positive versus negative prompts, not by decomposing the steering vector itself. Because the JumpReLU encoder is nonlinear with per-feature thresholds, sign(f_i(a_+) - f_i(a_-)) need not equal sign(<d_i, v>) or the sign of a sparse reconstruction coefficient of v; thresholding can suppress one of the paired activations and flip the apparent sign. The paper concedes that this method 'does not provide a ground truth decomposition,' yet it is the only quantitative support for the claim that meaningful negative projections are common. I would like to see the proxy validated against direct projections <d_i, v> or against coefficients obtained by a linear sparse approximation of v in the SAE dictionary, together with a robustness check that the sign pattern is stable under threshold variations. Without this, the paper's second headline reason is under-supported.","section":"Section 3.2 / Appendix C.2, Table 3"},{"comment":"The zero-vector control convincingly shows that the encoder bias dominates for the original, small-norm steering vector. However, the additional claim that scaling the steering vector does not fix the out-of-distribution problem is supported by the default-component argument but not quantified. A direct comparison of f(alpha*v) and f(0) over a range of alpha, including values chosen to match the median activation norm, would make the claim quantitative and would rule out the alternative that a sufficiently large alpha makes W_enc v dominate b_enc. As written, Figure 2 is suggestive but the scaling claim is not demonstrated as rigorously as the zero-vector control.","section":"Section 3.1, Figure 2"}],"minor_comments":[{"comment":"The right panel would be clearer if it stated that the 6.12 activation comes from direct decomposition of the steering vector rather than from the prompt-conditioned activations; as written, 'rarely activates' and 'SAE activation 6.12' appear contradictory.","section":"Section 3.2, Figure 3"},{"comment":"The text says 'steeringcorrigibility' in §3; this should be 'steering corrigibility'.","section":"Section 1 / Section 3"},{"comment":"The notation |X| is standard but should be defined as the number of contrastive prompt pairs; also, the subscript L is typeset inconsistently as a_L and aL across the paper.","section":"Equation (1)"},{"comment":"The top-100 cutoff is admittedly arbitrary; reporting the number of negative features at several cutoffs (e.g., top 50 and top 200) would make the 'around 50' claim more robust.","section":"Appendix C.2"},{"comment":"The sentence 'our arguments and findings in this paper are independent of whether ReLU or JumpReLU is used' is too strong in light of the threshold-sensitivity concern for the positive/negative prompt proxy; please qualify it to refer to the logical non-negativity argument rather than all empirical findings.","section":"Appendix A.2"}],"recommendation":"major_revision","confidential_remarks":"To the editor: this is a workshop-level diagnostic/negative result. The first reason for misleading SAE decompositions is convincingly established by the zero-vector control; the second is logically sound but its empirical quantification needs direct validation. A revision that adds direct projection or sparse-coefficient measurements should resolve this without changing the paper's scope. The manuscript appropriately acknowledges concurrent work and provides code, so the main barrier is the proxy validation, not the overall direction."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth your time if you work on SAEs or steering vectors. The paper gives a clear, systematic answer to why direct SAE reconstruction of steering vectors fails, and it does not oversell what it proves. The zero-vector control in Table 1 is the strongest piece: the top activating features for a real steering vector and a zero vector are nearly identical, which makes the encoder-bias dominance hard to argue with. The out-of-distribution argument is also well supported by the L2-norm distribution and the default-component illustration. Repeating the analysis across seven behaviors (Appendix C) shows the effect is not a quirk of one steering vector. That alone is a useful contribution.\n\nThe softer spot is the second reason, negative coefficients. The 51.2% figure and Table 3 come from comparing SAE activations on positive versus negative prompts, not from a direct projection of the steering vector onto feature directions. Because the JumpReLU encoder is nonlinear, the sign of the activation difference need not match the sign of the true coefficient. The authors explicitly say the method does not provide ground truth, yet they still lean on it as evidence that negative projections are prevalent. The single worked example with features 14004 and 3517 is illustrative, not a prevalence measurement. So the second reason is plausible but empirically under-supported. The scaling argument also relies on an unverified default-component model, though that is a secondary point.\n\nI do not think the central claim breaks: the OOD story is independently solid, and the paper is honest about the limits of its negative-coefficient evidence. The paper is also fair to prior work and does not fit any free parameters beyond an arbitrary top-100 cutoff that they show does not change the qualitative result. Code is released.\n\nThis is a paper for mechanistic interpretability people who want to know why SAE decompositions of steering vectors fail and what to avoid. It deserves a serious referee. The main revision should be to either validate the prompt-difference proxy against direct projections on a subset of features or soften the prevalence claim. I would accept it for review and would cite it for the OOD finding.","headline":"A clean diagnostic paper: the OOD/encoder-bias story is solid, but the negative-coefficient prevalence is backed by an unvalidated proxy rather than direct measurement.","tokens_in":9949,"tokens_out":1078,"would_cite":true,"duration_ms":11485,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Sparse autoencoders cannot faithfully decompose steering vectors because such vectors are out-of-distribution and carry negative feature projections SAEs cannot represent.","keywords":["sparse autoencoders","steering vectors","activation steering","contrastive activation addition","mechanistic interpretability","large language models","out-of-distribution vectors","negative feature projections"],"falsifier":"Take the corrigibility steering vector and compute its signed coefficients in the SAE feature basis with unconstrained least squares, allowing negative values. If the largest-magnitude coefficients are not predominantly negative, or if they disagree with the positive-versus-negative prompt activation differences, the paper's evidence for widespread negative projections is undermined.","tokens_in":9006,"feed_emoji":"🧭","tokens_out":8814,"duration_ms":69140,"temperature":0.7,"pith_summary":"Steering vectors let researchers push a language model's behaviour in a desired direction, but what those vectors actually represent remains unclear. This paper argues that the natural tool for answering that question, sparse autoencoders (SAEs), cannot be applied to steering vectors directly: the vectors fall outside the distribution SAEs were trained on, and the non-negative coefficients SAEs produce cannot express the negative projections such vectors contain. If the paper is right, existing SAE-based interpretations of steering vectors are systematically misleading, and interpretability work should decompose positive and negative prompt activations separately before subtracting, or find another way to represent signed coefficients. The stakes are practical: a correct interpretation would help explain why steering works for some behaviours and not others, and why combining steering vectors often fails.","feed_headline":"Sparse autoencoders mislead on steering vectors","feed_subtitle":"Direct SAE decomposition is bias-dominated and misses the negative projections steering vectors rely on.","key_machinery":"The argument turns on two mechanisms. Contrastive activation addition builds a steering vector as the averaged difference between activations on positive and negative prompt pairs; this subtraction removes the default components that are present in every activation, so the resulting vector sits far from the distribution the SAE encoder was calibrated for, and the encoder's bias term dominates the decomposition. The second mechanism is the SAE encoder's non-negative activation function, such as ReLU or JumpReLU, which forces every reconstruction coefficient to be non-negative; a contrastive difference vector, however, naturally has negative projections, and because SAE features can have negative cosine similarity with one another, a true negative projection can appear as a spurious positive activation in a different feature. Together these make direct SAE decomposition unreliable.","core_discovery":"The paper's central claim is that direct SAE-decomposition of steering vectors is misleading for two structural reasons. First, steering vectors fall outside the distribution of model activations SAEs are trained on: they have much smaller L2 norms and, because contrastive activation addition subtracts paired activations, they lack the default context components that SAE encoder biases are calibrated to offset; as a result, the decomposition is dominated by encoder bias, as shown by the near-identical top features for a steering vector and a zero vector. Second, SAEs enforce non-negative reconstruction coefficients, while a steering vector formed by subtracting activations naturally has meaningful negative projections in feature directions; the paper finds that for seven behaviours roughly half of the large-magnitude feature differences are negative, and that negative projections can masquerade as positive activations in features with negative cosine similarity. The conclusion is that the apparent SAE interpretation of a steering vector is not a reliable guide to what the vector does.","pith_inferences":["If this diagnosis holds, the meaning of a steering vector may be context-dependent: in different activation contexts the same vector can act as a positive write to one feature or a negative write to a feature with opposite direction, so any context-free interpretation may be impossible, a possibility the paper sketches in its appendix.","The out-of-distribution argument generalises beyond SAEs: any linear readout trained with a bias on activations will misread contrastive difference vectors, so other interpretability tools that operate in residual-stream space may carry the same blind spot.","A testable extension would be to train an SAE directly on contrastive difference vectors, or to compare the paper's prompt-activation proxy against an unconstrained least-squares decomposition of the steering vector in SAE feature space, which would provide the ground-truth check the paper's method lacks.","If negative projections are as common as reported, interpretability work should consider signed dictionary learning or separate positive and negative feature decompositions, not just non-negative sparse coding, for any object built from activation differences."],"forward_implications":["Any SAE feature ranking obtained by feeding a steering vector through the encoder is dominated by encoder bias; the top features are essentially the same as those produced by a zero vector.","Negative feature projections are not a corner case: across the seven behaviours studied, roughly half of the largest-magnitude feature differences are negative, so SAE interpretations silently omit a large fraction of the steering mechanism.","A true negative projection can appear as a false positive activation of a different feature when the corresponding feature directions have negative cosine similarity, so even the positive features reported by an SAE can point to the wrong concept.","Alternative sparse approximation methods that replace the SAE encoder address the out-of-distribution problem but still fail unless they allow negative coefficients, making negativity the more fundamental obstacle.","A more promising route is to decompose positive and negative prompt activations separately and then subtract the decompositions, keeping inputs in-distribution and permitting signed coefficients; the paper proposes this as future work."],"supporting_citations":[{"why":"Defines Contrastive Activation Addition and provides the corrigible-neutral-HHH dataset from which all steering vectors in the paper are extracted.","marker":"[13]"},{"why":"Supplies the open-source SAEs whose encoder bias and non-negative activation are shown to dominate and distort direct steering-vector decompositions.","marker":"[9]"},{"why":"Earlier study reporting that SAE-based steering-vector feature selection and reconstruction give mixed results, providing the empirical puzzle the paper explains.","marker":"[4]"},{"why":"Documents that SAE reconstructions of task vectors lose steering ability and proposes an alternative optimization approach that still leaves the negative-coefficient problem unresolved.","marker":"[8]"},{"why":"Establishes that SAE feature directions can have negative cosine similarity, the mechanism behind the spurious positive activations in Section 3.2.","marker":"[7]"},{"why":"Provides the text corpus used to measure the distribution of model activation norms that steering vectors fall outside of.","marker":"[6]"},{"why":"Supplies the idea of default components consistently present in model activations, which contrastive subtraction removes and which scaling cannot restore.","marker":"[21]"},{"why":"Defines the steerability metric the paper uses to select layer 14 and to confirm the steering vectors actually control behaviour.","marker":"[18]"}],"fun_headline_variants":["SAEs misread steering vectors: bias and negatives","Steering vectors expose SAE blind spots","Why SAE decompositions of steering vectors fail","Two structural flaws in SAE steering-vector analysis"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that differences in SAE feature activations between positive and negative prompts reveal how strongly the steering vector points along each feature; the authors state this is only a proxy and not a ground-truth decomposition.","fun_headline_variants_meta":{"raw":{"variants":["SAEs misread steering vectors: bias and negatives","Steering vectors expose SAE blind spots","Why SAE decompositions of steering vectors fail","Two structural flaws in SAE steering-vector analysis"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000211,"raw_usage":{"total_tokens":1360,"prompt_tokens":839,"completion_tokens":521,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":455,"completion_tokens_details":{"reasoning_tokens":461}},"tokens_in":455,"tokens_out":521,"duration_ms":5478,"temperature":1.0,"reasoning_tokens":461,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T21:19:03.708007+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the corrigibility steering vector and compute its signed coefficients in the SAE feature basis with unconstrained least squares, allowing negative values. If the largest-magnitude coefficients are not predominantly negative, or if they disagree with the positive-versus-negative prompt activation differences, the paper's evidence for widespread negative projections is undermined.","supporting_citations":[{"cited_title":"Progress update #1 from the GDM mech interp team","cited_arxiv_id":null,"evidence_quote":"Earlier study reporting that SAE-based steering-vector feature selection and reconstruction give mixed results, providing the empirical puzzle the paper explains."},{"cited_title":"Extract- ing sae task features for in-context learning","cited_arxiv_id":null,"evidence_quote":"Documents that SAE reconstructions of task vectors lose steering ability and proposes an alternative optimization approach that still leaves the negative-coefficient problem unresolved."}],"review_version":1}