{"id":"9a8f6836-9eca-4db3-82b9-9aaa857bcedd","arxiv_id":"2501.06254","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"PS-Eval, a Word-in-Context-based benchmark, shows that SAEs optimized for MSE-L0 do not necessarily extract better word-meaning features, and that separation improves in deeper layers and attention outputs.","lead":"This paper introduces PS-Eval, a benchmark that tests whether sparse autoencoders (SAEs) on language models separate different meanings of the same word into different internal features. It finds that optimizing the usual reconstruction/sparsity trade-off does not guarantee better meaning separation, and that deeper layers and attention modules show higher separation.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"PS-Eval's F1 metrics rest on the unvalidated assumption that the maximally activated SAE feature encodes the target word's context-specific meaning; if argmax is dominated by syntax or position, the ReLU > JumpReLU > TopK ranking does not measure monosemanticity.","rationale":"The central claim is that MSE-L0 Pareto improvements do not guarantee better monosemantic extraction, supported operationally by the ReLU > JumpReLU > TopK F1 ranking. That ranking is only meaningful if PS-Eval's confusion matrix measures semantic separation. The paper's own Section 8 identifies the load-bearing assumption--argmax feature equals meaning-bearing feature--and relegates a fix to future work. The validation in Section 5 is anecdotal (three words) and uses the same in-domain SAE, so it does not license dataset-wide inference. The proposed concrete test directly checks whether the argmax proxy is valid and whether the ranking survives a distributional metric. If the ranking flips, the central claim collapses; if it survives, the concern is resolved. I agree with the reader's weakest assumption. Secondary issues (specificity near ceiling for random SAEs; activation-function comparison not matched on L0) are real but less fundamental: specificity is not used for the central ranking, and the L0 confound could be addressed by matching sparsity after the proxy validity is established.","tokens_in":970,"tokens_out":870,"duration_ms":121033,"concrete_test":"Run two checks. (1) Recompute the Figure 4 confusion matrices using a distribution-level criterion instead of argmax equality--e.g., label a pair as 'same meaning' if the cosine similarity between f(x1) and f(x2) exceeds a threshold calibrated on mono-contexts, and 'different meaning' otherwise--then re-rank ReLU, JumpReLU, and TopK on F1. If ReLU is no longer best, the headline is an argmax artifact. (2) Audit the proxy: sample 100 PS-Eval pairs; for each context, take the argmax feature, decode it as W_dec(f_max), and have two independent annotators (or GPT-4 with the prompt in Appendix N) judge whether the decoded top tokens match the target word's context-specific meaning. Report the proportion of argmax features judged meaning-relevant; if below roughly 70%, the argmax proxy is not reliable enough to support the paper's conclusions.","verdict_should_be":"UNCHANGED","load_bearing_attack":"PS-Eval's confusion matrix (Table 2) and all derived metrics (accuracy, precision, recall, F1, specificity) classify a pair by whether the same latent index is the maximum activation of the SAE for the target token in the two contexts. The central claim's operational evidence--the ReLU > JumpReLU > TopK F1 ranking (Figure 4)--rests entirely on this argmax identity being a faithful proxy for 'the monosemantic feature that encodes the word's meaning in context.' Section 5's validation is qualitative: three example words, with top-7 logits of W_dec applied to the max feature. It shows those instances are semantically coherent, but it does not establish that the argmax is the meaning-bearing feature across the 1112 PS-Eval samples, nor that it is unaffected by syntactic/positional features or by a generic 'target token' feature. Section 8 concedes this: 'it is not guaranteed that the feature with the maximum activation always aligns with the intended word meanings.' Because TopK and JumpReLU systematically change activation distributions (fixed vs. thresholded sparsity), they can change argmax behavior for reasons unrelated to semantics; if argmax is noisy or off-meaning, the F1 ranking is an artifact of the proxy, not evidence that MSE-L0-optimized SAEs fail to extract monosemantic features.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PS-Eval, a benchmark built from the WiC dataset, which evaluates sparse autoencoders (SAEs) by comparing the maximally activated SAE feature for a target word in two contexts and forming a confusion matrix over whether the same latent index fires. The authors report that ReLU outperforms JumpReLU and TopK on PS-Eval F1, use this to argue that optimizing the MSE-L0 Pareto frontier does not necessarily improve the extraction of monosemantic features, and further claim that deeper layers and the Attention module contribute to distinguishing polysemy, based on Specificity. The manuscript includes a new dataset and metric, code and data release, and baselines including a random SAE and a dense SAE.","tokens_in":24176,"tokens_out":5548,"duration_ms":57422,"significance":"If the central negative result holds, PS-Eval would be a useful complement to MSE and L0 metrics for SAE development, and the paper's release of code, data, and baselines is a concrete contribution. The random-SAE and dense-SAE controls show awareness of metric pathologies. However, the main result depends on the unvalidated assumption that the argmax feature is the meaning-bearing feature, and the layer/attention conclusions rely on Specificity despite the paper's own warning that Specificity can be trivially high. The significance is therefore conditional on additional validation and re-analysis.","major_comments":[{"comment":"The paper explicitly warns in Section 4.2 that Specificity 'may reach an unfairly high value by definition' and recommends holistic metrics, and Appendix D shows that a random SAE attains Specificity ≈ 0.99999999668. Yet Section 7 and the abstract use Specificity alone to conclude that deeper layers and the Attention module contribute to distinguishing polysemy. These conclusions are unsupported: high Specificity can simply reflect low chance of identical argmax features rather than semantic discrimination. Please re-analyze the layer and component comparisons with F1, balanced accuracy, or a direct comparison against the random and dense baselines.","section":"Section 4.2, Appendix D, Section 7, Figures 5-6"},{"comment":"PS-Eval classifies each sample by whether the maximally activated SAE feature index is identical across the two contexts, and all headline comparisons (including the ReLU > JumpReLU > TopK ranking) rest on this identity being a faithful proxy for the meaning-bearing monosemantic feature. Section 8 concedes that 'it is not guaranteed that the feature with the maximum activation always aligns with the intended word meanings,' and Section 5 validates this assumption on only three words using qualitative top-7 logits. Because TopK and JumpReLU systematically change the activation distribution, their lower F1 could be an artifact of argmax instability rather than a genuine failure to extract monosemantic features. Please provide quantitative validation on a larger sample (e.g., automated or logit-lens-based interpretability labels for a random subset of PS-Eval instances) or a distribution-based variant of the metric (e.g., overlap or KL divergence of the activation vector) to show the ranking persists.","section":"Section 4.2, Section 5, Section 8, Figure 4"},{"comment":"The default training setting uses in-domain data, namely WiC, which is the same source as the PS-Eval evaluation set, while Appendix B shows that in-domain versus open-domain training changes the precision-recall trade-off substantially. The activation-function comparison in Figure 4 may therefore be affected by differential overfitting to WiC across ReLU, JumpReLU, and TopK. Please report the activation-function ranking for SAEs trained on open-domain data, or at minimum discuss this confound explicitly in the interpretation of the ranking.","section":"Section 3.2, Section 6.3, Appendix B"},{"comment":"The activation-function comparison is not matched for sparsity or reconstruction quality: Table 8 reports L0 = 531 for ReLU, 383 for TopK(k=384), 190 for TopK(k=192), and 829 for JumpReLU(jump=0.0001), while the paper itself shows that recall and F1 vary with k within TopK. The observed ranking may thus reflect different operating points on the sparsity spectrum rather than the activation function per se. Please include comparisons at matched L0 (or a joint analysis controlling for L0), and report confidence intervals in the main figures rather than only in the appendix version with error bars.","section":"Figure 4, Table 8"}],"minor_comments":[{"comment":"The caption of Figure 14 states that 'JumpReLU outperforms both TopK and standard ReLU in terms of F1 score, Precision, and Recall,' which directly contradicts the main-text result in Section 6.3 and Figure 4. This appears to be a copy error and must be corrected.","section":"Appendix G, Figure 14 caption"},{"comment":"The paper calls PS-Eval model-independent, but the dataset construction filters instances whose target word is tokenized as a single token in GPT-2 small; the open-SAE evaluation on Pythia and Gemma should clarify how multi-token target words in those tokenizers are handled, otherwise the claim of model independence is overstated.","section":"Section 4.1, Appendix K"},{"comment":"The text states 'expand ratio × dmodel = 32 × 768 = 23576,' but the correct product is 24576; the following probability calculation uses 24576, so this is a typo in the displayed equation.","section":"Appendix D"},{"comment":"There is a typo in 'monosemantic fetures' that should be 'features,' and the claim that accuracy saturates around an expand ratio of 64 is only partially supported by the somewhat non-monotonic layer-6 results in Table 10 (e.g., accuracy drops below chance at R=8).","section":"Section 6.2"},{"comment":"The caption repeats 'better better' and describes trends starting at 'layer 6 onwards' without specifying which points are statistically distinguishable; please clarify the layer labels and add error bars or a statistical comparison across layers.","section":"Figure 5"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is already published at ICLR 2025, which is worth noting when weighing this report. For a fresh submission, the specificity-based layer/attention claims and the unvalidated argmax proxy would need to be addressed before the central conclusions can be accepted. The paper's release of data and code is a strength, but the internal inconsistency between Section 4.2's warning about Specificity and Section 7's use of it is the most urgent issue for a revision or corrigendum."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"PS-Eval is a real addition to SAE evaluation. The dataset-and-metric combo is new: WiC-derived polysemy pairs scored by whether the same SAE latent wins the argmax across contexts. It is model-independent, cheap to run, and the paper ships code, data, and several baselines you would want—random SAE, dense SAE, open-domain vs in-domain training, and open SAEs from other LLMs. The central negative result, that MSE-L0 Pareto improvements do not automatically buy semantic separation, is important and likely correct in spirit. The paper is also unusually candid about its own weaknesses, which makes it easier to trust.\n\nThe soft spots are real, but they are mostly in the open. The stress-test concern about the argmax proxy is valid: the F1 ranking assumes the maximally activated feature encodes the target word's context-specific meaning, and the logit-lens validation only covers three words. The authors explicitly flag this in Section 8 and suggest distribution-based metrics as future work, so it is a known limitation rather than a hidden one. What bothers me more is that the activation-function comparison is not controlled for sparsity. ReLU (with L1), JumpReLU (with thresholds), and TopK (with fixed k) end up at very different L0 levels—Appendix Table 8 shows L0 ranging from 190 to 831—so the ReLU > JumpReLU > TopK ranking could be driven by sparsity level rather than by the activation function per se. That is not fatal, but it means the headline claim is under-supported.\n\nThe layer-depth and attention-module conclusions rest mostly on Specificity, which the paper itself shows is near-ceiling for a random SAE. Those claims should be softened or re-derived from F1 or a balanced metric. There are also minor inconsistencies: Appendix G has a caption saying JumpReLU outperforms ReLU, contradicting the main text, and the appendix's Sensitivity formula is mis-stated. These are fixable in revision.\n\nWho is this for? Anyone working on SAE evaluation or interpretability comparisons. It is not a complete benchmark, but it is a useful semantic check that complements MSE/L0. I would bring it to a reading group and would cite it. It deserves a serious referee; my own verdict is that it needs revision, not rejection, and the explicit limitations make the revision path clear.","headline":"PS-Eval is a genuinely useful, honestly-reported benchmark for SAE semantic quality; the headline activation-function ranking rests on an unproven argmax proxy, but the paper's transparency and extra baselines make it worth engaging.","tokens_in":24733,"tokens_out":1667,"would_cite":true,"duration_ms":19572,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A word-sense benchmark for sparse autoencoders, PS-Eval, shows that models optimized for reconstruction and sparsity separate word meanings worse than plain ReLU — the field's headline metrics diverge from interpretability.","keywords":["sparse autoencoders","mechanistic interpretability","polysemy","monosemantic features","evaluation metrics","Word-in-Context dataset","MSE-L0 Pareto frontier","large language models"],"falsifier":"Re-run PS-Eval on the same 1,112 Word-in-Context-derived samples using a distributional similarity over all nonzero SAE features, or a linear probe trained on the full feature vector, instead of the single maximum activation. If TopK or JumpReLU overtakes ReLU on F1 under that scoring, the paper's headline ranking is an artifact of the max-activation convention rather than a property of monosemantic content. A complementary check would apply the logit-lens decoding to every sample and count how often the top feature's strongest decoded tokens relate to the gold meaning; a low fraction would mean the confusion matrix measures contextual pattern differences, not word meanings.","tokens_in":23683,"feed_emoji":"🧩","tokens_out":26281,"duration_ms":185073,"temperature":0.7,"pith_summary":"The paper tries to establish that the standard metrics for sparse autoencoders — reconstruction error (MSE) and sparsity ($L_0$) — do not track whether an SAE is actually doing its advertised job: carving polysemantic LLM activations into monosemantic, single-meaning features. To make that case it introduces PS-Eval, a benchmark built from the Word-in-Context dataset, which checks whether the maximally activated SAE feature for a target word stays the same when the word has the same meaning and changes when its meaning differs. The headline finding is that SAEs tuned to push the MSE-$L_0$ frontier do not improve — and on F1 actually worsen — semantic separation, with the activation-function ranking ReLU > JumpReLU > TopK. A secondary claim locates where polysemy resolution happens in the network: specificity rises in deeper layers and is highest in the attention submodule, even where MSE and $L_0$ look worst. If these results hold, the community's optimization loop has been pointed at a proxy that can diverge from the actual goal of interpretability.","feed_headline":"Word-sense test: 'better' sparse autoencoders split meanings worse","feed_subtitle":"Why it matters: the metrics the field optimizes would have called it the other way.","key_machinery":"The load-bearing object is PS-Eval itself: a dataset plus a confusion-matrix protocol. The dataset is a filtered slice of the Word-in-Context (WiC) dataset, keeping only words that tokenize as a single token in GPT-2 small, yielding 1,112 samples split evenly between poly-contexts, where the target word means different things in the two sentences, and mono-contexts, where it means the same thing. The protocol computes the maximally activated SAE feature for the target token in each context and classifies each pair: same top feature under the same meaning (True Positive), different top feature under the same meaning (False Negative), same top feature under different meanings (False Positive), and different top features under different meanings (True Negative). From that 2×2 table the paper computes accuracy, precision, recall, specificity, and F1, and it validates the max-activation convention with a logit-lens check — decoding the top feature through the unembedding matrix yields meaning-consistent vocabulary in the examples shown. The same confusion matrix is then rerun as a network probe, reporting specificity by layer depth and by Transformer component (residual, MLP, attention).","core_discovery":"On the paper's own terms, the central claim is that monosemanticity is a measurable property of SAE features, testable with polysemous words, and that it moves independently of the MSE-$L_0$ trade-off. PS-Eval takes a target word that appears in two contexts — the same meaning in mono-contexts, different meanings in poly-contexts — and compares the single most strongly activated SAE feature for the target token in each context. The same top feature under the same meaning is a True Positive; different top features under different meanings are a True Negative; the two mismatches complete the confusion matrix, and from it the paper derives accuracy, precision, recall, specificity, and F1. Three results follow: expanding the latent dimension improves semantic separation but saturates around an expand ratio of 64; the activation functions built to win the MSE-$L_0$ frontier, TopK and JumpReLU, score below plain ReLU on F1; and both deeper layers and the attention submodule raise specificity even as MSE and $L_0$ worsen. A logit-lens check is offered as evidence that the top features really carry the context-specific senses — 'space' as the universe decodes toward flight and gravity, while 'space' as a gap decodes toward layout and occupancy.","pith_inferences":["Editorial extension: the reported plateau near an expand ratio of 64 implies an efficiency ceiling for dictionary scaling that scaling-law arguments do not model; a testable consequence is that PS-Eval accuracy stops improving with dictionary size even while the MSE-$L_0$ curve keeps improving.","The paper's findings point toward a training-side fix it does not propose: a semantic term in the SAE loss (for instance, contrastive separation of word senses) would make PS-Eval a design target rather than a post-hoc check.","The attention-specificity result hints at a mechanistic reading the authors leave implicit: polysemy may be disambiguated in the attention stream and propagated through MLP and residual paths, so SAEs trained on attention outputs would be the natural place to find sense-specific features.","The logit-lens validation covers only a handful of words ('space', 'save', 'ball'); scaling it to all 1,112 samples with an automated judge would convert a qualitative justification into a quantitative one and would test how often the max feature genuinely carries the gold sense rather than a correlated contextual cue."],"forward_implications":["SAE development should be checked against a semantics-focused metric like PS-Eval alongside MSE and $L_0$, because the paper shows the Pareto-frontier winners (TopK, JumpReLU) can be the semantic losers.","Scaling helps meaning separation only up to a point: accuracy saturates near an expand ratio of 64, so indiscriminate dictionary growth has diminishing returns for interpretability.","Layer and component choice matters more than reconstruction numbers suggest: deeper layers raise specificity even as MSE and $L_0$ worsen, and the attention submodule separates senses best on specificity despite a lower F1.","Because PS-Eval is model-independent, the same yardstick can rank SAEs trained on different base LLMs, as the paper demonstrates on GPT-2 small, Pythia 70M, and Gemma2-2B.","The logit-lens results imply that at least some individual SAE features each carry a single sense of a target word, which is direct evidence for the monosemanticity claim at the feature level."],"supporting_citations":[{"why":"Supplies the Word-in-Context dataset whose same/different-meaning sentence pairs are the raw material of PS-Eval.","marker":"Pilehvar & Camacho-Collados, 2019"},{"why":"Provides the TopK activation, SAE scaling laws, and the MSE-L0 Pareto-frontier framing that PS-Eval's results are measured against.","marker":"Gao et al., 2024"},{"why":"Introduces JumpReLU, the activation-function comparison that scores below ReLU on PS-Eval.","marker":"Rajamanoharan et al., 2024b"},{"why":"Formulates the superposition hypothesis and the polysemantic/monosemantic distinction that defines what SAEs are supposed to achieve.","marker":"Elhage et al., 2022"},{"why":"Supplies the scaling practice (expand ratio 32) and the feature-interpretation evidence that motivates measuring monosemanticity.","marker":"Templeton et al., 2024"},{"why":"Supplies the in-domain versus open-domain training-data setup and the critique that prior SAE evaluations are task-specific.","marker":"Makelov et al., 2024"},{"why":"Supplies Ghost Grads, the auxiliary loss the paper uses to reduce dead latents, which improves PS-Eval scores.","marker":"Jermyn & Templeton, 2024"},{"why":"GPT-2 small is the base LLM whose activations are decomposed and whose tokenization defines the PS-Eval dataset filter.","marker":"Radford et al., 2019"},{"why":"Earlier evidence that SAE features are interpretable and the IOI-task evaluation that PS-Eval positions itself against.","marker":"Cunningham et al., 2023"}],"fun_headline_variants":["Word-sense test flips SAE leaderboard: ReLU beats TopK","Dot-product metrics miss polysemy: new SAE eval","SAE interpretability not tied to MSE-L0 frontier","Polysemous words expose SAE feature confusion","Better MSE, worse sense separation in sparse autoencoders"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"PS-Eval assumes that the single most strongly activated SAE feature for a target word in a given context encodes that word's meaning in that context, so comparing the top features across two contexts reveals whether the two senses have been separated; the paper itself flags in its limitations section that this alignment is not guaranteed.","fun_headline_variants_meta":{"raw":{"variants":["Word-sense test flips SAE leaderboard: ReLU beats TopK","Dot-product metrics miss polysemy: new SAE eval","SAE interpretability not tied to MSE-L0 frontier","Polysemous words expose SAE feature confusion","Better MSE, worse sense separation in sparse autoencoders"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000331,"raw_usage":{"total_tokens":1889,"prompt_tokens":1035,"completion_tokens":854,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":651,"completion_tokens_details":{"reasoning_tokens":769}},"tokens_in":651,"tokens_out":854,"duration_ms":8033,"temperature":1.0,"reasoning_tokens":769,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:21:16.499009+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run PS-Eval on the same 1,112 Word-in-Context-derived samples using a distributional similarity over all nonzero SAE features, or a linear probe trained on the full feature vector, instead of the single maximum activation. If TopK or JumpReLU overtakes ReLU on F1 under that scoring, the paper's headline ranking is an artifact of the max-activation convention rather than a property of monosemantic content. A complementary check would apply the logit-lens decoding to every sample and count how often the top feature's strongest decoded tokens relate to the gold meaning; a low fraction would mean the confusion matrix measures contextual pattern differences, not word meanings.","supporting_citations":[{"cited_title":"Ghost grads: An improvement on resampling","cited_arxiv_id":null,"evidence_quote":"Supplies Ghost Grads, the auxiliary loss the paper uses to reduce dead latents, which improves PS-Eval scores."}],"review_version":1}