{"id":"15fc025a-7fcf-41b7-bfd2-b4f702e312bc","arxiv_id":"2505.07831","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"A GPT2-XL analysis reports that a neuron's highest-activation tokens are also the ones most similar to multiple categorical subclusters, offered as evidence for an intra-neuronal vector-space view of polysemy.","lead":"This paper proposes that each artificial neuron in a language model can be viewed as its own internal 'categorical vector space,' made of smaller token groups inherited from the previous layer. It then reports correlations showing that a neuron's most strongly activated tokens sit close to several of these groups at once.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The monotonic activation–proximity result may be built into the construction: proximity scores include a token's cosine with itself when the token lies in the taken-cluster, and cluster membership is coupled to target activation by construction.","rationale":"The reader's weakest assumption points to the same area: taken-clusters and proximity scores are not independent measurements of semantic sub-dimensions, because the same core-token sets define both the activation ordering and the proximity score. My concern is more specific and more damning: the proximity score includes the token's cosine with itself whenever the token is in the taken-cluster, and membership in a taken-cluster is itself determined by activation thresholds in the target and precursor neurons. Because the target neuron's activation is an affine function of precursor activations, high target activation is mechanically associated with membership in multiple clusters, which inflates s_ijk via self-similarity. The central monotonicity result is therefore consistent with a null model in which no categorical vector space exists: it only requires that high-activation tokens are more likely to be in the clusters used to compute their own proximity scores. This is a structural confound, not a matter of effect size or external consensus, and it directly undermines the falsifiable prediction stated in Section 4.1 and Table 1. The proposed leave-one-out check cleanly separates the self-inclusion artifact from any genuine embedding-geometry signal. Given that the paper's headline contribution depends on this evidence, and the authors do not provide code, baselines, or a control for cluster membership, the reader's REJECT verdict remains appropriate; if the leave-one-out check preserved the pattern, the argument would need to be reassessed, but the current manuscript does not support the central claim.","tokens_in":29629,"tokens_out":4090,"duration_ms":44101,"concrete_test":"Recompute Tables 1 and 2 with a leave-one-out proximity score: for each core-token j and each taken-cluster K, compute s_ijk as the mean cosine similarity between embedding(j) and the embeddings of tokens in K \\ {j}, excluding j whenever j ∈ K. Then repeat the group comparison and the Kendall analyses on the same 614/612 neurons, and report the statistics separately for tokens that belong to at least one taken-cluster versus tokens in none. If the 78% positive d and the τ ≈ 0.8 correlations persist with similar magnitude, the confound is not solely self-inclusion; if they collapse, the monotonicity result is an artifact of the cluster construction.","verdict_should_be":"REJECT","load_bearing_attack":"The central empirical claim (Sections 4.1, Tables 1–2) is that, for a layer-1 neuron, a core-token's activation rank is monotonically related to its mean dimensional proximity to the neuron's principal categorical sub-dimensions. A sub-dimension k is operationalized as a taken-cluster K_{i,p} = C_i ∩ C_p, where C_i and C_p are the 100 most-activated tokens of the target neuron and of one of its ten strongest L0 precursors (Sections 3.2–3.3). For each core-token j, s_ijk is the mean GPT2-XL embedding cosine similarity between j and all tokens in K_{i,p}. If j ∈ K_{i,p}, the averaged set includes j itself, contributing a perfect cosine of 1. Cluster membership is not independent of activation: to belong to K_{i,p}, a token must be among the top-100 activations of a strongly connected L0 neuron and of the target neuron, and since the target activation is an affine function of those L0 activations, high target activation and cluster membership are coupled by construction. Thus high-activation tokens are disproportionately represented in the clusters, their scores are inflated by self-similarity, and the reported d > 0 in 78% of neurons and Kendall τ ≈ 0.78–0.85 can arise even if the embedding geometry carries no semantic sub-dimension structure at all. The paper's note that averaging uses n < 10 sub-dimensions does not address this confound; the clusters are built from the same activation ordering that the test is meant to validate.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an alternative geometric account of polysemy in GPT2-XL neurons, in which a layer-n neuron is interpreted as a categorical vector space whose non-orthogonal basis consists of categorical sub-dimensions clipped from its strongly connected layer n-1 precursors. The central empirical claim, tested in Section 4.1, is that a token's activation level is a positive, monotonic function of its dimensional proximity to those categorical sub-dimensions, operationalized as mean GPT2-XL embedding cosine similarity to the neuron's taken-clusters (intersections of the target neuron's 100 most-activated tokens with the 100 most-activated tokens of each of its ten strongest layer-0 precursors). The paper reports group comparisons (Table 1), Kendall correlations on an average neuron (Table 2), and a PCA of sub-dimension proximities (Section 4.2), interpreting the results as evidence for a structured intra-neuronal space with a 'critical' less-polysemous zone.","tokens_in":29996,"tokens_out":3279,"duration_ms":36348,"significance":"If the central claim were established, the proposed intra-neuronal categorical vector space would be a conceptually novel and potentially useful explainability construct, and the paper does formulate a clearly falsifiable monotonicity hypothesis. The work also has strengths: it builds on publicly available GPT2-XL data, explicitly contrasts its account with the superposition framework rather than dismissing it, computes effect sizes (Cliff's delta) alongside p-values, and makes its operationalization concrete enough to be criticized and improved. However, the empirical core is currently undermined by a constructional coupling between the activation ordering and the proximity measure, and by statistical analyses that pool across neurons without accounting for neuron-level variability. The paper is more persuasive as a theoretical proposal than as a validated empirical finding.","major_comments":[{"comment":"The main association is partly constructed by the measurement design. A taken-cluster K_{i,p} is defined as the intersection of the target neuron's 100 most-activated core-tokens with a precursor's 100 most-activated core-tokens, and s_ijk averages cosine similarity between token j and all members of K_{i,p}. If j belongs to K_{i,p}, its self-similarity contributes a perfect score of 1, and cluster membership is itself determined by high activation of j in the target neuron. High-activation tokens are therefore disproportionately included in the clusters and have inflated proximity scores, so the reported d > 0 in 78% of neurons and Kendall tau ≈ 0.78–0.85 can arise even if embedding geometry carries no categorical sub-dimension structure. The paper's note that averaging is over n < 10 sub-dimensions does not address this confound. Please provide a permutation baseline or a hold-out evaluation that removes self-similarity (e.g., leave-one-out cosine) and decouples cluster definition from the target neuron's activation ranking.","section":"§4.1, Tables 1–2"},{"comment":"The Kendall tau coefficients are computed on a single 'average neuron' formed by pooling 612 neurons after rank-alignment, and the reported p-values treat these 100 rank points as independent observations. This is a pseudoreplicated analysis: it does not characterize the distribution of neuron-level correlations that the hypothesis is about, and it can mask substantial heterogeneity. Please report the distribution of neuron-level Kendall tau values with confidence intervals (or a mixed-effects/hierarchical analysis), rather than a single pooled coefficient.","section":"§4.1, Table 2"},{"comment":"The analyses use different inclusion thresholds across the three results without justification: n = 614 neurons in Table 1 (at least three taken-clusters of at least six tokens each and a cumulative union of at least 40 tokens), n = 612 neurons in Table 2 (exactly three taken-clusters), and n = 463 neurons in Section 4.2 (exactly four taken-clusters). The paper does not show that the central monotonicity result is robust to these choices, and the ad hoc thresholds make the reported effect sizes difficult to interpret. A sensitivity analysis or a principled selection rule is needed.","section":"§3.2, §4.1, §4.2"},{"comment":"The operationalization of a 'categorical sub-dimension' as a taken-cluster is assumed rather than validated. The paper's geometric interpretation requires that these intersections of top-100 activation sets correspond to semantically coherent sub-dimensions, but no evidence is provided that they are more coherent than random token sets matched for frequency or embedding norms, nor is there a control condition comparing the observed embeddings to a null model (e.g., shuffled cluster labels or random embeddings). Without such a control, the PCA structure in Section 4.2 and the monotonicity in Section 4.1 could reflect generic properties of token-frequency or embedding geometry rather than the proposed categorical vector space.","section":"§3.2, §4.1, §4.2"}],"minor_comments":[{"comment":"Please define the symbols αmin, αmax, σmin, σmax, δ, and Δc in the table captions or text; they are used without definition.","section":"§4.1, Tables 1–2"},{"comment":"Report p-values as p < 0.0001 rather than p = .0000, which implies a zero probability.","section":"§4.1, Table 2"},{"comment":"The French paragraph followed by its English translation is redundant in a journal article; retaining only one language would improve readability.","section":"§1.5, near 'Le détourage catégoriel'"},{"comment":"The caption says 'Comparison of mean activations between categorical clusters from hierarchical classification on tokens' embeddings (layer 0)', but the text says the graph is produced from the same data as Table 2 (layer 1, taken-clusters); please clarify the mismatch.","section":"Graph n°1 caption"},{"comment":"Several references appear off-topic or misattributed (e.g., [1] on quantum code generation, [5] and [12] are used inconsistently), and there is a large proportion of self-citations to unpublished preprints; please review the bibliography for relevance and accuracy.","section":"Bibliography"}],"recommendation":"reject","confidential_remarks":"The paper reads as part of an ongoing self-published research program, with heavy citation of the authors' own preprints and many off-topic or unverifiable references. Beyond the technical circularity in the main analysis, I am concerned about fit with a cs.CL venue: the empirical content is confined to a single model and two layers, and the theoretical framing borrows heavily from cognitive psychology and philosophy in ways that are not tested. The central idea could be salvaged by a rigorous re-analysis with proper baselines, but as submitted the load-bearing evidence is invalid. I recommend rejection rather than major revision because the required changes are substantial and effectively constitute a new experimental design."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Punchline: the monotonic activation–proximity result in Tables 1–2 is probably mechanical. Because taken-clusters are the intersection of a neuron's top-100 tokens with its precursors' top-100 tokens, and each core-token's proximity score averages cosine with that same cluster, any token inside a cluster gets its own cosine of 1. High-activation tokens are preferentially in clusters (target activation is affine in precursors), so their scores are inflated for free. Without a permutation baseline or exclusion of self-similarity, the paper's main evidence supports nothing beyond the construction.\n\nWhat's real: the paper offers a clear geometric alternative to superposition — a neuron as an intra-neuronal vector space with non-orthogonal basis — and it's honest that this is an interpretation, not a rejection of prior work. The non-parametric methods are appropriate for small samples, and the authors report an effect size. The idea of a 'critical categorical zone' is testable in principle.\n\nThe soft spots are not minor. The self-cosine artifact is load-bearing: it explains the 78% positive difference and the τ ≈ 0.8 correlations without any semantic sub-dimension. The analysis also switches inclusion thresholds (614, 612, 463), computes Kendall τ on an averaged neuron rather than across neurons, provides no code or data, and Section 1.2 cites Haider et al. [1] for a claim that the bibliography entry (a quantum-code dataset paper) cannot support. The conceptual vocabulary — categorical convergence, clipping, intra-neuronal attention — is from the authors' prior papers; this submission adds a correlational restatement, not a new mechanism or intervention.\n\nWho it's for: people working on neuron interpretability could read this as a cautionary example of how operationalization can create the effect under test. The framework itself may inspire a better-designed study, but as submitted the evidence is not there. I'd reject, but I'd explicitly invite a revision that reruns with self-similarity excluded and a random-top-100 baseline. If that comes back, it's worth serious review.","headline":"The main empirical claim is likely built into the construction; the paper's geometric hypothesis is clear but unsupported as presented.","tokens_in":30529,"tokens_out":3032,"would_cite":false,"duration_ms":32959,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A neuron in a language model may be a small categorical vector space of its own.","keywords":["polysemantic neurons","interpretability","categorical vector space","non-orthogonal basis","activation space","intra-neuronal attention","GPT2-XL","superposition"],"falsifier":"Recompute the activation-versus-proximity monotonicity after replacing each taken-cluster with a frequency-matched random set of tokens drawn from the same layer-0 neuron's core tokens; if the positive difference d > 0 and Kendall's tau around .8 persist unchanged, then token frequency or embedding geometry, rather than categorical sub-dimensional membership, explains the ordering.","tokens_in":29419,"feed_emoji":"🧠","tokens_out":7580,"duration_ms":71957,"temperature":0.7,"pith_summary":"This paper proposes a geometric account of why a single neuron in a language model can respond to many different kinds of input. Rather than treating a neuron only as one feature direction embedded in a layer-wide space, it models each neuron in layer n as a categorical vector space whose non-orthogonal basis consists of categorical sub-dimensions clipped from strongly connected neurons in layer n-1. Within that space, the paper claims, a token's activation level is a positive function of its coordinates on those sub-dimensions, so the most strongly activated tokens sit at the intersection of several sub-dimensions and are less polysemous. If true, this gives a directly testable way to locate a less-polysemous 'critical' zone inside each neuron, and it complements rather than contradicts the usual superposition explanation of polysemy.","feed_headline":"Activation tracks token proximity to a neuron's categorical core","feed_subtitle":"In GPT2-XL's first two layers, the highest-activated tokens sit closest to the intersection of a neuron's categorical sub-dimensions.","key_machinery":"The load-bearing object is the taken-cluster, defined as the set of tokens simultaneously among the 100 most-activated tokens of a layer-1 neuron and among the 100 most-activated tokens of one of its ten most strongly connected layer-0 precursor neurons; each taken-cluster is treated as the extensional realization of one principal categorical sub-dimension. Dimensional proximity, the mean cosine similarity between a token and a taken-cluster in GPT2-XL's input-embedding space, operationalizes the token's coordinate on that sub-dimension, and averaging these coordinates across sub-dimensions yields the value compared with activation rank. The mechanism claimed to generate this geometry is categorical clipping, driven by three effects: categorical priming (the x effect), inter-neuronal categorical attention (the w effect), and categorical phasing (the sum effect). Together they are said to make highly activated tokens fall at the intersection of several sub-dimensions, producing categorical convergence and reduced polysemy at high activation levels.","core_discovery":"The central claim is that the polysemantic thought category of a neuron in layer n can be modeled as a categorical vector space with a non-orthogonal basis, where each basis element is a categorical sub-dimension clipped from the category of a strongly connected precursor neuron in layer n-1. Operationally, each principal categorical sub-dimension is represented by a taken-cluster: the intersection of the layer-1 neuron's 100 most-activated core-tokens with the 100 core-tokens of one of its ten strongest layer-0 precursor neurons. For each core-token, its coordinate on a sub-dimension is measured as the mean cosine similarity, in GPT2-XL's input-embedding space, between that token and the taken-cluster tokens. The paper reports that 78.01% of 614 qualifying layer-1 neurons show a positive difference between the mean dimensional proximity of their most activated and least activated core-tokens, with a mean difference of .0274 and a Cliff's delta of .2407, and that across 612 neurons with exactly three sub-dimensions, Kendall's tau between activation rank and mean proximity ranges from .78 to .85. It concludes that activation magnitude is a positive, monotonic function of a token's coordinates in the neuron's categorical vector space, with the highest activations marking the intersection of several sub-dimensions and thus a less-polysemous critical zone.","pith_inferences":["Beyond the paper: if the monotonicity holds, a purely activation-based token ranking could be used to select intervention targets for steering a model's output, without needing sparse autoencoders or separately trained concept detectors.","Beyond the paper: the same construction should be testable on later layers, since the paper's mechanism predicts that the critical zone of a layer-n neuron should become the source of layer-(n+1) sub-dimensions, so the monotonic relationship should reappear at each layer transition.","Beyond the paper: replacing the fixed input embeddings with contextual or layer-specific embeddings would test whether the categorical vector space is an intrinsic feature of the neuron or an artifact of the observation frame, since the paper's operationalization uses only GPT2-XL's input-embedding space."],"forward_implications":["Activation rank can serve as a cheap, local readout of categorical membership: the most strongly activated tokens of a neuron are its least polysemous ones, concentrated near the intersection of its sub-dimensions.","The intra-neuronal critical zone identified this way becomes the natural source for categorical clipping into the next layer, giving a concrete mechanism for how categories are refined layer by layer.","The non-orthogonal basis description reconciles the standard superposition view with the observation that neurons look monosemantic at their top tokens: both are true at different activation scales.","Dimensional proximity scores computed from taken-clusters offer an interpretability tool that explains individual neuron behavior without training supervised probes.","The monotonic activation-proximity relationship gives a quantitative handle on the abstract notion of intra-neuronal attention, turning it from a qualitative description into a measurable ordering of tokens."],"supporting_citations":[{"why":"Supplies the GPT2-XL neuron-explainer dataset of the 100 most-activated tokens per neuron from which core-tokens and taken-clusters are built.","marker":"[12]"},{"why":"Earlier study defining categorical divergence (discontinuity and inhomogeneity) among moderately activated core-tokens, the phenomenon the paper re-explains.","marker":"[73]"},{"why":"Earlier study defining categorical convergence at high activations and postulating the decomposition of a neuron's category into categorical sub-dimensions.","marker":"[74]"},{"why":"Defines the three extraction effects, categorical priming, inter-neuronal categorical attention, and categorical phasing, used to explain how sub-dimensions are clipped.","marker":"[75]"},{"why":"Defines categorical clipping and its properties, grounding the notion of a categorical sub-dimension as an extracted homogeneous token subset.","marker":"[76]"},{"why":"Defines categorical restructuring, partial categorical confluence, and activation dispersion, establishing the non-orthogonal partial-confluence geometry of sub-dimensions.","marker":"[77]"},{"why":"Defines intra-neuronal attention and the increase of categorical homogeneity with activation, the mechanism the present study formalizes as a coordinate-proximity monotonicity.","marker":"[78]"},{"why":"Provides the observation that neurons appear monosemantic on top-activated tokens but polysemantic at lower activation levels, the central empirical pattern the paper's geometry reconciles.","marker":"[19]"},{"why":"Cited 2025 study showing polysemantic neurons encode different concepts in separate activation ranges, the contrast against which the paper's monotonic vector-space account is positioned.","marker":"[1]"}],"fun_headline_variants":["Neuron activation scores proximity to categorical sub-dimension core","GPT2-XL neurons: activation monotonic with categorical coordinate","Inside a neuron: activation reflects distance to categorical hub","Non-orthogonal basis predicts neuron activation in language models","Polysemantic neurons have a geometric core that drives activation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument assumes that a 'taken-cluster' (the intersection of a layer-1 neuron's 100 most-activated tokens with a layer-0 neuron's 100 most-activated tokens) is the true extensional realization of a categorical sub-dimension and that GPT2-XL input-embedding cosine similarity measures a token's coordinate on that sub-dimension, so if those intersections are artifacts of token frequency or embedding geometry, the monotonicity result would not establish the proposed vector space.","fun_headline_variants_meta":{"raw":{"variants":["Neuron activation scores proximity to categorical sub-dimension core","GPT2-XL neurons: activation monotonic with categorical coordinate","Inside a neuron: activation reflects distance to categorical hub","Non-orthogonal basis predicts neuron activation in language models","Polysemantic neurons have a geometric core that drives activation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000605,"raw_usage":{"total_tokens":2820,"prompt_tokens":940,"completion_tokens":1880,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":556,"completion_tokens_details":{"reasoning_tokens":1798}},"tokens_in":556,"tokens_out":1880,"duration_ms":15381,"temperature":1.0,"reasoning_tokens":1798,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:59:34.662725+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute the activation-versus-proximity monotonicity after replacing each taken-cluster with a frequency-matched random set of tokens drawn from the same layer-0 neuron's core tokens; if the positive difference d > 0 and Kendall's tau around .8 persist unchanged, then token frequency or embedding geometry, rather than categorical sub-dimensional membership, explains the ordering.","supporting_citations":[],"review_version":1}