{"id":"e9942a31-bdc2-4f5d-bb1e-5b9e9aa5a878","arxiv_id":"2506.01629","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"During pre-training, multilingual models compress language-specific concept representations into shared cross-lingual neurons, and activating these shared neurons can trigger concept-relevant text in another language.","lead":"This paper traces how multilingual language models reorganize their internal neurons during pre-training, finding that concept-specific neurons gradually become shared across languages instead of staying separate. It matters because it offers a compression-based explanation for why multilingual models generalize across languages, and it exposes a bias toward high-resource languages in shared representations.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Neuron steering experiment has no no-manipulation or random-neuron baseline, so the late-checkpoint shift to English may reflect the model's default output distribution rather than cross-lingual concept neurons.","rationale":"The paper's strongest claim is causal: cross-lingually aligned neurons encode language-independent semantics, demonstrated by the steering experiment in Section 8. That experiment lacks control conditions. At late checkpoints the model may generate English from a BOS token regardless of neuron manipulation, and concept relevance is only qualitatively assessed. This is the single most load-bearing concern because it directly undercuts the abstract's 'Remarkably' statement. The parallel-translation issue raised by the reader is also valid for the correlational measures, but the steering experiment is where the causal interpretation is explicitly asserted. I recommend testing with no-manipulation and random-neuron baselines. If the effect persists, the claim is substantially strengthened; if not, the headline claim should be weakened to a correlational finding. I agree with the reader's CONDITIONAL verdict and propose this specific check to move it toward acceptance or rejection.","tokens_in":75913,"tokens_out":6483,"duration_ms":75022,"concrete_test":"At checkpoints 10k and 400k, run three generation conditions: (1) no neuron manipulation (BOS only), (2) top-500 random MLP neurons set to their median activations, (3) top-500 concept-specific neurons identified from Spanish as in the paper. Generate 100 outputs per condition with nucleus sampling p=0.9 and temperature t=0.8. Compare LANGDETECT language distributions and concept relevance, scored by a classifier trained on the English positive/negative concept sentences. The steering claim is supported only if condition (3) produces English and concept-relevant text at significantly higher rates than conditions (1) and (2).","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that cross-lingually aligned neurons 'encode generalized semantic representations, independent of the original language encoding' rests on the generation experiment in Section 8. The paper calls these 'controlled neuron manipulation experiments' (contribution iii), but the protocol in Appendix E has no control conditions. It sets the top-500 concept neurons (identified from Spanish/Chinese data) to their concept-specific median values, prompts only with <s>, and measures the language distribution of the generated text (Figure 7). At step 400k most generations are English, whereas at step 10k they are mostly Spanish. The interpretation is that concept knowledge has become language-independent. However, the model is a pretrained multilingual LM; from a BOS token alone, it may naturally generate English or Chinese because those dominate BLOOM's training corpus. Without a baseline generation run with no neuron manipulation, or a control with random neurons set to their medians, the language shift cannot be attributed to the semantic content of the selected neurons. Additionally, 'concept-relevant' is only assessed qualitatively via example outputs (Table 1); no quantitative measure of concept presence is reported. Thus, the abstract's 'Remarkably' claim is not yet supported. This is separate from, but compounds, the reader's concern about parallel-translated data in Section 6: even if the alignment measures are valid, the causal demonstration is uncontrolled.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper investigates how multilingual language models (BLOOM-560M, BLOOM-7B1, and a custom XGLM-based model) evolve from language-specific to cross-lingual representations during pre-training. Using linear probing, the authors show that language identity becomes less decodable in middle layers over training. They then compute concept-specific 'expert neuron' scores (average precision) for 200 WordNet concepts across languages, using NLLB-translated parallel sentences, and measure cross-lingual alignment via correlation, mutual information, and top-500 neuron overlap. All three measures increase during training, and the effect is replicated across model scales. A layer-wise analysis localizes the strongest cross-lingual overlap in middle layers. Finally, a neuron-steering experiment manipulates concept neurons derived from Spanish/Chinese data and shows a shift from source-language to English generation at later checkpoints, which the authors interpret as evidence that aligned neurons encode language-independent semantic content.","tokens_in":76157,"tokens_out":4163,"duration_ms":50242,"significance":"If the central claim holds, this would be a valuable mechanistic account of cross-lingual generalization: shared concept neurons forming a language-independent semantic layer during pre-training, with practical implications for controllable generation and for understanding high-resource language bias. The paper's strengths are its longitudinal design, the use of three independent alignment measures, replication across BLOOM-560M, BLOOM-7B1 and a custom model, and the public release of checkpoints and code. The layer-wise analysis (Section 7) is a thoughtful attempt to separate lexical overlap from semantic sharing. However, the headline claim rests on a generation experiment that currently lacks the control conditions needed to attribute the observed language shift to the manipulated concept neurons, and the alignment measures are computed on machine-translated parallel data, leaving a translation-induced artifact as a plausible alternative explanation. These issues are addressable, but they are load-bearing for the strongest interpretation.","major_comments":[{"comment":"The steering experiment described in Section 8 and Appendix E has no no-manipulation baseline and no random-neuron control. The protocol sets the top-500 concept neurons to concept-specific median values, prompts only with a BOS token, and measures the language of generated text. The shift from Spanish at step 10k to English at step 400k could simply reflect the model's default output distribution from an unconstrained prompt, especially given English's dominance in BLOOM's training corpus; the paper itself invokes corpus prominence to explain Chinese generations but does not apply the same reasoning to English. Without a baseline (e.g., generation from BOS with no neuron manipulation) and a control with the same number of random neurons set to their medians, the language shift cannot be attributed to the semantic content of the selected neurons. Additionally, 'concept-relevant' is supported only by qualitative examples in Table 1; a quantitative measure (classifier, human rating, or concept-specific metric) is needed to show that the manipulated neurons specifically evoke the target concept rather than generic fluent text. This is load-bearing for the abstract's claim that aligned neurons 'encode generalized semantic representations, independent of the original language encoding.'","section":"Section 8 / Appendix E"},{"comment":"The expert neuron scores are computed on the same English sentences machine-translated by NLLB into each target language, creating parallel corpora with identical semantic content. Cross-lingual alignment of AP vectors could therefore be inflated by translationese, shared subwords, cognates, or the trivial preservation of content across languages, rather than by the model's formation of language-independent abstractions. The layer analysis in Section 7 partially separates subword overlap from semantic overlap by showing that alignment is strongest in middle layers, but it does not control for the fact that the underlying text is semantically identical by construction. To support the compression conclusion, the authors should add a control: e.g., compute alignment on non-parallel monolingual corpora for the same concepts, randomly permute concept labels within one language before computing cross-lingual correlation, or compare the observed overlap against a shuffled-neuron null distribution. Without such a control, the increasing alignment is consistent with the model simply reflecting the parallel data's shared content.","section":"Section 6, Data"},{"comment":"The top-k overlap measure is reported only for k=500, and the conclusion that 'approx. 1/6 of the top 500 concept-selective neurons are shared' is given without sensitivity analysis. Since the degree of overlap trivially decreases with k, and the choice of k is a free parameter, reporting at least one other k (e.g., k=100 and k=1000) would establish that the temporal trend is not an artifact of the specific threshold.","section":"Section 6, Neuron Overlap (k-sensitivity)"}],"minor_comments":[{"comment":"The three panels of Figure 4 share the x-axis 'Step' but the y-axis labels and units are not visible in the text; please label them explicitly (e.g., Pearson r after Fisher Z, overlap proportion, MI in nats).","section":"Section 6, Figure 4"},{"comment":"The phrase 'approx. 1/6 of the top 500 concept-selective neurons are shared between any pair of languages' is ambiguous; it should state that the mean pairwise overlap proportion is approximately 1/6, i.e., about 83 of 500 neurons on average.","section":"Section 6, Neuron Overlap"},{"comment":"The Limitations section does not mention the absence of control conditions in the generation experiment or the reliance on translated parallel data; these are the two main threats to the paper's central claim and should be acknowledged.","section":"Limitations"},{"comment":"The claim of being 'the first to analyze the development of cross-lingual semantic generalization during pre-training' (contribution ii) should be tempered in light of Blevins et al. (2022), who analyze cross-lingual pretraining dynamics even if with different tasks; suggest rephrasing to 'the first to our knowledge at the level of individual concept neurons in decoder-only models.'","section":"Section 2, Related Work"},{"comment":"The notation 'zc,l m = {zc,l m,i}N i=1' is hard to parse; define zc,l m,i as the activation of neuron m on the i-th sentence of concept c in language l.","section":"Section 6, Expert score definition"}],"recommendation":"major_revision","confidential_remarks":"The paper's observational analysis of neuron alignment is a solid contribution and likely publishable after revision. The main issue is the overinterpretation of the generation experiment: the 'Remarkably' claim in the abstract is not supported without baselines. This is fixable—add no-manipulation and random-neuron controls plus a quantitative concept-relevance measure, and temper the abstract accordingly. The parallel-data concern is also addressable with null controls. I recommend major_revision rather than rejection because the core measurements and trends are useful and the revision is within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe headline: this is a careful, replicable study of how concept-selective neurons become shared across languages during pre-training. The core empirical trend — correlation, mutual information, and top-k overlap all increasing over training, in three model sizes, with code and data released — is real and worth taking seriously.\n\nWhat is new: applying Suau et al.'s neuron-expertise method to pre-training checkpoints and tracing cross-lingual alignment of concept neurons is a genuine extension. Blevins et al. probed tasks; this paper zooms into individual semantic concepts. The layer analysis is the strongest part: early layers show surface-level lexical overlap, middle layers (10–17) develop the cross-lingual overlap, and the final layers stay language-specific. That pattern supports the compression story without relying on any single metric.\n\nWhere it gets soft: Section 8's steering experiment is underspecified. The protocol sets top-500 Spanish-derived concept neurons to their median activations, prompts with only <s>, and observes English-dominant output at late checkpoints. There is no no-manipulation baseline and no random-neuron control, so the language shift could simply be BLOOM's default distribution, since English and Chinese dominate its corpus. \"Concept-relevant\" is judged from qualitative examples (Table 1), not a quantitative measure. So the abstract's \"Remarkably... independent of the original language encoding\" overstates what the experiment shows. That is fixable — add the two controls and a concept-presence metric — but the claim should be conditional until then.\n\nThe parallel-translated data concern is real but lesser. NLLB translations could introduce lexical or cognate overlap, but the layer-wise analysis partially separates lexical from semantic overlap, and the trend replicates across three models and three measures. I would not call it a fatal confound.\n\nBottom line: the paper's correlational core is solid, the causal gloss is not yet supported. This deserves a serious referee — the controls can be added in revision. If you work on multilingual interpretability, this is a useful citation for the alignment trend, just not for the steering result.\n\nRecommendation: send to peer review, and require control conditions for Section 8.","headline":"Solid evidence for growing cross-lingual neuron alignment during pre-training; the steering experiment oversells the causal claim, but the correlational core holds up.","tokens_in":76671,"tokens_out":2405,"would_cite":true,"duration_ms":27589,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Probing shows MLLMs build language-free concept neurons","keywords":["cross-lingual generalization","compression hypothesis","multilingual language models","mechanistic interpretability","concept neurons","pre-training dynamics","cross-lingual alignment","neuron steering"],"falsifier":"Take two sets of human-written, non-parallel sentences about the same 200 WordNet concepts in, say, Spanish and English, measure top-500 concept-neuron overlap in a late BLOOM checkpoint, and compare it to the overlap measured on the NLLB-translated parallel corpus used in the paper; if the overlap drops to chance or near-zero on the independent texts, the compression-to-shared-neurons conclusion would be an artifact of translated input. A second check: rerun the steering experiment with the top 500 expert neurons replaced by 500 random neurons at the same layers, and if random-neuron activation also shifts generated language toward English, the effect is not concept-specific.","tokens_in":75709,"feed_emoji":"🧠","tokens_out":4578,"duration_ms":46127,"temperature":0.7,"pith_summary":"The paper argues that multilingual language models develop a language-independent semantic layer during pre-training, driven by compression pressure that forces the model to reuse the same neurons for the same concepts across languages. The authors track individual concept expert neurons over training checkpoints in BLOOM models and a custom model, showing that neurons initially tied to one language gradually align across languages. The decisive behavioral evidence is that activating concept neurons identified from Spanish text makes the model generate coherent English text about that concept, showing the neurons encode the concept, not the language. If right, this reframes cross-lingual transfer as a byproduct of compression rather than of shared subwords or linguistic similarity.","feed_headline":"Probing shows MLLMs build language-free concept neurons","feed_subtitle":"Neurons found from Spanish text steer the model to generate coherent English, showing the code is the concept, not the language.","key_machinery":"The central object is the concept expert neuron: an MLP neuron whose activation pattern, scored by average precision against positive and negative sentence sets for a WordNet sense, reliably predicts the presence of a concept. The paper defines per-language expert score vectors over all neurons, then measures cross-lingual alignment via Pearson correlation (averaged with Fisher's Z transform), mutual information (via k-nearest-neighbor entropy estimation), and direct overlap of top-500 neuron sets. The layer-wise version of the overlap measure separates lexical (early-layer) from semantic (middle-layer) sharing, and the generation steering protocol, setting top expert neurons to their median concept activation and sampling with no prompt tokens, is the causal test that carries the argument.","core_discovery":"The central claim is that cross-lingual generalization in decoder-only multilingual language models is realized by concrete shared neurons: individual MLP neurons that respond to the same semantic concept across languages, emerging and strengthening during pre-training. The authors show that by late training, roughly one sixth of the top 500 concept-selective neurons are shared between any pair of languages, that this overlap peaks in middle layers (10-17) where language-identification ability is weakest, and that the trend replicates from a 257M-parameter model to BLOOM-7B1. The most direct evidence is causal: setting the top 500 concept expert neurons (identified from Spanish or Simplified Chinese text) to their concept-median activations, with no language cue given, biases the model to generate semantically coherent text, and in later checkpoints that text is overwhelmingly English. The authors conclude that aligned concept neurons encode generalized semantics independent of the surface language, and that their emergence is consistent with the information bottleneck and compression hypothesis.","pith_inferences":["If the compression account is causal rather than correlational, deliberately increasing model capacity should delay or reduce cross-lingual neuron sharing, a testable prediction the paper does not run.","The reliance on NLLB-translated parallel data leaves open that translationese, calques, or cognate retention inflate the measured overlap; an orthogonal check would measure overlap on independently written same-concept sentences in each language.","The English-output steering result may partly reflect the model's output prior favoring high-resource languages, since Swahili-derived neurons never yield Swahili text; a cleaner demonstration would control for output-language priors.","Extending the concept inventory from concrete nouns to verbs, adjectives, or relational concepts could reveal whether shared encoding is a general property of lexical semantics or specific to the sampled word senses."],"forward_implications":["If shared concept neurons are a general feature of multilingual models, zero-shot cross-lingual transfer should become predictable from neuron overlap measured before fine-tuning, rather than from downstream task scores.","The layer-resolved overlap curve suggests that interventions targeting middle layers should be the route for language-agnostic concept control in generation.","The same measurement pipeline can be applied to other model families with released checkpoints to test whether the compression-to-sharing trajectory is universal or architecture-dependent.","Concept steering becomes a practical technique: a concept learned from one language can be used to elicit that concept from the model while it generates in another language.","The observed English and Chinese dominance in steering outputs implies that shared representations are not neutral; the compressed layer may be anchored to the most frequent pre-training languages."],"supporting_citations":[{"why":"Contributes the concept expert neuron identification method (average precision scoring) and the neuron-activation manipulation used in the generation experiments.","marker":"Suau et al. (2022)"},{"why":"Supplies ONESEC, the source of WordNet-sense-annotated sentences from which the 200 concept datasets are sampled.","marker":"Scarlini et al. (2019)"},{"why":"Provides NLLB 1.3B, the translation system that creates the parallel multilingual versions of the concept datasets.","marker":"Costa-jussà et al. (2022)"},{"why":"Provides the BLOOM family whose public training checkpoints are the main subject of the tracing analysis.","marker":"BigScience Workshop (2022)"},{"why":"Underpins the k-nearest-neighbor entropy estimation used for the mutual information alignment measure.","marker":"Kraskov et al. (2004)"},{"why":"Supplies the language detection library used to classify the language of the steering-generated texts.","marker":"Shuyo (2010)"}],"fun_headline_variants":["MLLMs develop shared neurons that encode concepts across languages","Neurons found in Spanish text steer English generation, showing language-free concepts","Cross-lingual generalization emerges from shared concept neurons","Concept neurons align across languages during MLLM pre-training"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire alignment measurement rests on parallel data: the English concept sentences are machine-translated into every target language with NLLB, so the growing top-neuron overlap across languages could in principle reflect translation artifacts like shared subwords, cognates, or translationese rather than a genuinely shared semantic code.","fun_headline_variants_meta":{"raw":{"variants":["MLLMs develop shared neurons that encode concepts across languages","Neurons found in Spanish text steer English generation, showing language-free concepts","Cross-lingual generalization emerges from shared concept neurons","Concept neurons align across languages during MLLM pre-training"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000669,"raw_usage":{"total_tokens":3011,"prompt_tokens":869,"completion_tokens":2142,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":485,"completion_tokens_details":{"reasoning_tokens":2072}},"tokens_in":485,"tokens_out":2142,"duration_ms":17205,"temperature":1.0,"reasoning_tokens":2072,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T11:37:14.245362+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take two sets of human-written, non-parallel sentences about the same 200 WordNet concepts in, say, Spanish and English, measure top-500 concept-neuron overlap in a late BLOOM checkpoint, and compare it to the overlap measured on the NLLB-translated parallel corpus used in the paper; if the overlap drops to chance or near-zero on the independent texts, the compression-to-shared-neurons conclusion would be an artifact of translated input. A second check: rerun the steering experiment with the top 500 expert neurons replaced by 500 random neurons at the same layers, and if random-neuron activation also shifts generated language toward English, the effect is not concept-specific.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Contributes the concept expert neuron identification method (average precision scoring) and the neuron-activation manipulation used in the generation experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the language detection library used to classify the language of the steering-generated texts."}],"review_version":1}