{"id":"fced32e8-9afe-45ad-8db6-4f1747c45afa","arxiv_id":"2506.18011","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Minimal single-token substitutions in BERT show that common tokens induce small embedding shifts, rare tokens induce larger shifts, and perturbation effects grow with layer depth, which the authors interpret as support for early-layer explanations.","lead":"This paper swaps single words in movie reviews for their closest lookalike word and measures how much the model's internal representations shift. It finds that common words barely move the representation, rare words move it more, and the effects grow in deeper layers, which the authors read as support for using early layers in model explanations.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper's core inference from layer-wise L2 displacement to 'input intermixing' and 'early layers as explanation proxies' is unestablished: the metric is unnormalized, has no random baseline, and no faithfulness check.","rationale":"The reader's CONDITIONAL verdict is appropriate. My stress-test pass did not find an internal inconsistency or a fraud-level flaw; the main issue is that the paper's central interpretability claim goes beyond what the reported quantity can support. The load-bearing step is in Section 3.3: Figure 3 shows growing mean L2 distance between original and minimally perturbed hidden states, and this is interpreted as evidence that input information becomes intermixed and that early layers are better explanation proxies. That interpretation requires an equivalence between sensitivity/robustness and faithfulness that the paper never establishes. Also, the raw distance is not normalized and no random baseline is provided, so even the trend itself may be a scale or amplification effect. My proposed check is an external faithfulness evaluation per layer; if it does not reproduce the expected depth trend, the central claim should be treated as speculative. The conclusion aligns with prior work, which reduces novelty but does not make it wrong. Since the reader already flagged the same assumption and recommended conditions, the verdict remains unchanged.","tokens_in":7251,"tokens_out":7151,"duration_ms":81727,"concrete_test":"Take the same 500 sequences and minimal substitutions; for each layer l (embedding and layers 1..12), compute a faithfulness score for explanations derived from h_l by ranking tokens with gradient times activation at that layer and measuring the drop in predicted positive-class probability when the top-k tokens are removed versus bottom-k removal, or AUPRC over token-level erasure. If the faithfulness of early-layer explanations is not higher than later-layer ones, or no better than random ranking, the paper's 'validates early layers as explanation proxies' conclusion is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract and Section 5 claim that the experiments 'validate' the assumption that first layers are proxies for explanations. The only evidence for this in Section 3.3 is Figure 3, a plot of mean L2 distance between hidden states of an input and its minimally perturbed variant as a function of layer. For that plot to support the claim, the distance must be a measure of information preservation and interpretability. The paper provides no independent justification for that equivalence, and the measure is not scale-invariant: hidden-state norms grow with depth in BERT, and later layers amplify any input variation, so an increasing unnormalized distance can occur even when the relative information change is constant or when the perturbation is semantically vacuous. No random-substitution control is reported, so one cannot tell whether the trend is specific to minimal perturbations or is the generic sensitivity of deep layers. The qualitative closest-token degeneration in Section 3.3 is also compatible with embedding anisotropy rather than token intermixing. The conclusion that early layers are 'more robust to perturbations' and therefore better explanation proxies is read off the same quantity being measured; no faithfulness or explanation-quality metric is ever computed.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a minimal token perturbation framework for probing the embedding space of a BERT model fine-tuned on IMDb sentiment analysis. A token is replaced by its nearest neighbor under cosine similarity in the embedding space, and the resulting shift is measured in the embedding and in the hidden states of successive layers. The experiments report that punctuation and frequent tokens produce minimal shifts, that rare tokens lead to larger embedding shifts, and that the mean L2 distance between original and perturbed hidden states increases with depth except for the final layer. From these observations, the authors argue that input information becomes increasingly intermixed in deeper layers and that early layers are better proxies for model explanations.","tokens_in":7453,"tokens_out":4659,"duration_ms":47468,"significance":"If established, the proposed perturbation-based analysis would be a simple and inexpensive tool for studying information propagation in Transformers, and it would provide direct evidence for the common assumption that early-layer representations are more interpretable. The manuscript is clearly written, the experimental setup is simple enough to reproduce, and the qualitative closest-token evolution example in Section 3.3 is illustrative. However, the central claim is not currently supported by the evidence: the main metric lacks control conditions and normalization, and no independent faithfulness or explanation-quality measure is computed. The contribution is thus at the level of a hypothesis-generating empirical observation rather than a validated interpretability result.","major_comments":[{"comment":"The increasing mean L2 distance with depth is presented as evidence that 'input information is increasingly intermixed' and that early layers are better proxies for explanations. This inference is not justified without control conditions. A random-token-substitution baseline and a maximal-perturbation baseline are needed to determine whether the trend is specific to minimally perturbed tokens or is a generic property of deep-layer sensitivity to any input change. Additionally, because hidden-state norms grow with depth in BERT-like models, the unnormalized L2 distance can increase even when the relative information change is constant; the paper should report normalized distances or per-layer norm ratios.","section":"§3.3, Figure 3"},{"comment":"The load-bearing premise is that robustness to perturbations in hidden-state space is equivalent to faithfulness or interpretability. The paper states in Section 3.3 that early layers 'appear to be more robust to perturbations' and then uses this to support their use as explanation proxies, and Section 5 repeats the claim as a validation. This conclusion is read off the same quantity that is being measured. An independent metric is required, such as token identifiability rates following Brunner et al. (2019), output-level attribution, or a probing classifier's performance, to test whether early layers actually provide better explanations.","section":"§3.3 and §5"},{"comment":"The replacement token is selected by minimizing cosine distance in the embedding space, but the propagation analysis in Section 3.3 measures unnormalized L2 distances between hidden states. The relationship between these two metrics is not established; the 'minimal' perturbation in the embedding space need not be the minimal perturbation in terms of L2 displacement at deeper layers. The authors should either justify the compatibility of the two metrics or compare results with perturbations selected by L2 distance in the embedding space.","section":"§2 and §3.3"},{"comment":"The commonness–distance regression is the quantitative support for the claim that rare tokens induce larger shifts, but the paper reports only a decreasing trend with a 'tight 95% confidence interval' and does not give the regression slope, intercept, R², or a significance test. The correlations in Section 3.1 are likewise reported without confidence intervals. Please report these statistics and clarify how the binning procedure, including the footnote that some bins contain fewer than 50 unique tokens, affects the regression.","section":"§3.2, Figure 2"}],"minor_comments":[{"comment":"The phrase 'the frequency of which tokens yield to minimal shifts' should be 'the frequency with which tokens yield minimal shifts'.","section":"Abstract"},{"comment":"The explanation that frequent tokens such as punctuation and numbers are minimally perturbing is plausible, but it would be strengthened by a control comparison with random frequent-token substitutions rather than only the minimal-perturbation selection.","section":"§3.1"},{"comment":"The binning description is unclear: 'select 50 distinct sentences per bin' is immediately qualified by 'higher commonness bins contain fewer samples.' Please specify exactly how tokens, sentences, and commonness bins are assigned and how the regression weights unequal bin sizes.","section":"§3.2, footnote 2"},{"comment":"The sentence 'the first two tokens lead to smaller shifts... the first two tokens are indeed the most similar in terms of meaning' is ambiguous; it refers to the top-5 least-impactful substitution ranks shown in Figure 3, not to token positions in the input. Please rewrite to avoid confusion.","section":"§3.3"},{"comment":"The closest-token output for the last hidden state contains apparent formatting artifacts such as 'scenerycuit' and 'siskill'; please clean or reformat the example.","section":"§3.3"},{"comment":"The conclusion states that early layers 'preserve the input more faithfully.' Given the concerns above about the perturbation-distance metric, 'faithfully' should be qualified as 'faithful with respect to the perturbation-distance measure' unless an independent faithfulness evaluation is added.","section":"§5"},{"comment":"A statement on code and data availability would improve reproducibility, especially since the experiments are described as runnable in under 10 minutes.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's central interpretability claim is not yet supported by the experiments as designed. The missing control experiments and independent faithfulness evaluation are substantial but addressable within the scope of the paper. I would recommend requiring these additions before publication, and also asking the authors to clarify the novelty relative to Brunner et al. (2019), whose token-identifiability analysis already studies information flow across BERT layers."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This is a small, readable empirical study. What's actually new is the specific protocol: replace a token with its cosine-nearest neighbor, then measure the L2 shift in embedding space and across BERT layers. That's a straightforward assembly of existing techniques, and the headline findings — rare tokens shift more, hidden states drift from token identities in deeper layers — are already in the papers the authors cite (Piantadosi et al. 2011; Brunner et al. 2019; Zhang et al. 2022). So the contribution is incremental, not a breakthrough.\n\nCredit where it's due: the paper is clearly written, the experiments are easy to follow, and the limitations section is honest about the single-task, single-architecture scope. The authors also cite the relevant prior work correctly. That matters.\n\nThe soft spot is the interpretive leap. The abstract and Section 5 say the experiments \"validate\" the assumption that first layers are proxies for explanations. The evidence for that is Figure 3: mean L2 distance between perturbed and original hidden states grows with depth. But that distance isn't normalized, there's no random-replacement or maximal-perturbation baseline, and no faithfulness or explanation-quality metric is computed. The increasing unnormalized distance could simply reflect growing hidden-state norms or generic sensitivity of deep layers, not token intermixing in the sense the authors claim. The conclusion is largely read off the same quantity being measured. The regression in Figure 2 also lacks significance tests or effect sizes, and the correlations in Section 3.1 are reported without intervals. These are addressable issues, not fatal errors, but they mean the \"validates\" language is too strong.\n\nWho is this for? Readers working on interpretability diagnostics might find the protocol worth trying, and the paper is a fine example of how to state limitations clearly. But it doesn't settle a debate or change practice beyond reinforcing existing guidance about early layers.\n\nMy recommendation: send it to peer review, but condition acceptance on major revisions. The authors should add a random baseline, normalize the embedding distances (e.g., by hidden-state norm), provide significance tests, release code and data, and soften the conclusions to \"consistent with\" rather than \"validate.\" The core idea is reasonable, but the evidence doesn't yet carry the weight the paper puts on it.","headline":"A small, readable empirical study that recombines known ingredients into a new perturbation protocol, but whose central claim about early layers as explanation proxies overreaches the evidence it provides.","tokens_in":8017,"tokens_out":1390,"would_cite":false,"duration_ms":16403,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Minimal token perturbations show that a BERT model's early layers stay closer to the original input than its deeper layers, supporting the use of early-layer representations as explanations.","keywords":["minimal token perturbations","embedding space","Transformer interpretability","token frequency","layer-wise propagation","BERT","hidden state shifts","token identifiability"],"falsifier":"Take the same 500 sequences and, for the top-k tokens flagged by each layer's closest-token retrieval, erase or replace those tokens, then measure the change in the model's sentiment prediction. If attributions from deeper layers track the prediction change at least as well as early-layer ones, the claim that first layers are better explanation proxies would be contradicted; alternatively, recomputing the layer-wise perturbation curve with per-layer normalized distances would show whether the increasing trend survives after removing scale effects.","tokens_in":7009,"feed_emoji":"🔤","tokens_out":6187,"duration_ms":57670,"temperature":0.7,"pith_summary":"Using a fine-tuned BERT sentiment model, the paper replaces a single token in a movie review with its nearest embedding neighbor (by cosine similarity) and measures how far the sentence's hidden states move. It finds that the tokens whose swaps shift the embedding least are overwhelmingly common words, punctuation, and numbers, while rare tokens produce larger shifts. Tracking the same minimal perturbations through the twelve transformer layers, the distance between perturbed and original hidden states grows with depth, except in the final layer, which the authors attribute to task-specific fine-tuning. The paper takes these trends as evidence that input information becomes increasingly intermixed in deeper layers and concludes that early-layer representations are more faithful to the input and therefore safer proxies for model explanations.","feed_headline":"Mild token swaps show BERT's early layers mirror the input best","feed_subtitle":"Replacing a token with its nearest neighbor grows more disruptive in deeper layers, supporting early-layer explanations.","key_machinery":"The central mechanism is the minimal token perturbation: for each token in an input sentence, replace it with the vocabulary token whose embedding has the highest cosine similarity, then measure the shift $\\|E(x)-E(x')\\|$ produced in the embedding space under the $\\ell^1$, $\\ell^2$, and $\\ell^\\infty$ norms. The same replacement is passed through the encoder to compute hidden-state distances at each layer. This construction is what allows the paper to connect token frequency, embedding sensitivity, and layer depth in one interpretability probe.","core_discovery":"On the paper's own terms, the central discovery is that minimal token perturbations expose a layer-wise fidelity gradient: BERT's embedding and early hidden states remain close to the original input after a small token replacement, while deeper hidden states drift progressively further away. The paper argues this validates the widely used assumption that first-layer representations can stand in for model explanations. It also establishes an empirical regularity relating token frequency to representational sensitivity: frequent tokens such as '.', ',', and numbers yield the smallest embedding shifts, whereas rare tokens yield the largest. The authors attribute this to common tokens carrying less semantic load and having closer semantic neighbors, while rare tokens are underdetermined in the embedding space.","pith_inferences":["The paper does not separate the growing distance between perturbed and original hidden states from the growth in hidden-state scale across layers; normalizing per-layer distances before comparing would test whether the trend reflects information mixing or simply larger activations.","A natural extension the authors do not draw is to use the same minimal perturbations to measure causal importance: if rare tokens produce larger embedding shifts, they may also be the tokens whose removal most changes the sentiment prediction, a claim that erasure experiments could verify.","The closest-token retrieval used to illustrate deep-layer drift could itself be turned into a quantitative faithfulness metric: computing token-recovery accuracy at each layer and correlating it with the perturbation distance would connect this work directly to the identifiability literature it cites."],"forward_implications":["If early layers preserve input information better, interpretability methods that read attributions or nearest-token associations from the first layers inherit that fidelity, while deeper-layer readings should carry a caveat.","Rare-token analysis is the sensitive spot: because rare tokens move the representation most, explanations built around them are the ones most likely to change under small input edits.","The final layer's deviation from the trend means task-specific layers should be treated as output machinery, not as faithful input encodings.","Because all three norms rank the minimally perturbing tokens almost identically, the choice of distance metric is unlikely to change which tokens an explanation pipeline should inspect."],"supporting_citations":[{"why":"Supplies the token-identifiability result that identifiability degrades with depth and that cosine distance recovers tokens, which the paper extends to perturbation shifts.","marker":"[Brunner et al., 2019]"},{"why":"Defines the BERT architecture that all experiments are run on.","marker":"[Devlin et al., 2019]"},{"why":"Provides the IMDb movie-review corpus and sentiment labels used to fine-tune the model and to select test sentences.","marker":"[Maas et al., 2011]"},{"why":"Grounds the claim that rare words carry more information per token, which the paper uses to explain larger shifts for rare tokens.","marker":"[Piantadosi et al., 2011]"},{"why":"Supports the observation that rare words are underrepresented in BERT-like embeddings, explaining why their replacements perturb the space more.","marker":"[Schick and Schütze, 2020]"},{"why":"Supplies the prior result that first layers better preserve information, which the perturbation trend is said to confirm.","marker":"[Zhang et al., 2022]"},{"why":"Frames the categorization of interpretability methods and the common assumption that first layers can serve as explanation proxies.","marker":"[Fantozzi and Naldi, 2024]"}],"fun_headline_variants":["Tiny token tweaks reveal why early layers explain best","Rare tokens cause larger embedding shifts than frequent ones","Probing BERT: early layers hold input, deeper layers mix it","First-layer representations survive minimal token swaps best","Token perturbation probe: fidelity fades going deeper in BERT"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The conclusion rests on treating the $\\ell^2$ distance between the hidden states of an input and its minimally perturbed version as a direct measure of how faithfully the layer preserves input information, so the claim that early layers are better explanation proxies collapses if that identification is wrong.","fun_headline_variants_meta":{"raw":{"variants":["Tiny token tweaks reveal why early layers explain best","Rare tokens cause larger embedding shifts than frequent ones","Probing BERT: early layers hold input, deeper layers mix it","First-layer representations survive minimal token swaps best","Token perturbation probe: fidelity fades going deeper in BERT"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001018,"raw_usage":{"total_tokens":4213,"prompt_tokens":778,"completion_tokens":3435,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":394,"completion_tokens_details":{"reasoning_tokens":3354}},"tokens_in":394,"tokens_out":3435,"duration_ms":26820,"temperature":1.0,"reasoning_tokens":3354,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:56:11.958567+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the same 500 sequences and, for the top-k tokens flagged by each layer's closest-token retrieval, erase or replace those tokens, then measure the change in the model's sentiment prediction. If attributions from deeper layers track the prediction change at least as well as early-layer ones, the claim that first layers are better explanation proxies would be contradicted; alternatively, recomputing the layer-wise perturbation curve with per-layer normalized distances would show whether the increasing trend survives after removing scale effects.","supporting_citations":[{"cited_title":"BERT: Pre-training of deep bidirectional transformers for language understand- ing","cited_arxiv_id":null,"evidence_quote":"Defines the BERT architecture that all experiments are run on."},{"cited_title":"Learning word vectors for sentiment analysis","cited_arxiv_id":null,"evidence_quote":"Provides the IMDb movie-review corpus and sentiment labels used to fine-tune the model and to select test sentences."},{"cited_title":"Word lengths are optimized for efficient communication.Proceedings of the National Academy of Sciences, 108(9):3526–3529,","cited_arxiv_id":null,"evidence_quote":"Grounds the claim that rare words carry more information per token, which the paper uses to explain larger shifts for rare tokens."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supports the observation that rare words are underrepresented in BERT-like embeddings, explaining why their replacements perturb the space more."},{"cited_title":"The explainability of transformers: Current status and directions.Computers, 13(4):92,","cited_arxiv_id":null,"evidence_quote":"Frames the categorization of interpretability methods and the common assumption that first layers can serve as explanation proxies."}],"review_version":2}