{"id":"c6062716-668f-4b08-b3b0-69731d2f7a18","arxiv_id":"2501.11041","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Steering a handful of attention heads with a bias vector derived from consistent prompt pairs improves semantic consistency of LLaMA-2-7B on paraphrased NLU and NLG tasks.","lead":"This paper proposes a lightweight method to make large language models give more consistent answers when the same question is phrased in different ways. Instead of expensive fine-tuning, it finds and adjusts a small set of internal attention heads, which takes far less compute and also reveals which parts of the model matter.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper's component selection rests on a correlational probe; without activation-patching validation, the claim that the edited heads cause the inconsistency is unsupported.","rationale":"Good-faith reading: the paper's contribution is a cheap, interpretable activation-steering recipe; the empirical results include honest SFT comparisons and useful OOD/ablation checks. The central argument, however, hinges on locating the components that 'cause' inconsistency (Section 4.2). A linear probe trained on consistency labels derived from the model's own outputs can score highly simply by reading off information that is already causally downstream in the same network; this is a known limitation of probing (Alain and Bengio, 2016), and the paper does not attempt a causal intervention before editing. Activation patching is the standard, direct way to test whether a component's state is causally important for a behavior. If the patch-based ranking disagrees with the probe ranking, the interpretability claim is not supported. I therefore agree with the reader's weakest assumption. The existing ablations (Table 5) are helpful but only show that the chosen components and directions outperform random ones; they do not establish causality. Verdict stays conditional: the method may be useful, but its central interpretability claim needs either causal evidence or a softened framing.","tokens_in":11712,"tokens_out":8083,"duration_ms":94044,"concrete_test":"On a held-out set of prompt pairs from the same construction, perform causal activation patching for each candidate component: for each pair ([p,q], c) with c=0, run the model with the component's last-token hidden state at p replaced by the corresponding state from q (and vice versa), and measure how often the pair's consistency label flips to 1; do the same for c=1 pairs swapping to c=0. Rank components by median patch effect, and compare this ranking with the linear-probe ranking used in Section 4.2 (Eqs. 3-6). If the top-K sets overlap at chance level, or if patching in the editing direction does not increase consistency, then the probe-selected components are not causally responsible, and the main interpretability claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing assumption is in Section 4.2: a component whose hidden states let a linear classifier predict the consistency label c is assumed to 'actually cause' the model's semantic inconsistency. This is never tested causally. Because c is computed from the target LLM's own outputs, the last-token representation of any late-layer component is informationally correlated with c; a high probe accuracy on the locate set may only mean the component carries signal about the eventual prediction, not that intervening on it changes consistency. The only interventions are the additive biases of Eqs. (7)-(9), and the ablations in Table 5 show that random components and random directions reduce performance, but that contrast does not establish that the probe-selected components are the causal drivers. If this assumption fails, the paper's central interpretability claim—that it 'precisely diagnose[s] the key components that contribute to semantic consistency'—is unsupported; the method may still shift outputs, but not because it edits consistency-specific causes.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a locate-then-edit method to improve the semantic consistency of LLMs. It first constructs prompt pairs with equivalent semantics and labels them as consistent or inconsistent based on the target model's own outputs. It then trains linear probes on the hidden states of attention heads and MLPs to predict these consistency labels, selects the top-K components with highest probe accuracy, and adds a bias to those components along the difference between the mean activation of consistent samples and the mean activation of all samples. The method is evaluated on newly constructed NLU benchmarks (RobustSST2, RobustMRPC, RobustBOOLQ) and on NLG datasets (PopQA), reporting improved standard deviation of accuracy (for NLU), mean pairwise cosine similarity (for NLG), and task accuracy compared to the unedited Llama2-7B-Chat model. The paper also includes ablations on the number of edited components, the bias strength, random component/direction controls, out-of-domain generalization, and a comparison with supervised fine-tuning.","tokens_in":11849,"tokens_out":4410,"duration_ms":46251,"significance":"If the proposed method works as claimed, it offers a cheap, inference-time alternative to fine-tuning for improving prompt robustness, and the interpretability framing could provide insight into where inconsistency originates in transformer LMs. The paper contributes three new NLU benchmarks for semantic-consistency evaluation, which could be useful to the community. However, the current evidence has two load-bearing gaps: the NLU consistency metric does not directly measure the paper's own definition of per-instance consistency, and the causal interpretability claim (that probe-selected components are the causes of inconsistency) is not validated. These issues need to be addressed before the central claims can be accepted.","major_comments":[{"comment":"The NLU consistency metric, the standard deviation of accuracy across instruction templates, does not measure the paper's own definition of consistency (same answer for the same instance across paraphrases, as stated in Figure 1 and the Introduction). Standard deviation of accuracy is a group-level aggregate; two models can have identical per-instance consistency yet different standard deviations. The reported improvements in 'semantic consistency' for NLU tasks therefore do not directly support the central claim. Please report a per-instance consistency metric, such as the fraction of instances for which all paraphrases yield the same answer, or the average pairwise agreement, and present it alongside the standard deviation.","section":"§6.2"},{"comment":"The locating step trains linear probes to predict the consistency label c from component hidden states, and then assumes that components with high probe accuracy 'actually cause' the semantic consistency problems. This is a correlational inference: a component whose hidden state is informative about c may simply be carrying information about the model's eventual prediction, not driving the inconsistency. The ablations in Table 5 (random components and random directions) show that the selected components and directions matter, but they do not test whether the selected components are causally responsible. Please add activation-patching or interchange-intervention experiments to validate the causal role of the located components, or substantially soften the interpretability claim that the method 'precisely diagnose[s] the key components'.","section":"§4.2"},{"comment":"The NLU test-set construction selects prompts based on the target model's prediction diversity: if the 6 paraphrases yield N distinct outcomes, one prompt from each distinct outcome is included. This selection procedure intentionally oversamples instances where the model is inconsistent and may interact with the editing method differently than with the baseline, potentially exaggerating the measured improvement. Please analyze the distribution of N before and after editing, and report results on a randomly constructed test set (without diversity-based selection) to ensure the reported consistency gains are not an artifact of the selection rule.","section":"§5"},{"comment":"The main experiments use alpha=5.0, but the ablation in Table 4 shows that alpha=3.0 yields better results on RobustSST2 (standard deviation 2.98 vs. 4.54). The paper does not specify how alpha and the component count K are chosen for each dataset, nor whether these hyperparameters were selected on the test sets. This is important because the reported improvements may be the result of tuning on the evaluation data. Please clarify the hyperparameter selection procedure and report the sensitivity of the results to K and alpha across all datasets.","section":"§4.3 and §6.5.2"}],"minor_comments":[{"comment":"Equation (4) contains an extra closing parenthesis in the definition of f(h_{i,j}, p, q); the notation should be f(h_{i,j}, p, q) = [h^p_{i,j,last}; h^q_{i,j,last}].","section":"Eq. (4)"},{"comment":"The summation notation in Eq. (7) is ambiguous: \\sum_{p,c=1} should be specified as a sum over prompts p with label c=1, and the means should be defined clearly over the probe set.","section":"Eq. (7)"},{"comment":"The phrase 'mass mean' is unusual; consider 'mean' or 'centroid' for clarity.","section":"§4.3"},{"comment":"The abstract claims 'up to 23 times faster', but Table 10 shows speedups of 12X to 23X; please rephrase to 'up to 23 times faster in our experiments' or report the range.","section":"Abstract and Table 10"},{"comment":"The comparison with SFT uses only 500 training samples for both methods; the conclusion that SFT is more accurate may depend on this budget. A brief discussion of how the comparison would change with more fine-tuning data would be helpful.","section":"§6.5.5"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely problem and the proposed method is interesting, but the central claims currently rest on a metric that does not match the paper's own definition of consistency and on an unvalidated causal assumption in the component-locating step. Both are fixable with additional experiments and analyses. The test-set selection procedure also needs scrutiny. I would be willing to reconsider after a major revision that addresses these points."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a decent engineering paper that shows you can improve prompt consistency by injecting bias into a handful of attention heads and MLPs, at a fraction of SFT's cost. That result looks real. What is not established is the causal-interpretability story: the paper claims to 'precisely diagnose' the components causing inconsistency, but the evidence is correlational.\n\nThe new bits are modest: applying the locate-then-edit paradigm to semantic consistency, and three NLU benchmarks (RobustSST2, RobustMRPC, RobustBOOLQ) built with GPT-4 paraphrases. Those benchmarks could be genuinely useful if released. The method itself is a straightforward combination of linear probing and activation steering (Alain & Bengio; Li et al.; Jorgensen et al.), but the application is new and the comparison with SFT is honest: SFT wins on most tasks, and the paper admits it. The efficiency numbers (12–23x fewer GPU hours) are the main practical selling point.\n\nThe soft spots are real but not fatal. First, the key assumption in Sec 4.2 — that a component whose hidden states let a linear probe predict the consistency label is 'actually the cause' — is exactly that: an assumption. High probe accuracy on the locate set only tells you the component carries information about the eventual prediction, not that intervening on it changes consistency. The random-component ablation shows selection matters, but it doesn't prove the selected heads are causal drivers. The paper should either run activation-patching interventions or soften the causal language substantially. Second, the NLU consistency metric (std. dev. of accuracy across templates) does not measure the paper's own definition of consistency — same answer for the same instance across paraphrases. It is a reasonable proxy but should be validated against pairwise agreement. Third, hyperparameters (K, alpha) are tuned on RobustSST2 and the same dataset is used for the reported main result; alpha=3.0 gives lower std than the reported alpha=5.0. That is a minor cherry-pick. The test-selection rule (sampling one prompt per distinct outcome) amplifies inconsistency in the test set; that's defensible but should be stated more clearly.\n\nOverall: the empirical phenomenon is probably real, the benchmarks are a contribution, and the limitations section is appropriately candid. The paper is ready for a serious referee, but it needs a major revision focused on the causal claim and the metric. I'd bring it to a reading group on model editing, and I'd cite it if I worked on prompt robustness. Verdict: conditional, leaning positive after revisions.","headline":"Useful activation-steering application to prompt consistency, but the causal interpretability claim outruns the evidence.","tokens_in":12411,"tokens_out":3880,"would_cite":true,"duration_ms":37851,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that injecting biases into the outputs of attention heads and MLPs selected by linear probing improves LLM semantic consistency and task performance without changing model parameters.","keywords":["semantic consistency","model editing","activation steering","linear probing","attention heads","prompt robustness","interpretability","large language models"],"falsifier":"Edit the lowest-scoring components from the same linear probes, with the same bias formula, and compare against editing the highest-scoring components on RobustSST2 or PopQA_capital; if the low-scoring components produce comparable consistency gains, the probe is selecting correlates rather than causes and the central claim fails.","tokens_in":11506,"feed_emoji":"🎯","tokens_out":11887,"duration_ms":120219,"temperature":0.7,"pith_summary":"The paper tries to establish that semantic inconsistency of a large language model—different answers to prompts with the same meaning—can be reduced by a cheap, interpretable model-editing step instead of fine-tuning. The authors locate attention heads and feed-forward blocks whose hidden states at the last token allow a linear classifier to predict whether the model will answer two paraphrased prompts consistently. They then add a fixed bias to those components along the average direction separating consistent from inconsistent activations. The reported result is that this shift improves consistency and, on most tested tasks, task accuracy, without changing any model weights and at a fraction of the compute of supervised fine-tuning. The evidence includes three newly constructed natural-language-understanding benchmarks, two natural-language-generation question-answering datasets, and out-of-domain transfer tests.","feed_headline":"Injected biases curb LLM inconsistency without retraining","feed_subtitle":"Probe-selected activation shifts improve semantic consistency and task accuracy at 12-23x lower GPU cost than fine-tuning.","key_machinery":"The load-bearing object is the linear-probing locate-then-edit loop. For every attention head and MLP in every layer, a linear classifier is trained on the concatenated last-token hidden states of a paraphrase pair to predict whether the model's two responses agree; the components whose classifiers score highest are treated as the cause of inconsistency. The edit is an activation shift: $h \\leftarrow h + \\alpha \\, b$, where $b$ is the difference between the mean hidden state on consistent pairs and the mean hidden state over all probe pairs, and $\\alpha$ is a strength hyperparameter (set to 5.0 in the main experiments). This mechanism carries the whole argument because it both selects where to intervene and supplies the direction of the intervention from the same probe data.","core_discovery":"On the paper's own terms, the central discovery is that prompt-paraphrase inconsistency in a decoder-only LLM is concentrated in a small set of mid-to-late-layer components, and that shifting those components' activations by a consistency bias measurably improves both consistency and accuracy. The selection is done by linear probing: concatenating the last-token hidden states of each attention head and MLP for a prompt pair, training a linear classifier to predict the pair's consistency label, and ranking components by classification accuracy. The editing is a mean-shift: the bias added to a selected component is the difference between its mean activation on consistent pairs and its mean activation over all pairs, scaled by a strength hyperparameter. With this recipe on a 7-billion-parameter chat model, the paper reports lower response variance across synonymous prompts, higher average accuracy on most tasks, and rough parity or small gains on out-of-domain tasks, while leaving weights untouched. The authors also report that supervised fine-tuning still achieves larger gains on most benchmarks, but at a substantially higher compute cost.","pith_inferences":["If the probe-as-cause assumption holds, the same recipe transfers to other behavioral labels—truthfulness, toxicity, style—by swapping the consistency label for the target label.","The paper's limitations section notes that locality and portability are not yet measured; until those are checked, the edit's side-effect profile on unrelated knowledge remains unknown.","A natural next experiment would test the method on human-written paraphrases rather than model-generated ones, since the bias is estimated from the same paraphrase distribution used for evaluation.","The paper's own comparison shows supervised fine-tuning still wins on most benchmarks, so the method is best read as a low-cost patch or a preliminary intervention rather than a replacement for fine-tuning when maximum accuracy is the goal."],"forward_implications":["LLM providers could patch paraphrase sensitivity after deployment by adding a few hundred paraphrased prompt pairs and editing activations, with no retraining.","The reported 12 to 23 times reduction in GPU hours makes consistency improvement feasible in compute-constrained settings where supervised fine-tuning is not.","Because the located components cluster in mid-to-late layers, consistency failures are not spread uniformly across the network, pointing to concentrated targets for future fixes.","Editing and supervised fine-tuning appear partly complementary: applying editing after fine-tuning yields a small additional consistency gain."],"supporting_citations":[{"why":"This reference supplies the linear-probing protocol used to score each component's hidden states against the consistency label.","marker":"Alain and Bengio, 2016"},{"why":"This reference provides the inference-time intervention idea of editing attention-head activations to steer model behaviour.","marker":"Li et al., 2023"},{"why":"This reference supplies the mean-centring style of activation bias used to compute the editing direction.","marker":"Jorgensen et al., 2023"},{"why":"This reference contributes the NLG semantic-consistency benchmark and the mean pairwise cosine similarity metric used for NLG evaluation.","marker":"Rabinovich et al., 2023"},{"why":"This reference is the prior prompt-engineering approach to semantic consistency that the paper compares against.","marker":"Raj et al., 2023"},{"why":"This reference defines the supervised fine-tuning baseline whose performance and GPU cost are compared with the editing method.","marker":"Ouyang et al., 2022"},{"why":"This reference gives the target 7-billion-parameter chat model used in all experiments.","marker":"Touvron et al., 2023"},{"why":"This reference establishes the locate-then-edit paradigm that the paper adopts for consistency editing.","marker":"Meng et al., 2022"}],"fun_headline_variants":["Attention-head bias injection boosts LLM consistency at fraction of fine-tuning cost","Probe-selected activation shifts align paraphrases, cutting LLM inconsistency without retr","Editing attention heads makes LLM outputs consistent across paraphrases, no retraining","Bias shifts in key attention heads yield 12-23x cheaper LLM consistency gains"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that a part of the model that can predict, from its internal state, whether the model will be inconsistent is itself the cause of the inconsistency, so changing that part will fix it; no causal test is done before editing.","fun_headline_variants_meta":{"raw":{"variants":["Attention-head bias injection boosts LLM consistency at fraction of fine-tuning cost","Probe-selected activation shifts align paraphrases, cutting LLM inconsistency without retraining","Editing attention heads makes LLM outputs consistent across paraphrases, no retraining","Bias shifts in key attention heads yield 12-23x cheaper LLM consistency gains"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000526,"raw_usage":{"total_tokens":2567,"prompt_tokens":999,"completion_tokens":1568,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":615,"completion_tokens_details":{"reasoning_tokens":1481}},"tokens_in":615,"tokens_out":1568,"duration_ms":12276,"temperature":1.0,"reasoning_tokens":1481,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T18:41:06.444929+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Edit the lowest-scoring components from the same linear probes, with the same bias formula, and compare against editing the highest-scoring components on RobustSST2 or PopQA_capital; if the low-scoring components produce comparable consistency gains, the probe is selecting correlates rather than causes and the central claim fails.","supporting_citations":[],"review_version":1}