{"id":"c4708a8f-29d7-4eb1-91f7-c55264e87ded","arxiv_id":"2511.23231","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Injecting two precomputed hidden-state vectors during decoding—one to align non-English reasoning with English, one to restore target-language output—raises multilingual reasoning accuracy by ~5.5% on average across six LLMs/LVLMs and eight languages.","lead":"This paper introduces a training-free inference-time method that steers a model's internal representations so non-English questions are reasoned about 'in English' while answers are still produced in the user's language, improving multilingual reasoning accuracy by about 5.5% on average across six LLMs and LVLMs. The approach could make reasoning models fairer and more useful for low-resource languages without retraining or external translation.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Test-set contamination: the 100 MGSM/MathVerse problems used to build v_r are not held out from Table 1, so the headline +5.48% may be in-sample.","rationale":"The reader correctly flagged that fixed vectors and hyperparameters are selected on the same benchmarks used for headline results, but did not explicitly identify the concrete overlap between the 100 sampled vector-construction problems and the evaluation sets. This overlap is a more specific and more serious form of leakage: for MGSM, using 100 of 250 test problems to compute v_r means 40% of the evaluation set directly influences the method. Even without using gold labels, transductive construction from test inputs can inflate performance on those exact problems, especially when combined with layer/strength tuning on the same benchmark. The OOD results on MSVAMP and MMathVista are encouraging and partially support generalization, but the abstract's headline average includes MGSM and MMathVerse, so the central number may not be trustworthy. A clean held-out split of the vector-construction set is a simple, decisive check. If the gains persist on disjoint problems, the concern is resolved; if not, the central claim requires revision. The verdict remains conditional pending this re-evaluation.","tokens_in":21342,"tokens_out":8108,"duration_ms":93749,"concrete_test":"Re-evaluate MRRE on the 150 MGSM problems not used to compute v_r (and on a held-out split of MMathVerse for LVLMs), using the same hyperparameters; report per-language gains and the aggregate average. If the average gain on these disjoint splits drops materially below 5.48% or is not consistently positive, the central claim is an artifact of test-set overlap. Ideally also run the same check with hyperparameters chosen on a separate development set.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The most load-bearing issue is that the main evaluation benchmarks are not held out from vector construction or hyperparameter selection. In §4.1, the authors 'randomly sample 100 instances from MGSM and MathVerse to construct MRRE vectors' (Eq. 3); Table 1 then reports MGSM and MMathVerse accuracies without excluding these 100 problems. MGSM has only 250 problems per language, so up to 40% of the test set is used to build v_r. Additionally, §5.4/Tables 11-12 select l, l', α1, α2 by performance on MGSM. Thus the headline '+5.48% average' is not an estimate of out-of-sample improvement; it is partly an in-sample/transductive result. The MSVAMP and MMathVista columns provide some OOD support, but the abstract's average and 'low-resource +7.54%' claims are dominated by the contaminated benchmarks. If the vector is tuned to the exact test problems, even a high-dimensional mean vector can memorize problem-level hidden-state shifts and inflate accuracy without generalizing.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MRRE, a training-free, inference-time representation-engineering method that improves multilingual reasoning in LLMs and LVLMs. Two precomputed vectors are injected into hidden states during decoding: a cross-lingual reasoning enhancement vector, computed as the mean hidden-state difference between English and target-language reasoning chains (Eqs. 2–3), and a target-language output anchoring vector, computed from language-forcing prompts (Eq. 4). The method is evaluated on six models and four benchmarks, with reported average non-English accuracy gains of 5.48%, low-resource gains up to 7.54%, and language-consistency gains of 3.78%. The paper also reports ablations of layer choice, injection strength, debiasing variants, cross-modal vector transfer, and case studies.","tokens_in":21725,"tokens_out":8116,"duration_ms":80723,"significance":"If the empirical claims hold, MRRE is a practically attractive method: it requires no training data, no external translation tools, and works across both LLMs and LVLMs. The paper's strengths include a simple, falsifiable intervention; out-of-distribution transfer to MSVAMP and MMathVista; and cross-backbone vector transfer from language models to LVLMs (Table 3), which is a genuinely independent piece of evidence. The detailed ablations and case studies are useful. However, the headline result is weakened by the fact that the main benchmarks are partly used to construct the vectors and to select hyperparameters, and by an apparent inconsistency in the reported average gain. These issues are addressable but require additional experiments and clearer reporting.","major_comments":[{"comment":"The headline evaluation is partly transductive. Section 4.1 states that 100 instances are randomly sampled from MGSM and MathVerse to construct the MRRE vectors, yet Table 1 reports MGSM and MMathVerse accuracy without excluding those instances. Since MGSM contains 250 problems per language, up to 40% of each language's test set is used to compute v_r in Eq. (3). Moreover, the hyperparameters l, l', α1, and α2 are selected using MGSM performance (§5.4, Tables 11–12). Consequently, the abstract's +5.48% and +7.54% claims are not estimates of out-of-sample improvement. The MSVAMP and MMathVista columns, and the cross-backbone experiment in §5.2, provide genuine out-of-distribution support, but the paper should either construct vectors on a held-out split and rerun Tables 1–2, or explicitly report results excluding the 100 construction examples and use a validation split for hyperparameter","section":"§4.1 and Table 1"},{"comment":"The central quantitative claim is not reproducible from the reported tables. Averaging the per-model, per-benchmark gains over the eight non-English languages in Table 1 gives roughly 4.5 percentage points, not 5.48%. The low-resource average is plausible, but the aggregation rule is unspecified. If 5.48% is computed from a different set of numbers, such as Table 2's Overall scores, or with a particular weighting, that must be stated explicitly. As written, the reader cannot verify the headline number.","section":"Abstract and §4.2"},{"comment":"Most experiments use stochastic decoding (do_sample=True, temperatures up to 0.7), yet every reported accuracy is a single run with no random seeds, no variance, and no significance tests. Some improvements are small (e.g., Qwen2.5-7B on MSVAMP Spanish, +0.4 points; Qwen3-8B on MGSM Thai, +0.4 points), so it is unclear which gains are robust. The authors should report multiple seeds with means and standard errors, or at least fix and disclose seeds, and confirm that the main conclusions are stable under decoding variance.","section":"Appendix B.1 and Tables 1–2"}],"minor_comments":[{"comment":"'low-source languages' should be 'low-resource languages'.","section":"Abstract"},{"comment":"The labels 'V anilla' and 'V ectors' contain unintended spacing; they should read 'Vanilla' and 'Vectors'.","section":"Tables 3 and 4"},{"comment":"The random sample of 100 instances used to construct MRRE vectors is not accompanied by a seed or exact list of instances; include this for reproducibility.","section":"§4.1"},{"comment":"The t-SNE and KDE visualizations would be easier to interpret with a quantitative distance measure between language distributions, e.g., centroid distances or KL divergences, in addition to the plots.","section":"Figure 2 and Figure 4"},{"comment":"The Japanese language-forcing prompt contains apparent typos ('日本を使って考え' instead of a natural imperative). Since this prompt is part of the baseline, native-speaker checking would improve confidence in the comparison.","section":"Table 8"},{"comment":"The translation-quality check uses Average Predicted Consistency, which measures output consistency rather than semantic equivalence. Reporting a human evaluation of translation adequacy, even on a small sample, would strengthen the constructed benchmarks.","section":"Appendix A"},{"comment":"No code release or reproducibility package is mentioned. Given the method is inference-time and inexpensive, releasing code would substantially help adoption and verification.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The main risk is the test-set overlap: the headline numbers are partly in-sample, and this should be fixed before publication. I do not see grounds for rejection, because the OOD columns (MSVAMP, MMathVista) and the cross-backbone transfer experiment provide evidence that the method has real transferability beyond the construction set. The arithmetic mismatch for the 5.48% figure also needs a clear explanation. If the held-out rerun and the numerical reporting are corrected, this paper would be a solid contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the two-vector MRRE idea is new and mostly works—steering non-English hidden states toward English at mid-layer and then anchoring back to the target language at late layers. The evidence base is broad: six models (LLMs and LVLMs), four benchmarks, eight languages. The most convincing part is the OOD transfer: MSVAMP and MMathVista were not used to build the vectors, and the gains there are consistent, as is the cross-backbone experiment (LLM-derived vectors on LVLMs). That tells me the method captures a generalizable direction rather than memorizing a specific test set.\n\nThe soft spot is exactly what the stress-test note flags. The v_r vectors are built from 100 MGSM and MathVerse problems, and Table 1 reports accuracy on those same benchmarks without holding out the 100. MGSM has 250 problems per language, so that's up to 40% of the test set being used to construct the steering direction. Hyperparameter selection (§5.4) is also done on MGSM. So the headline +5.48% average and the +7.54% for Thai/Swahili are partially in-sample. The authors should re-report on a held-out split (the remaining 150 MGSM problems, and a similar split for MathVerse) and show the OOD benchmarks separately. This is a fixable reporting problem, not a fatal one.\n\nTwo smaller notes. First, v_a is computed from just two fixed prompts per language, which is a thin estimate; the LC improvements suggest it's doing the job, but it would be good to see sensitivity to that choice. Second, several models are run with sampling (temperature 0.7) and no variance is reported, so the effect sizes could be noisy. Fixed vectors (one per layer per language) are a strong assumption, but the OOD and cross-backbone results are evidence the assumption isn't badly wrong.\n\nBottom line: the paper deserves peer review. The central idea is interesting and the OOD support is real, but the empirical claims need the contamination fixed. I'd send it to a serious referee with a request for held-out numbers, code, and the translated data. It would be a good reading-group paper, mostly because of the evaluation-hygiene discussion.","headline":"A training-free multilingual reasoning steering method with real OOD promise, but the headline numbers are partly in-sample because the vector-construction and tuning sets overlap the main test sets.","tokens_in":22139,"tokens_out":4011,"would_cite":true,"duration_ms":36901,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Injecting two precomputed hidden-state vectors during inference lifts non-English reasoning accuracy by 5.48% on average across six large language and vision-language models.","keywords":["multilingual reasoning","representation engineering","inference-time intervention","large language models","vision-language models","hidden-state steering","low-resource languages","language consistency"],"falsifier":"Evaluate MRRE on a held-out language not among the eight tested, using vectors built only from the standard 100 sampled problems and with layers and strengths frozen at the reported settings, then check whether accuracy still beats baseline. Alternatively, compute the steering vector per instance from the actual English translation of each question and compare: if the fixed average vector performs much worse than per-instance vectors, the transferability assumption fails.","tokens_in":21292,"feed_emoji":"🌐","tokens_out":4067,"duration_ms":38398,"temperature":0.7,"pith_summary":"This paper claims that the reasoning gap between English and non-English in LLMs and LVLMs can be narrowed without retraining, extra data, or translation tools, by steering hidden states at inference time. It proposes MRRE, which injects a cross-lingual reasoning enhancement vector in middle layers to shift non-English reasoning representations toward English, then a target-language output anchoring vector in later layers to restore the output language. Across six models and four benchmarks, the method reports an average 5.48% accuracy gain on non-English reasoning, up to 7.54% in low-resource languages such as Thai and Swahili, and a 3.78% improvement in input-output language consistency. Because the approach is training-free and tool-free, it offers a cheap, post-hoc route to fairer multilingual reasoning; the paper also notes it requires access to internal model representations, so it cannot serve closed-source models.","feed_headline":"Non-English reasoning jumps 5.48% with inference-time steering","feed_subtitle":"Two precomputed hidden-state vectors shift non-English reasoning toward English, then back, without retraining or translators.","key_machinery":"Two precomputed hidden-state direction vectors, applied sequentially with L2-normalized scaling. The cross-lingual reasoning enhancement vector v_r is the mean of per-problem differences between English and non-English last-token hidden states at layer l, computed over 100 sampled reasoning problems; adding it during inference pushes non-English reasoning into English-like representation space. The target-language output anchoring vector v_a is the difference between the last-token hidden states of a target-language forcing prompt and an English forcing prompt at layer l'; adding it later pulls output representations back toward the target language. These two vectors constitute the entire me","core_discovery":"The central claim is that a fixed pair of average difference vectors, computed once from a small sample of reasoning problems, can be applied at inference time to make a model reason about non-English questions as if they were in English while still answering in the original language. The first vector aligns the last-token hidden state of a non-English reasoning chain with its English counterpart at a middle layer; the second restores the target-language output distribution at a later layer. The authors demonstrate that this two-stage intervention consistently improves accuracy and language consistency across three LLMs and three LVLMs, including on out-of-distribution benchmarks and across","pith_inferences":["If the fixed-average-vector assumption holds, a similar recipe could be built from any stronger 'anchor' language or even a stronger model, transferring reasoning skill from one language or model to another without fine-tuning.","The two-stage decomposition suggests a testable mechanistic claim: mid-layer states carry language-independent reasoning while late-layer states carry output-language identity; ablating only the anchoring step should collapse language consistency while preserving accuracy.","Because the layers and strengths were tuned on the same benchmarks used for the headline results, reported gains may be upper bounds; freezing settings on a development set and evaluating on untouched languages would provide a stricter test.","The cross-modal transfer result hints that LVLMs inherit multilingual reasoning directions from their language backbones, so representation steering could be performed once on a strong LLM and applied to a family of downstream multimodal models."],"forward_implications":["MRRE improves non-English reasoning accuracy by an average of 5.48% across four benchmarks, with the largest gains (up to 7.54%) in low-resource languages like Thai and Swahili.","Input-output language consistency rises by 3.78%, so models more often answer in the user's language rather than slipping into English after the steering intervention.","The vectors transfer across datasets: vectors built from MGSM and MathVerse problems also improve MSVAMP and MMathVista, including general-reasoning subsets.","Vectors derived from a language backbone transfer to the corresponding LVLM, suggesting the multilingual reasoning direction is shared across model variants.","Because the method needs hidden-state access, it applies to open-weight models but not to closed-source APIs.","The intervention improves all five visual-reasoning categories of MMathVerse, indicating the effect is not limited to pure text reasoning."],"fun_headline_variants":["MRRE: two vectors boost multilingual reasoning by 5.48%","Training-free vectors boost multilingual reasoning by 5.48%","Two hidden-state vectors unlock non-English reasoning in LLMs and LVLMs","Inference-time steering lifts low-resource language reasoning up to 7.54%"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that a single average hidden-state difference, computed from 100 sampled problems and two fixed prompts, is a stable direction that transfers across inputs, models, and benchmarks; if the English-versus-non-English gap is instance-dependent or varies with reasoning strategy, the fixed steering will not generalize.","fun_headline_variants_meta":{"raw":{"variants":["MRRE: two vectors boost multilingual reasoning by 5.48%","Training-free vectors boost multilingual reasoning by 5.48%","Two hidden-state vectors unlock non-English reasoning in LLMs and LVLMs","Inference-time steering lifts low-resource language reasoning up to 7.54%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001407,"raw_usage":{"total_tokens":5512,"prompt_tokens":723,"completion_tokens":4789,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":467,"completion_tokens_details":{"reasoning_tokens":4708}},"tokens_in":467,"tokens_out":4789,"duration_ms":31325,"temperature":1.0,"reasoning_tokens":4708,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T19:32:59.505366+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Evaluate MRRE on a held-out language not among the eight tested, using vectors built only from the standard 100 sampled problems and with layers and strengths frozen at the reported settings, then check whether accuracy still beats baseline. Alternatively, compute the steering vector per instance from the actual English translation of each question and compare: if the fixed average vector performs much worse than per-instance vectors, the transferability assumption fails.","supporting_citations":[],"review_version":1}