Pith. sign in

REVIEW 3 major objections 5 minor 40 references

Cross-lingual accuracy gaps in LLMs are mostly response noise, not missing knowledge.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 09:21 UTC pith:TXDSRRYJ

load-bearing objection A useful reframing of cross-lingual gaps as variance-dominated, but the main diagnostic conflates bias with variance; the claim needs a better test before it is treated as established. the 3 major comments →

arxiv 2510.15551 v2 pith:TXDSRRYJ submitted 2025-10-17 cs.CL cs.AIcs.LG

Rethinking Cross-lingual Gaps from a Statistical Viewpoint

Vihari Piratla , Purvam Jain , Darshan Singh , Trevor Cohn , Preethi Jyothi , Partha Talukdar This is my paper
classification cs.CL cs.AIcs.LG
keywords cross-lingual gapbias-variance decompositionresponse varianceknowledge barriersensemblinglarge language modelsknowledge transferinference-time mitigation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that when a large language model answers a question correctly in one language but fails in another, the usual explanation—that the knowledge never transferred into the model's parameters for the target language—is mostly not the cause. The authors model the gap as a bias-variance decomposition of response distributions, and find that target responses are typically spread around the correct source answer rather than strongly shifted to a wrong answer. Ensembling several responses, whether by repeated sampling or by prompting the model with multiple translations, shrinks source-target gaps across five LLMs, exactly as predicted if variance dominates. They estimate that on 90–95% of examples the gap is variance-dominated. If correct, mitigation shifts from expensive pretraining changes to cheap inference-time variance reduction.

Core claim

The paper's central claim is that the cross-lingual gap—the accuracy drop when knowledge is queried in a target language rather than the source—is dominated by unbiased response variance in the target, not by parametric knowledge fragmentation. Formally, target logits are a mixture of a high-variance component (same mean as source, higher spread) and a high-bias component (a different mode), with unknown mixing coefficient π. The authors prove that reducing response variance decreases source-target agreement when the bias component dominates and increases agreement when the variance component dominates; ensembling multiple responses is the variance-reduction intervention that discriminates t

What carries the argument

The load-bearing object is a bias-variance decomposition of the cross-lingual gap. The model treats logits as sampled from a normal distribution; if the knowledge transferred, target logits have the same mean as source but higher variance and a flatter softmax (μ_s/τ, ησ_s²), while knowledge barriers would shift the mean entirely (μ_b). The mixing coefficient π between the variance component and the bias component is what the experiments estimate. Propositions 1 and 2 state that reducing response variance decreases source-target agreement when bias dominates and increases it when variance dominates; ensembling multiple sampled responses is used as the practical variance-reduction interventio

Load-bearing premise

The central diagnostic assumes that majority voting over several sampled responses is equivalent to actually lowering the logit-noise variance in the model's response distribution; if ensembling shrinks the gap through some other mechanism (e.g., correcting a biased but multi-modal error distribution), the experiments do not establish variance dominance.

What would settle it

Build a benchmark where target responses are artificially biased, e.g., by shifting the reference answer to a systematically wrong year in the target language, and run the same response-ensembling experiment. The theory predicts that increasing ensemble size should leave the source-target gap unchanged or make it worse when the bias dominates; if ensembling still shrinks the gap on biased examples, the diagnostic fails to separate variance from bias. Alternatively, inspect target response distributions directly: if the target mode systematically differs from the source mode on a large fraction

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Ensembling multiple responses at inference time reduces the cross-lingual gap without adding any new knowledge; transfer scores rise by up to 12 absolute points (relative gains 8% to over 50%) across models.
  • Simple prompt-level ensembles—presenting several translations (TrEn) or asking the model to translate then answer (TTA)—are sufficient to recover much of the gap.
  • Cross-lingual gaps shrink as source-language confidence rises, because source and target response variances are proportional; the authors confirm this relationship empirically.
  • Mitigation should focus on post-training and inference-time variance reduction rather than on re-aligning embeddings or pretraining for knowledge transfer.
  • The analysis applies to well-represented languages in pretraining data; the authors state it will not apply to unseen languages.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If variance dominance holds beyond the evaluated benchmarks, repeated sampling from a single frozen model could act as a cheap cross-lingual confidence estimator: agreement across samples in the target language would directly flag reliable answers, without any training.
  • The variance-dominance claim implies that measuring the entropy (dispersion) of target-language responses could predict where cross-lingual failures will occur, which is testable on existing datasets and could yield a per-query reliability score.
  • The authors' spot-checks attribute residual bias to translation errors in the benchmark itself; a plausible extension is that part of the measured cross-lingual gap is a dataset artifact, and cleaner parallel data would shrink the gap even without model changes.
  • The same ensembling diagnostic could be applied to other reported performance disparities—such as text-input vs audio-input gaps or long-context degradation—to test whether those gaps are also variance-dominated, a generalization the paper only mentions in passing.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes that cross-lingual knowledge gaps in closed-book QA are caused primarily by increased response variance in the target language rather than by fragmented parametric knowledge. It introduces a two-component mixture model for target logits — an unbiased high-variance component centered at the source mean and a biased component — and derives propositions stating that variance reduction should increase source-target agreement under unbiased variance and decrease it under pure bias. The experimental program uses response ensembling (Fig. 4), input ensembling via TrEn/TTA (Table 1), and source-confidence analysis (Fig. 6) to argue that the unbiased component dominates, with estimated π ≈ 0.9–0.95.

Significance. If the variance-dominance claim is established, the paper is significant: it would reframe cross-lingual gaps as a test-time variance problem rather than a parametric-knowledge fragmentation problem, redirecting mitigation toward ensembling and post-training rather than representation alignment. The paper's strengths are its formal bias-variance decomposition, the breadth of models and benchmarks (five LLMs across ECLeKTic, MMLU-mixup, Year-ECLeKTic, and MultiLoKo), and the consistent transfer-score gains from TrEn/TTA. However, the central ECLeKTic diagnostic as reported cannot distinguish bias from variance; the main conclusion currently rests on an estimator whose behavior under the null hypothesis (pure bias) is not what the text claims. With a corrected estimator or a direct plateau comparison, the claim would be much better supported.

major comments (3)
  1. [§4.1.1, Fig. 4(a), §K.6] The ECLeKTic π estimate is uninformative as reported. For any source/target response distributions with means μ_s, μ_t and covariances Σ_s, Σ_t, the expected squared L2 distance between N-averaged embeddings is E||x̄_s,N − x̄_t,N||² = ||μ_s − μ_t||² + (tr Σ_s + tr Σ_t)/N. Therefore the distance decreases with ensemble size even when every example is in the biased component (κ=0, π=0). The estimator 'fraction of examples where distance decreased from N=1 to N=10' will be close to 1 under pure bias, so the reported values 0.85–0.96 do not establish that the unbiased component dominates. The oracle line could resolve the ambiguity if the distance clearly plateaued near it, but no numerical plateau comparison is reported. The claim in §K.6 that 'source-target divergence improves with number of examples only when the noise is unbiased' is contradicted by the elementary calculation above.
  2. [§2.1, Propositions 1–2 vs. §4.1.1] Propositions 1 and 2 concern the probability that a single source sample and a single target sample agree, as the logit variance σ_s², σ_t² is reduced. The experiments instead reduce variance by averaging N response embeddings or N response distributions, and no formal link is provided showing that averaging implements the same operation as lowering the generative logit variance in the mixture model. This is not merely a formal gap: under pure bias, averaging reduces the observed L2 distance even though the mode of the target distribution remains wrong. The authors should either prove the analogous proposition for their ensemble statistic, or demonstrate with a synthetic simulation that the ensemble statistic separates the two mixture components as claimed. Without this, the central diagnostic in §4.1.1 does not establish the variance-dominance claim.
  3. [§4.2, Fig. 6] The source-confidence analysis is suggestive but has a confound: source confidence may correlate with question difficulty and with the probability that the target distribution is centered on the correct answer even under a biased model. A more direct test would be to condition on question difficulty or on source accuracy at the individual-example level, or to compare the slope of agreement-vs-confidence under the two mixture components. As presented, Fig. 6 supports the paper's story but does not sharply discriminate variance from bias.
minor comments (5)
  1. [Abstract vs. Table 1] The abstract says the prompt instruction 'improved target accuracy by 20–25%', while Table 1 reports transfer-score improvements of up to 12 absolute points and the introduction quotes 'up to 12 absolute points' with relative gains of 8–50%. Please reconcile these numbers.
  2. [Fig. 4(a)] The oracle line is described only qualitatively. Please report the numerical oracle distance and the N=10 distance, with confidence intervals, so readers can assess whether the ensemble curve is actually approaching the oracle.
  3. [Appendix D] The proofs are sketches that approximate Gumbel noise by Gaussians with different variances (N(0,2) in Proposition 1, N(0,1) in Proposition 2). This approximation should be stated in the main text, and its effect on the claimed monotonicity should be validated empirically or with a sensitivity analysis.
  4. [References] Chua et al. 2024a and 2024b refer to the same arXiv paper; please consolidate or differentiate them.
  5. [§K.6] The statement that the fraction-of-decrease estimator is valid because 'divergence improves ... only when the noise is unbiased' is incorrect for L2 distance, as explained in Major Comment 1. The distinction between ECLeKTic and MMLU estimators should be rewritten to acknowledge the averaging artifact and justify why the chosen statistic has the claimed behavior.

Circularity Check

1 steps flagged

The ECLeKTic π estimator is the ensemble-decrease observation relabeled as evidence of unbiased noise; that statistic shrinks under pure bias too, making the central Fig. 4 diagnostic partially circular.

specific steps
  1. self definitional [Section 4.1.1 (Fig. 4) and Appendix K.6]
    "π estimate. We present in each plot the value of π=E[κ] estimated as the fraction of examples on which the average L2 distance decreased from ensemble size of one to ten. ... we used the fraction of examples on which ensembling improved L2 distance as a proxy to estimate π on ECLeKTic. This is a valid estimate because as we show in Section 2, source-target divergence improves with number of examples only when the noise is unbiased, i.e., κ=1."

    The ECLeKTic π estimator is defined as the fraction of examples whose average source-target embedding L2 distance decreased from ensemble size 1 to 10, and that same decrease is then presented as evidence that κ≈1. But under the paper's own κ=0 (pure bias) component, the analogous expected squared distance is ||μ_s−μ_t||² + tr(Σ_s)/N + tr(Σ_t)/N, which also decreases with N while asymptoting to the positive bias gap. Thus a high fraction of 'decreased' examples is expected even if every example is biased, so the estimator cannot identify π. Appendix K.6 asserts the proxy is valid because 'source-target divergence improves ... only when the noise is unbiased' — exactly the claim under test rather than a property of the measured statistic. The oracle asymptote could have disambiguated bias v

full rationale

The paper's formal framework is not circular: Propositions 1–3 are genuine forward statements about how shared-response probability and mode confidence respond to logit-variance changes, and those predictions could have failed empirically. The TrEn/TTA transfer-score improvements (Table 1), Year-ECLeKTic MAE/accuracy trends (Appendix G), and source-confidence/agreement curves (Section 4.2) are independent, externally falsifiable observations. The circularity is concentrated in the Section 4.1.1/Fig. 4 diagnostic that carries the paper's central claim. The ECLeKTic π estimate is defined as the fraction of examples where ensembling decreased the average embedding L2 distance, yet this decrease occurs under both κ=0 and κ=1: averaging reduces sampling noise around a biased mean just as it does around an unbiased mean. Appendix K.6 states the proxy is valid because 'source-target divergence improves with number of examples only when the noise is unbiased,' which is the conclusion being argued rather than a derived identification. Because the reported π≈0.9 is therefore, to a substantial extent, an artifact of the estimator's definition, the 'variance, not knowledge fragmentation' conclusion is partially circular. However, the paper retains meaningful independent evidence in the form of input-ensembling transfer gains and the confidence-agreement trends, so a score of 6 is appropriate rather than 8 or 10.

Axiom & Free-Parameter Ledger

4 free parameters · 7 axioms · 0 invented entities

The paper introduces no new physical or architectural entities; its 'high-variance component' and 'high-bias component' are analytic constructs of the statistical model. The main free parameters are the mixture coefficient π and the unmeasured scaling factors τ and η, plus the never-estimated bias-component parameters. The most load-bearing axioms are the generative softmax-sampling model and the asserted equivalence between ensembling and lowering logit variance.

free parameters (4)
  • π (mixing coefficient) = ≈0.88–0.97 across models
    Introduced in Sec. 2 as the unknown probability that a target response comes from the high-variance unbiased component. It is estimated from the same ensembling-divergence observations (L2 decrease fraction on ECLeKTic; soft mode-mismatch on MMLU) and then cited as evidence that unbiased noise dominates (Sec. 4.1.1).
  • τ (logit flattening factor)
    Scale factor μ_s/τ in the high-variance target component (τ≥1), Sec. 2. Never estimated; appears in Propositions 2 and 3 and in the confidence lower bounds.
  • η (variance multiplier)
    Multiplier ησ_s² on target logit variance in the high-variance component (η≥1), Sec. 2. Never estimated; required for the claim that target variance is proportionally larger than source.
  • μ_b, σ_b² (bias-component parameters)
    Mean and variance of the alternative biased target distribution in the mixture (Sec. 2). Never estimated; the paper argues this component is non-dominant from the π estimates and spot-checks.
axioms (7)
  • domain assumption Forward-pass stochasticity (floating-point errors and MoE routing) sufficiently captures response variance for a single fixed model.
    Sec. 2: 'We operate under the assumption that floating point errors and MoE routing uncertainties sufficiently capture the response variance.' Without this, bias-variance analysis would need multiple trained models.
  • domain assumption The response space is enumerable and shared between source and target after normalizing responses to language-agnostic concepts.
    Sec. 2: 'we assume that the response space is enumerable and shared between source and target... normalize... only encode concept while ignoring the language.' This makes source-target response distributions comparable.
  • domain assumption LLM decoding can be modeled as z ~ N(μ, σ²I) followed by y ~ Categorical(softmax(z)).
    Sec. 2 gives this generative model; all propositions and the mixture formulation are derived from it.
  • ad hoc to paper The target response distribution is exactly a two-component mixture: a high-variance unbiased component N(μ_s/τ, ησ_s²I) and a biased component N(μ_b, σ_b²I) with unknown mixing π.
    Sec. 2: 'we model target responses as a mixture of both the distributions with an unknown mixing coefficient π.' The specific Gaussian/scaling forms are posited for this analysis and are not derived from model internals.
  • ad hoc to paper Ensembling multiple responses (majority voting/averaging) constitutes the same 'reduced response variance' that Propositions 1-2 reason about.
    Sec. 2.1: 'we may reduce variance by simply ensembling multiple responses...'. This equivalence is asserted, not proven, and is the bridge that turns the propositions into a diagnostic.
  • standard math Gumbel(0,1) sampling noise can be approximated by N(0,1) or N(0,2) for bounding arguments.
    Appendix D proofs use this approximation to derive upper/lower bounds; it is a proof technique, not an empirical claim.
  • domain assumption LLM-as-judge accuracy and multilingual embedding distances are adequate for measuring source-target divergence (with Year-ECLeKTic used where judge noise is a concern).
    Sec. 3: autorater >95% on spot-checks but higher error on non-English; embeddings are language-sensitive, so π for ECLeKTic uses a proxy and PCA drops language-dominated components (Appendix C, K.6).

pith-pipeline@v1.3.0-alltime-deepseek · 20370 in / 16376 out tokens · 125735 ms · 2026-08-04T09:21:45.479819+00:00 · methodology

0 comments
read the original abstract

Any piece of knowledge is usually expressed in one or a handful of natural languages on the web or in any large corpus. Large Language Models (LLMs) act as a bridge by acquiring knowledge from a source language and making it accessible when queried using target languages. A cross-lingual gap is a drop in accuracy incurred when querying knowledge in a target language rather than the source language. Existing research focused on modeling or training failures leading to cross-lingual gaps. In this work, we take an alternative view to characterize the nature of cross-lingual error, and hypothesize that the variance of responses in the target language is a key cause of this gap. For the first time, we formalize the cross-lingual gap in terms of biased and unbiased errors. We empirically validate our hypothesis through multiple inference-time interventions that control variance and reduce the cross-lingual gap. We demonstrate a few test-time ensemble methods that reduce response variance, and thereby improve source-target transfer scores by up to 12 absolute points yielding relative gains of 8% to over 50% across various LLMs.

Figures

Figures reproduced from arXiv: 2510.15551 by Darshan Singh, Partha Talukdar, Preethi Jyothi, Purvam Jain, Trevor Cohn, Vihari Piratla.

Figure 1
Figure 1. Figure 1: If the model has knowledge barriers, we expect the responses in target biased as shown in [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Cross-lingual performance gaps on ECLeKTic and MMLU (with mixup) with different [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: Response ensembling from multiple forward passes gradually diminishes the source-target [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Prompt templates for two input ensembling approaches. k is the number of translations [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Cross-lingual gaps diminishes with reduced variance in source language. Answer confi [PITH_FULL_IMAGE:figures/full_fig_p009_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Illustration of Source Borrowed Entities in Target (SBET) and its performance. SBET [PITH_FULL_IMAGE:figures/full_fig_p013_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Replicating Figure [PITH_FULL_IMAGE:figures/full_fig_p014_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: An example from the MMLU (with mixup) dataset. [PITH_FULL_IMAGE:figures/full_fig_p018_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Empirical validation that high confidence in source leads to high confidence in target [PITH_FULL_IMAGE:figures/full_fig_p019_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Reproducing results from Figure [PITH_FULL_IMAGE:figures/full_fig_p019_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Reproducing results from Figure [PITH_FULL_IMAGE:figures/full_fig_p019_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Performance of DeepSeek model, observing improved source target agreement with [PITH_FULL_IMAGE:figures/full_fig_p020_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Additional results from Figure 1b on ECLeKTic. Let ⃗rs, ⃗rt represent the representations of an example in source and target. The representations are expected to contain a language-agnostic and language-specific components, which we assume linearly compose the final representation, i.e., ⃗rs = ⃗r+⃗rl , ⃗rt = ⃗r+⃗rl ′ , l ̸= l ′ . We use a linear model to inspire the effects of language component, thereby … view at source ↗
Figure 15
Figure 15. Figure 15: Additional results from Figure 1c on ECLeKTic. K.2 RESPONSE SUMMARIZER PROMPT Please find the full prompt used for summarizing a list of strings into their unique values and counts in response_summarizer_prompt.txt in supplementary material. K.3 AUTOCHECKER PROMPT Please find the full prompt used for autochecking if a response matched the reference in autocheker_prompt.txt attached in supplementary materi… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

40 extracted references · 18 linked inside Pith

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should not add it explicitly Type <Return> for now, but then later remove the command n...

  3. [3]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@first@sw \@firstoftwo \@ifundefined NAT@b*@#2 \@firstoftwo @num @NAT@ctr \@secondoft...

  4. [4]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibsetup #1 @NAT@ctr @ @openbib .11em \@plus.33em \@minus.07em 4000 4000 `\.\@m @bibit...

  5. [5]

    Achiam, S

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, R. Avila, I. Babuschkin, S. Balaji, V. Balcom, P. Baltescu, et al. Gpt-4 technical report, 2024

    2024

  6. [6]

    AI, 2025

    V. AI, 2025. URL https://cloud.google.com/vertex-ai

    2025

  7. [7]

    M. S. Ayhan and P. Berens. Test-time data augmentation for estimation of heteroscedastic aleatoric uncertainty in deep neural networks. In Medical Imaging with Deep Learning, 2018

    2018

  8. [8]

    Ben-David, J

    S. Ben-David, J. Blitzer, K. Crammer, and F. Pereira. Analysis of representations for domain adaptation. Advances in neural information processing systems, 19, 2006

    2006

  9. [9]

    C. Blum, K. Filippova, A. Yuan, A. Ghandeharioun, J. Zimmert, F. Zhang, J. Hoffmann, T. Linzen, M. Wattenberg, L. Dixon, et al. Beyond the rosetta stone: Unification forces in generalization dynamics. arXiv preprint arXiv:2508.11017, 2025

    Pith/arXiv arXiv 2025

  10. [10]

    Brinkmann, C

    J. Brinkmann, C. Wendler, C. Bartelt, and A. Mueller. Large language models share representations of latent grammatical concepts across typologically diverse languages. arXiv preprint arXiv:2501.06346, 2025

    Pith/arXiv arXiv 2025

  11. [12]

    L. Chua, B. Ghazi, Y. Huang, P. Kamath, R. Kumar, P. Manurangsi, A. Sinha, C. Xie, and C. Zhang. Crosslingual Capabilities and Knowledge Barriers in Multilingual Large Language Models . arXiv preprint arXiv:2406.16135, 2024 b

    Pith/arXiv arXiv 2024

  12. [13]

    Comanici, E

    G. Comanici, E. Bieber, M. Schaekermann, I. Pasupat, N. Sachdeva, I. Dhillon, M. Blistein, O. Ram, D. Zhang, E. Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261, 2025

    Pith/arXiv arXiv 2025

  13. [14]

    deepseek-ai/ D eep S eek- V 3-0324 · H ugging F ace --- huggingface.co

    Deepseek. deepseek-ai/ D eep S eek- V 3-0324 · H ugging F ace --- huggingface.co. https://huggingface.co/deepseek-ai/DeepSeek-V3-0324, 2025. [Accessed 24-09-2025]

    2025

  14. [15]

    Dumas, V

    C. Dumas, V. Veselovsky, G. Monea, R. West, and C. Wendler. How do llamas process multilingual text? a latent exploration through activation patching. In ICML 2024 Workshop on Mechanistic Interpretability, 2024

    2024

  15. [16]

    Ferrando, O

    J. Ferrando, O. Obeso, S. Rajamanoharan, and N. Nanda. Do i know this entity? knowledge awareness and hallucinations in language models. arXiv preprint arXiv:2411.14257, 2024

    Pith/arXiv arXiv 2024

  16. [17]

    Fierro, N

    C. Fierro, N. Foroutan, D. Elliott, and A. S gaard. How do multilingual language models remember facts? arXiv preprint arXiv:2410.14387, 2024

    Pith/arXiv arXiv 2024

  17. [18]

    Ganin, E

    Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. March, and V. Lempitsky. Domain-adversarial training of neural networks. Journal of machine learning research, 17 0 (59): 0 1--35, 2016

    2016

  18. [19]

    W here you can use the G emini web app - G emini A pps H elp --- support.google.com

    Gemini. W here you can use the G emini web app - G emini A pps H elp --- support.google.com. https://support.google.com/gemini/answer/13575153, 2025. [Accessed 20-09-2025]

    arXiv 2025

  19. [20]

    G emma 3 model card | G oogle A I for D evelopers --- ai.google.dev

    Gemma. G emma 3 model card | G oogle A I for D evelopers --- ai.google.dev. https://ai.google.dev/gemma/docs/core/model_card_3, 2025. [Accessed 20-09-2025]

    2025

  20. [21]

    Goldman, U

    O. Goldman, U. Shaham, D. Malkin, S. Eiger, A. Hassidim, Y. Matias, J. Maynez, A. M. Gilady, J. Riesa, S. Rijhwani, et al. Eclektic: a novel challenge set for evaluation of cross-lingual knowledge transfer. arXiv preprint arXiv:2502.21228, 2025

    arXiv 2025

  21. [22]

    G P T -5 S ystem C ard --- openai.com

    GPT. G P T -5 S ystem C ard --- openai.com. https://openai.com/index/gpt-5-system-card/, 2025. [Accessed 20-09-2025]

    2025

  22. [23]

    D. Guo, D. Yang, H. Zhang, J. Song, R. Zhang, R. Xu, Q. Zhu, S. Ma, P. Wang, X. Bi, X. Zhang, X. Yu, Y. Wu, Z. F. Wu, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025

    2025

  23. [24]

    T. Hastie. Trees bagging random forests and boosting. Standford: Stanford University, 2003

    2003

  24. [25]

    Hendrycks, C

    D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300, 2020

    Pith/arXiv arXiv 2009

  25. [26]

    Hupkes and N

    D. Hupkes and N. Bogoychev. Multiloko: a multilingual local knowledge benchmark for llms spanning 31 languages. arXiv preprint arXiv:2504.10356, 2025

    Pith/arXiv arXiv 2025

  26. [27]

    Jaech, A

    A. Jaech, A. Kalai, A. Lerer, A. Richardson, A. El-Kishky, A. Low, A. Helyar, A. Madry, A. Beutel, A. Carney, A. Iftimie, A. Karpenko, et al. Openai o1 system card, 2024

    2024

  27. [28]

    Jiang, A

    Z. Jiang, A. Anastasopoulos, J. Araki, H. Ding, and G. Neubig. X-factr: Multilingual factual knowledge retrieval from pretrained language models. arXiv preprint arXiv:2010.06189, 2020

    Pith/arXiv arXiv 2010

  28. [29]

    Kassner, P

    N. Kassner, P. Dufter, and H. Sch \"u tze. Multilingual lama: Investigating knowledge in multilingual pretrained language models. arXiv preprint arXiv:2102.00894, 2021

    Pith/arXiv arXiv 2021

  29. [30]

    W. M. Kouw and M. Loog. A review of domain adaptation without target labels. IEEE transactions on pattern analysis and machine intelligence, 43 0 (3): 0 766--785, 2019

    2019

  30. [31]

    Liu and J

    D. Liu and J. Niehues. Middle-layer representation alignment for cross-lingual transfer in fine-tuned llms. arXiv preprint arXiv:2502.14830, 2025

    Pith/arXiv arXiv 2025

  31. [32]

    M. Lu, R. Zhang, C. Eickhoff, and E. Pavlick. Paths not taken: Understanding and mending the multilingual factual recall pipeline. arXiv preprint arXiv:2505.20546, 2025

    Pith/arXiv arXiv 2025

  32. [33]

    K. Meng, D. Bau, A. Andonian, and Y. Belinkov. Locating and editing factual associations in gpt. Advances in neural information processing systems, 35: 0 17359--17372, 2022

    2022

  33. [34]

    J. Qi, R. Fern \'a ndez, and A. Bisazza. Cross-lingual consistency of factual knowledge in multilingual language models. arXiv preprint arXiv:2310.10378, 2023

    Pith/arXiv arXiv 2023

  34. [35]

    Ranaldi, G

    L. Ranaldi, G. Pucci, and A. Freitas. Empowering cross-lingual abilities of instruction-tuned large language models by translation-following demonstrations. arXiv preprint arXiv:2308.14186, 2023

    Pith/arXiv arXiv 2023

  35. [36]

    Sch \"a fer, S

    A. Sch \"a fer, S. Ravfogel, T. Hofmann, T. Pimentel, and I. Schlag. The role of language imbalance in cross-lingual generalisation: Insights from cloned language experiments. arXiv preprint arXiv:2404.07982, 2024

    Pith/arXiv arXiv 2024

  36. [37]

    Vertex AI

    G. Vertex AI. G et text embeddings | G enerative A I on V ertex A I | G oogle C loud --- cloud.google.com. https://cloud.google.com/vertex-ai/generative-ai/docs/embeddings/get-text-embeddings#supported-models, 2024. [Accessed 23-09-2025]

    2024

  37. [38]

    o tgen, and H. Sch \

    M. Wang, H. Adel, L. Lange, Y. Liu, E. Nie, J. Str \"o tgen, and H. Sch \"u tze. Lost in multilinguality: Dissecting cross-lingual factual inconsistency in transformer language models. arXiv preprint arXiv:2504.04264, 2025

    Pith/arXiv arXiv 2025

  38. [39]

    W. Wang, M. Wu, B. Haddow, and A. Birch. Bridging the language gaps in large language models with inference-time cross-lingual intervention. arXiv preprint arXiv:2410.12462, 2024 a

    Pith/arXiv arXiv 2024

  39. [40]

    Y. Wang, F. Wang, J. Dong, and H. Luo. Cl2cm: Improving cross-lingual cross-modal retrieval via cross-lingual knowledge transfer. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 5651--5659, 2024 b

    2024

  40. [41]

    Yuksekgonul, V

    M. Yuksekgonul, V. Chandrasekaran, E. Jones, S. Gunasekar, R. Naik, H. Palangi, E. Kamar, and B. Nushi. Attention satisfies: A constraint-satisfaction lens on factual errors of language models. arXiv preprint arXiv:2309.15098, 2023

    Pith/arXiv arXiv 2023