Pith. sign in

REVIEW 4 major objections 5 minor 39 references

Predicting Multilingual Classification and Translation Performance of LLMs with Cross-Lingual Alignment $\unicode{x2013}$ Is English Enough?

T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read English alignment predicts translation quality as well as direct source-target alignment.

desk verdict First solid test of whether CLA scores predict MT quality; the English-pivot evidence is real but rests on a partly circular validation of the new PMI metric. read the letter →

arxiv 2608.03446 v1 pith:ZP2RH77E submitted 2026-08-04 cs.CL

Adnan Al Ali , Kathy Hämmerl , Jindřich Libovický , Alexander Fraser This is my paper · ORCID
classification cs.CL
keywords cross-lingualalignmentmachinetranslationqualitypredictionEnglishpivotlanguagepointwisemutualinformationmetricsentenceembeddingsfromLLMsmultilingualevaluationconvexcombinationweightsfew-shotembeddingextraction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether a model's cross-lingual alignment (CLA) with English is enough to predict how well it will translate between two non-English languages. It compares 27 ways of measuring alignment and introduces a pointwise mutual information (PMI) translation metric that is less tied to the target-language writing system than chrF. Using this metric across 44 languages, the authors find that source-English and English-target alignment predict translation quality comparably to or better than direct source-target alignment, and the direct term gets near-zero weight in an optimal combination. This makes English alignment a cheap prediction tool and supports the idea that LLMs translate through an English internal representation. The exception is closely related language pairs, where direct alignment clearly carries the signal.

What carries the argument

Two devices carry the argument. First, the PMI translation metric: instead of comparing a translation against a reference, it computes $$\log \frac{p(y|x)}{p(y)}$$ under the model itself, normalizing the conditional likelihood of a candidate translation by its unconditional likelihood, making scores roughly comparable across target languages. Second, the convex-combination weight analysis: with all three alignment directions available, the paper searches over weights $\alpha$ and $\beta$ to maximize correlation between the combined alignment score and translation quality; the optimal weights reveal which alignment direction actually carries the signal.

What would settle it

Run the same convex-combination analysis with translation quality measured by human ratings or by a reference-based metric corrected for script, instead of PMI. If across 44 target languages the optimal source-target weight rises clearly above zero, or the English-side alignments stop predicting quality, the English-pivot claim would fail. A simpler check: on gemma-3 instruction-tuned, where PMI and chrF disagree, decide which metric reflects true quality; if chrF is right, the near-zero source-target result for that model is an artifact.

Watch

Extended reading notes

Core claim

The central claim is that the information an LLM's hidden representations carry about translation quality is mostly about how each language aligns with English, not how close source and target are to each other. When translation quality is measured with PMI and predicted by an optimally weighted combination of source-English, English-target, and source-target alignment scores, the source-target weight is consistently near zero or zero for the best-correlating settings across three model families. This pattern survives out-of-domain validation on BOUQuET. For seven pairs of mutually intelligible languages the reverse holds, with source-target alignment dominant. The paper also reproduces earl

Load-bearing premise

The central claim depends on the PMI metric being a faithful, near-target-language-independent measure of translation quality; the paper itself notes that full independence cannot be formally established and that one model, gemma-3 instruct on Flores, shows a notably weak PMI-chrF correlation of 26.2%.

Editorial extensions

If this is right

  • English-side alignment scores suffice as a generation-free predictor of LLM translation quality for model selection and language-pair prioritization.
  • The English-pivot account is strengthened: translation performance itself is governed by English alignment rather than the source-target relation.
  • The closely-related-language exception shows the pivot is not mandatory; when translatability is high, direct alignment becomes the better predictor.
  • Practitioners should prefer position-weighted mean embeddings plus average neuron-wise correlation (ANC), with tokenizer-level alignability as a fallback when hidden states are unavailable.
  • The PMI metric enables correlation studies across mixed target languages that chrF cannot support, because chrF clusters target languages by character information density.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the English-pivot interpretation is right, models with more English training data or later instruction tuning might show an even stronger English-target weight; this is directly measurable across more models.
  • The near-zero source-target weight could partly reflect high correlation among the three alignment directions; a decisive test would use language pairs where source-English and source-target alignment diverge sharply.
  • PMI, although proposed as a within-model research tool, is a candidate reference-free quality estimate; comparing it to human judgments would test its apparent target-language independence.
  • The related-language result suggests a boundary for the pivot hypothesis, perhaps a threshold in lexical or typological distance beyond which direct alignment outweighs English alignment.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents a comparative analysis of 27 cross-lingual alignment (CLA) score variants—obtained by combining five sentence-embedding extraction methods (including a newly proposed few-shot method), six alignment metrics, and one tokenizer-level metric—for predicting LLM performance on SIB-200 topic classification, Belebele reading comprehension, and Flores/BOUQuET machine translation. The experiments cover six model variants (Qwen3, gemma-3, Ministral-3, base and instruct) and 44 languages. To analyze translation performance across target languages, the paper introduces a PMI-based translation quality metric intended to be less target-language-dependent than chrF. The central claim is that English-centric CLAs (src-en and en-tgt) predict translation quality comparably to or better than source-target CLA, providing new evidence that LLMs use English as an internal pivot language. The analysis also includes an out-of-domain validation on BOUQuET and a related-languages side experiment.

Significance. The paper is a useful and broad empirical contribution. It substantially extends the study of CLA scores from classification to translation, provides the first comparison of 27 CLA variants on modern decoder-only LLMs, and releases code. The out-of-domain validation on BOUQuET is a notable strength, as is the inclusion of six model variants and 44 languages. The less language-dependent PMI translation metric is an interesting idea, and if its target-language independence were established, the English-pivot result would be a significant addition to the literature. However, the load-bearing evidence for the pivot claim relies critically on the PMI metric's cross-target comparability, and the current validation of that property is not conclusive. The paper also contains a circular evaluation involving the Dali metric and Belebele, and the in-sample selection of the best of 27 variants and ensemble weights is not accounted for. These issues are fixable and do not negate the value of the comparative results, but they currently prevent the strongest conclusions from being accepted as stated.

major comments (4)
  1. [Sections 6.1, 6.3, Appendix C, Limitations] The claim that en-tgt CLA predicts translation quality—and hence the English-pivot conclusion—depends on the PMI metric being comparable across target languages. The within-target correlation with chrF (Table 4) validates PMI only as a quality proxy for a fixed target; it does not validate cross-target comparability. The mixed-target evidence in Appendix C is circular: the strong correlation of the combined alignment (which includes en-tgt) with PMI is used both to establish PMI's target-language independence and to infer the pivot. The Limitations section explicitly concedes that "full independence cannot be formally established." Moreover, the Flores correlation for gemma-3-instruct is only 26.2% (Table 4), and the note to Figure 8 concedes that this leads to "wrong parameter estimation." Since the cross-target PMI analysis in Table 7 is the primary support for the English-pivot interp
  2. [Sections 3.1 and 5, Table 9] Dali scores are estimated on the Belebele dataset (Section 3.1: "Dali scores are estimated on the Belebele dataset, instead of Flores") and then correlated with Belebele accuracy in Section 5. For example, Table 9 reports Dali correlations as high as 93.1% (Ministral-3-instruct). This is circular: the predictor is derived from the same task and data distribution as the target, so the high correlation does not demonstrate that Dali predicts reading comprehension in general; it may simply reflect the model's retrieval accuracy on the same benchmark. This is especially relevant because Dali is included in the 27 variants and the paper recommends retrieval-based metrics in Section 7. Please recompute Dali on a separate dataset (or held-out portion) and re-evaluate, or clearly exclude the Belebele-based Dali results from the correlational claims.
  3. [Sections 6.2 and 6.3, Tables 6, 7, 13, 15, 17] The headline correlations in Tables 6 and 7 are the maximum over 27 CLA variants and over grid-searched ensemble weights (step size 1/25), all selected on the Flores devtest set. The BOUQuET validation later applied to the selected configuration is a good practice, but the reported in-sample numbers (e.g., ρ⊕ = 91.2% in Table 7) are subject to multiple comparisons; no correction or stability analysis is provided. More importantly, the conclusion that the src-tgt weight is "consistently near-zero or zero" (Section 6.3) is based on the argmax weights, yet Table 13 shows that adding src-tgt improves the correlation over src-en alone by at most a few points and often by zero. The weight landscape is likely flat, so the zero-weight result is underdetermined. Please report the distribution of optimal weights, confidence intervals, or a bootstrap analysis to demonstrate that the weights are not
  4. [Section 6.4 and Limitations] The related-languages experiment (Section 6.4) is presented as a boundary condition on the English-pivot claim: for close language pairs, the src-tgt weight becomes dominant (Table 7 bottom). However, this experiment uses only 14 data points (7 pairs, base models only), and the Limitations admit that "token overlap between related languages may skew the PMI and CHRF distributions." Given that PMI's target-language independence is already under question, the related-languages result cannot distinguish between the interpretation "no English pivot when translation is trivial" and the artifact "PMI is unreliable for high-overlap language pairs." This does not invalidate the experiment, but it should be interpreted much more cautiously, or supplemented with an analysis that controls for token overlap.
minor comments (5)
  1. [Table 6 caption] The caption says "optimal convex combination of en-tgt and src-tgt" but Equation (6) and the surrounding text use src-en and src-tgt. This is a typo and should be corrected.
  2. [Figure 4] The x-axis label appears to be missing the symbol for alpha (shows "weight for src-en ( )"). Please fix the formatting.
  3. [Appendix C] The phrasing "we propose that a strong correlation within mixed target languages ... indicates target-language independence. Note that this is only a one-sided implication" is logically confusing: if the correlation is the same one used to infer the pivot, the implication is not merely one-sided but circular. The distinction should be clarified.
  4. [General] The paper repeatedly refers to "CHRF" with uppercase; the standard name is "chrF" (Popović, 2015). Please use consistent notation.
  5. [Sections 2 and 3.1] In the description of prompt-based embeddings, the phrase "This sentence: \"[text]\" means in one word: \"" is rendered with escaped quotes in the text; please ensure the prompt is typeset cleanly.

Circularity Check

2 steps flagged · score 6.0 of 10

Partial circularity: PMI's target-language independence is validated by the same mixed-target correlation used to infer the English pivot, and Dali scores are computed on the Belebele benchmark they are then tested against.

  1. self definitional [Section 2 (Eq. 3), Section 3.1, Section 5; Tables 3 and 9]
    "Dali weak (Ravisankar et al., 2026) computes retrieval accuracy on a corpus where each sample consists of a context c and completions a1,...,an... Retrieval is successful if the correct source-language context-completion pair has greater cosine similarity to its target-language translation than to any incorrect target-language completion. ... We only pair Dali with the last token sentence representation... Dali scores are estimated on the Belebele dataset, instead of Flores."

    Dali is a retrieval-accuracy score over Belebele contexts and completions: a language receives a high Dali score exactly when the model can select the correct Belebele answer among distractors. Correlating Dali with Belebele accuracy therefore compares two executions of essentially the same task on the same dataset, so shared variance is forced partly by construction rather than by an independent alignment property. This inflates the variant comparison in Table 9 and makes Dali's 'prediction' of Belebele performance a renamed task score, not an independent test of cross-lingual alignment.

  2. other [Section 6.3 and Appendix C; Limitations]
    "Nonetheless, we propose that a strong correlation within mixed target languages (such as that shown in Table 7, top; for experimental setup see Section 6.3) indicates target-language independence. Note that this is only a one-sided implication... Importantly, the weight of CLA src-tgt in the optimal convex combination is consistently near-zero or zero, suggesting that most information about the translation quality is contained in the CLAsrc-en and the CLAen-tgt scores. ... full independence cannot be formally established."

    The pivot conclusion is read from the near-zero src-tgt weight in the mixed-target PMI correlation (Table 7). Appendix C then uses that same Table 7 correlation as evidence that PMI is target-language independent, which is the precondition for interpreting the weight result. This is a circular dependency: the metric's validity is asserted from the very computation that produces the conclusion. If PMI retains a target-language-specific offset, en-tgt CLA—which varies only across target languages—will correlate with that offset, and the near-zero src-tgt weight becomes a confound rather than evidence for an internal English pivot. The Limitations admit 'full independence cannot be formally established,' so the central inference and its metric validation rest on the same unverified correlatio

full rationale

The paper's main contribution is a correlational comparison rather than a formal derivation, so most of the analysis is not circular in the 'derivation equals input' sense. Two specific circular elements remain. (1) Dali is computed on Belebele and then used to predict Belebele accuracy; because Dali is a retrieval-accuracy score over the same contexts and completions, this correlation is partly self-comparison, inflating the systematic comparison of 27 CLA variants. This is localized and not the basis of the English-pivot claim. (2) The English-pivot evidence in Table 7 depends on PMI being target-language-independent, but Appendix C validates that independence by the same mixed-target correlation that yields the near-zero src-tgt weight. The Limitations explicitly concede that 'full independence cannot be formally established,' so the central inference and the validation of its metric share the same computation. If PMI retains residual target-language offsets, en-tgt CLA can spuriously capture the offset and the weight analysis is confounded. The paper does retain independent support: within a fixed target language, src-en beats src-tgt using chrF without PMI (Table 6); the BOUQuET out-of-domain validation is genuine; and the related-languages experiment shows the same protocol can produce a dominant src-tgt weight, making the near-zero result a meaningful contrast rather than a mechanical outcome. The in-sample α,β optimization adds optimism, though BOUQuET validation mitigates it. No load-bearing self-citation chain was found; self-citations such as Hämmerl et al. are not central to the claimed pivot result. Overall, there is partial circularity in the central PMI-based argument plus one localized self-definitional score, yielding 6/10.

Assumptions & free parameters 4 free parameters · 5 assumptions · 2 invented entities

The central claim rests on domain assumptions about alignment metrics, translation quality proxies, language independence, and domain representativeness. The free parameters are the fitted ensemble weights and a few hand-selected hyperparameters. No new physical or causal entity is postulated.

free parameters (4)
  • Ensemble weights alpha (src-en) and beta (en-tgt) = Varies per model and metric; only 1-alpha-beta is reported in Tables 7 and 15, often near 0 for the main language set an
    Fitted on Flores to maximize correlation with chrF or PMI, then applied to BOUQuET for validation. The near-zero src-tgt weight drives the English-pivot conclusion.
  • k in k-nearest-neighbor margin for xSIM dist and ratio = Not reported
    Equation (2) defines xSIM dist and ratio using k nearest neighbors, but the paper does not state the k value used. This affects all xSIM variants.
  • Number of few-shot examples = 3 for Belebele, 5 for translation, 5 for few-shot embeddings
    Hand-selected prompts; the paper does not justify these counts or test sensitivity to them.
  • Wikidata entity selection for few-shot embeddings = First five entities by QID with descriptions at least 10 characters
    Ad hoc selection rule in Section 3.1; no justification for why these entities or this threshold, and the authors note Wikidata description quality varies across languages.
assumptions (5)
  • domain assumption Alignment scores computed at the layer with maximum alignment are suitable for predicting downstream performance.
    Section 3.1 says the layer with maximum alignment score is selected, following Kargaran et al. (2025). This choice is not validated against downstream task performance.
  • domain assumption chrF is a trustworthy reference metric for translation quality, and PMI is valid to the extent it correlates with chrF.
    Section 6.1 validates PMI only by correlation with chrF, not with human judgments or other semantic metrics. chrF itself is target-language dependent.
  • domain assumption The PMI prior log p(y), computed from a target-language-only prompt, approximates a true unconditional target-language probability.
    Section 6.1 and Listing 7 define the prior using only target-language sentences. If the prior is misspecified, PMI still reflects target-language LM preferences and is not fully language-independent.
  • domain assumption Pearson correlations across languages can be interpreted as independent evidence, with no correction for language-family or script non-independence.
    Section 3 uses Pearson correlation across 44 languages, and Figure 5 shows clear script-based clusters. This can inflate correlation strength.
  • domain assumption Correlations computed on Flores are representative for other domains despite domain overlap between alignment and downstream data.
    The Limitations section admits that the shared domain may inflate correlations. BOUQuET partially addresses this for translation, but not for SIB-200 or Belebele.
invented entities (2)
  • PMI translation metric (log p(y|x)/p(y)) independent evidence
    purpose: Estimate translation quality with lower target-language dependence so en-tgt CLA can be compared across target languages.
    Validated by correlation with chrF in controlled settings, but not formally target-language independent; the authors concede this in Limitations.
  • few-shot sentence embedding extraction
    purpose: Produce sentence embeddings without an English prompt, avoiding implicit English-alignment bias.
    No external benchmark of embedding quality; depends on hand-chosen Wikidata entities and is only evaluated through downstream correlations in this paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Predicting Multilingual Classification and Translation Performance of LLMs with Cross-Lingual Alignment $\unicode{x2013}$ Is English Enough?." pith.science (2026). https://pith.science/paper/ZP2RH77E

@misc{pith2026260803446,
  author       = {Pith},
  title        = {Pith review of: Predicting Multilingual Classification and Translation Performance of LLMs with Cross-Lingual Alignment $\unicodex2013$ Is English Enough?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZP2RH77E}},
  note         = {Machine review of arXiv:2608.03446}
}
read the original abstract

Multilingual large language models (LLMs) have been shown to perform better on non-English classification tasks when the representations of the given language are more aligned to English within the model. Several cross-lingual alignment (CLA) scores have been proposed for use with LLMs, along with multiple approaches for extracting embeddings from the models. We provide a comparative analysis of 27 CLA score variants, examining how they differ and how well each predicts downstream performance across three tasks. Crucially, while LLMs are widely used for generative tasks such as machine translation, prior work has focused almost exclusively on classification. We therefore investigate whether CLA scores are similarly predictive of translation performance. To enable computing correlations across target languages, we propose a PMI-based translation metric, which is less dependent on the target language and correlates strongly with chrF. We find that CLA with English predicts translation quality comparably to or better than source-target CLA, providing new evidence that LLMs use English as an internal pivot language.

Figures

Figures reproduced from arXiv: 2608.03446 by the authors.

Figure 1
Figure 1. Example two-shot prompting conversation for [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Macro F1 score distribution across languages for SIB-200. Cropped outliers are marked by ▼. (Section 6). 4 SIB-200 Benchmark The task in SIB-200 (Adelani et al., 2024) is to categorise a text segment into one of seven topics. As the category is in English, we use an English prompt, stated in Listings 2 (base models) and 6 (instruction-tuned models) in Appendix B. We com￾pute the macro-averaged F1 score on the train … view at source ↗
Figure 4
Figure 4. Per-target-language average Pearson correla [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figures from the paper (5 more)
Figure 5
Figure 5. Figure 5: CHRF (right) clusters target languages by character information density, while clusters are considerably less pronounced for PMI (left). Each point is a language pair; non-Latin-script target languages are in orange. Both metrics are correlated with their respective op…
Figure 6
Figure 6. Figure 6: Correlation heatmap for the α and β grid search for the best-performing sentence representa￾tion/CLA metric pair for Qwen3 instr. when predicting Flores translation PMI. Values on the diagonal corres￾pond to zero src-tgt weight. The best correlation is marked red. The …
Figure 7
Figure 7. Figure 7: Comparison of how translation PMI (left) and [PITH_FULL_IMAGE:figures/full_fig_p021_7.png]
Figure 8
Figure 8. Figure 8: Comparison of how translation PMI (left) and [PITH_FULL_IMAGE:figures/full_fig_p022_8.png]
Figure 9
Figure 9. Figure 9: Comparison of how translation PMI (left) and [PITH_FULL_IMAGE:figures/full_fig_p023_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 7 canonical work pages

  1. [1]

    MEXA : Multilingual Evaluation of E nglish-Centric LLM s via Cross-Lingual Alignment

    Kargaran, Amir Hossein and Modarressi, Ali and Nikeghbal, Nafiseh and Diesner, Jana and Yvon, Fran c ois and Schuetze, Hinrich. MEXA : Multilingual Evaluation of E nglish-Centric LLM s via Cross-Lingual Alignment. Findings of the Association for Computational Linguistics: ACL 2025. 2025. doi:10.18653/v1/2025.findings-acl.1385

    work page doi:10.18653/v1/2025.findings-acl.1385 2025

  2. [2]

    Cross-lingual Similarity of Multilingual Representations Revisited

    Del, Maksym and Fishel, Mark. Cross-lingual Similarity of Multilingual Representations Revisited. Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2022. doi:10.18653/v1/2022.aacl-main.15

    work page doi:10.18653/v1/2022.aacl-main.15 2022

  3. [3]

    Margin-based Parallel Corpus Mining with Multilingual Sentence Embeddings

    Artetxe, Mikel and Schwenk, Holger. Margin-based Parallel Corpus Mining with Multilingual Sentence Embeddings. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019. doi:10.18653/v1/P19-1309

    doi:10.18653/v1/p19-1309 2019

  4. [4]

    Beyond Literal Token Overlap: Token Alignability for Multilinguality

    H. Beyond Literal Token Overlap: Token Alignability for Multilinguality. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers). 2025. doi:10.18653/v1/2025.naacl-short.63

    doi:10.18653/v1/2025.naacl-short.63 2025

  5. [5]

    Tokenization Impacts Multilingual Language Modeling: Assessing Vocabulary Allocation and Overlap Across Languages

    Limisiewicz, Tomasz and Balhar, Ji r \'i and Mare c ek, David. Tokenization Impacts Multilingual Language Modeling: Assessing Vocabulary Allocation and Overlap Across Languages. Findings of the Association for Computational Linguistics: ACL 2023. 2023. doi:10.18653/v1/2023.findings-acl.350

    doi:10.18653/v1/2023.findings-acl.350 2023

  6. [6]

    2026 , eprint=

    Can you map it to English? The Role of Cross-Lingual Alignment in Multilingual Performance of LLMs , author=. 2026 , eprint=

    work page 2026

  7. [7]

    Do Llamas Work in E nglish? On the Latent Language of Multilingual Transformers

    Wendler, Chris and Veselovsky, Veniamin and Monea, Giovanni and West, Robert. Do Llamas Work in E nglish? On the Latent Language of Multilingual Transformers. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024. doi:10.18653/v1/2024.acl-long.820

    doi:10.18653/v1/2024.acl-long.820 2024

  8. [8]

    and Mao, Yanke and Gao, Haonan and Lee, En-Shiun Annie

    Adelani, David Ifeoluwa and Liu, Hannah and Shen, Xiaoyu and Vassilyev, Nikita and Alabi, Jesujoba O. and Mao, Yanke and Gao, Haonan and Lee, En-Shiun Annie. SIB -200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialects. Proceedings of the 18th Conference of the European Chapter of the Association for Co...

    doi:10.18653/v1/2024.eacl-long.14 2024

Show all 39 references
  1. [9]

    The B elebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variants

    Bandarkar, Lucas and Liang, Davis and Muller, Benjamin and Artetxe, Mikel and Shukla, Satya Narayan and Husa, Donald and Goyal, Naman and Krishnan, Abhinandan and Zettlemoyer, Luke and Khabsa, Madian. The B elebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Lan...

    2024

  2. [10]

    2022 , eprint=

    No Language Left Behind: Scaling Human-Centered Machine Translation , author=. 2022 , eprint=

    2022

  3. [11]

    The F lores-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation

    Goyal, Naman and Gao, Cynthia and Chaudhary, Vishrav and Chen, Peng-Jen and Wenzek, Guillaume and Ju, Da and Krishnan, Sanjana and Ranzato, Marc ' Aurelio and Guzm \'a n, Francisco and Fan, Angela. The F lores-101 Evaluation Benchmark for Low-Resource and Multilingual Machine ...

    2022 doi

  4. [12]

    Emerging Cross-lingual Structure in Pretrained Language Models

    Conneau, Alexis and Wu, Shijie and Li, Haoran and Zettlemoyer, Luke and Stoyanov, Veselin. Emerging Cross-lingual Structure in Pretrained Language Models. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020. doi:10.18653/v1/2020.acl-main.536

    2020 doi

  5. [13]

    How Contextual are Contextualized Word Representations? C omparing the Geometry of BERT , ELM o, and GPT -2 Embeddings

    Ethayarajh, Kawin. How Contextual are Contextualized Word Representations? C omparing the Geometry of BERT , ELM o, and GPT -2 Embeddings. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natur...

    2019 doi

  6. [14]

    P rompt BERT : Improving BERT Sentence Embeddings with Prompts

    Jiang, Ting and Jiao, Jian and Huang, Shaohan and Zhang, Zihan and Wang, Deqing and Zhuang, Fuzhen and Wei, Furu and Huang, Haizhen and Deng, Denvy and Zhang, Qi. P rompt BERT : Improving BERT Sentence Embeddings with Prompts. Proceedings of the 2022 Conference on Empirical Me...

    2022 doi

  7. [15]

    BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding

    Devlin, Jacob and Chang, Ming-Wei and Lee, Kenton and Toutanova, Kristina. BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Hum...

    2019 doi

  8. [16]

    Attention is All you Need , url =

    Vaswani, Ashish and Shazeer, Noam and Parmar, Niki and Uszkoreit, Jakob and Jones, Llion and Gomez, Aidan N and Kaiser, ukasz and Polosukhin, Illia , booktitle =. Attention is All you Need , url =

  9. [17]

    2202.08904 , archivePrefix=

    Niklas Muennighoff , year=. 2202.08904 , archivePrefix=

    arXiv

  10. [18]

    Scaling Sentence Embeddings with Large Language Models

    Jiang, Ting and Huang, Shaohan and Luan, Zhongzhi and Wang, Deqing and Zhuang, Fuzhen. Scaling Sentence Embeddings with Large Language Models. Findings of the Association for Computational Linguistics: EMNLP 2024. 2024. doi:10.18653/v1/2024.findings-emnlp.181

    2024 doi

  11. [19]

    D ef S ent: Sentence Embeddings using Definition Sentences

    Tsukagoshi, Hayato and Sasano, Ryohei and Takeda, Koichi. D ef S ent: Sentence Embeddings using Definition Sentences. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Process...

    2021 doi

  12. [20]

    2025 , eprint=

    Qwen3 Technical Report , author=. 2025 , eprint=

    2025

  13. [21]

    2026 , eprint=

    Ministral 3 , author=. 2026 , eprint=

    2026

  14. [22]

    2025 , eprint=

    Gemma 3 Technical Report , author=. 2025 , eprint=

    2025

  15. [23]

    Brown, Tom B. and Mann, Benjamin and Ryder, Nick and Subbiah, Melanie and Kaplan, Jared and Dhariwal, Prafulla and Neelakantan, Arvind and Shyam, Pranav and Sastry, Girish and Askell, Amanda and Agarwal, Sandhini and Herbert-Voss, Ariel and Krueger, Gretchen and Henighan, Tom ...

    2020

  16. [24]

    Communications of the ACM , volume=

    Wikidata: a free collaborative knowledgebase , author=. Communications of the ACM , volume=. 2014 , publisher=

    2014

  17. [25]

    chr F : character n-gram F -score for automatic MT evaluation

    Popovi \'c , Maja. chr F : character n-gram F -score for automatic MT evaluation. Proceedings of the Tenth Workshop on Statistical Machine Translation. 2015. doi:10.18653/v1/W15-3049

    2015 doi

  18. [26]

    Global MMLU : Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation

    Singh, Shivalika and Romanou, Angelika and Fourrier, Cl \'e mentine and Adelani, David Ifeoluwa and Ngui, Jian Gang and Vila-Suero, Daniel and Limkonchotiwat, Peerat and Marchisio, Kelly and Leong, Wei Qi and Susanto, Yosephine and Ng, Raymond and Longpre, Shayne and Ruder, Se...

    2025

  19. [27]

    2024 , eprint=

    GPT-4 Technical Report , author=. 2024 , eprint=

    2024

  20. [28]

    2023 , eprint=

    Llama 2: Open Foundation and Fine-Tuned Chat Models , author=. 2023 , eprint=

    2023

  21. [29]

    Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERT

    Wu, Shijie and Dredze, Mark. Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERT. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)....

    2019 doi

  22. [30]

    The Prague Bulletin of Mathematical Linguistics , volume=

    Efficient word alignment with markov chain monte carlo , author=. The Prague Bulletin of Mathematical Linguistics , volume=. 2016 , publisher=

    2016

  23. [31]

    Australian & New Zealand Journal of Statistics , volume =

    Baba, Kunihiro and Shibata, Ritei and Sibuya, Masaaki , title =. Australian & New Zealand Journal of Statistics , volume =. doi:https://doi.org/10.1111/j.1467-842X.2004.00360.x , url =. https://onlinelibrary.wiley.com/doi/pdf/10.1111/j.1467-842X.2004.00360.x , abstract =

    2004 arXiv

  24. [32]

    Understanding Cross-Lingual A lignment --- A Survey

    H. Understanding Cross-Lingual A lignment --- A Survey. Findings of the Association for Computational Linguistics: ACL 2024. 2024. doi:10.18653/v1/2024.findings-acl.649

    2024 doi

  25. [33]

    Languages Transferred Within the Encoder: On Representation Transfer in Zero-Shot Multilingual Translation

    Qu, Zhi and Ding, Chenchen and Watanabe, Taro. Languages Transferred Within the Encoder: On Representation Transfer in Zero-Shot Multilingual Translation. Proceedings of Machine Translation Summit XX: Volume 1. 2025

    2025

  26. [34]

    Investigating Multilingual NMT Representations at Scale

    Kudugunta, Sneha and Bapna, Ankur and Caswell, Isaac and Firat, Orhan. Investigating Multilingual NMT Representations at Scale. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language...

    2019 doi

  27. [35]

    Pivot-based Transfer Learning for Neural Machine Translation between Non- E nglish Languages

    Kim, Yunsu and Petrov, Petre and Petrushkov, Pavel and Khadivi, Shahram and Ney, Hermann. Pivot-based Transfer Learning for Neural Machine Translation between Non- E nglish Languages. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and th...

    2019 doi

  28. [36]

    Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles , year=

    Efficient Memory Management for Large Language Model Serving with PagedAttention , author=. Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles , year=

  29. [37]

    Thomas Wolf and Lysandre Debut and Victor Sanh and Julien Chaumond and Clement Delangue and Anthony Moi and Pierric Cistac and Tim Rault and Rémi Louf and Morgan Funtowicz and Joe Davison and Sam Shleifer and Patrick von Platen and Clara Ma and Yacine Jernite and Julien Plu an...

    2020

  30. [38]

    Chain-of-Thought Prompting Elicits Reasoning in Large Language Models , url =

    Wei, Jason and Wang, Xuezhi and Schuurmans, Dale and Bosma, Maarten and ichter, brian and Xia, Fei and Chi, Ed and Le, Quoc V and Zhou, Denny , booktitle =. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models , url =

  31. [39]

    Andrews, Pierre and Artetxe, Mikel and Meglioli, Mariano Coria and Costa-juss \`a , Marta R. and Chuang, Joe and Dale, David and Duppenthaler, Mark and Ekberg, Nathanial Paul and Gao, Cynthia and Licht, Daniel Edward and Maillard, Jean and Mourachko, Alexandre and Ropers, Chri...

    2025

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.