Pith. sign in

REVIEW 4 major objections 4 minor 35 references

Evaluating the Cross-Lingual Effectiveness of Massively Multilingual Neural Machine Translation

T0 review · 4 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A single machine translation encoder trained on 102 languages to and from English beats multilingual BERT on zero-shot cross-lingual transfer in four of five tasks, including new state-of-the-art results on part-of-speech tagging and…

desk verdict A genuinely useful empirical study of a 103-language NMT encoder for cross-lingual transfer, but the headline comparison to mBERT is not fully controlled; the paper deserves review with a request for matched baselines and variance estimates. read the letter →

arxiv 1909.00437 v1 pith:Y7NZMU5A submitted 2019-09-01 cs.CL

Aditya Siddhant , Melvin Johnson , Henry Tsai , Naveen Arivazhagan , Jason Riesa , Ankur Bapna , Orhan Firat , Karthik Raman This is my paper · ORCID
classification cs.CL
keywords massivelymultilingualNMTcross-lingualtransferzero-shotlearningBERTpart-of-speechtaggingintentclassificationnamedentityrecognitionrepresentations
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether the encoder of a single neural machine translation model trained on 102 languages can serve as a shared representation layer for other NLP tasks, the way multilingual masked-language models have been shown to. The authors pre-train a 103-language translation encoder (MMTE), fine-tune it on five classification and sequence-labeling tasks spanning over 50 languages, and compare it with multilingual BERT (mBERT). They report that MMTE beats mBERT in zero-shot cross-lingual transfer on four of the five tasks, sets a new state of the art on zero-shot part-of-speech tagging and intent classification, and matches it on document classification. If the result holds, a supervised translation objective trained on noisy web-scale parallel text is a viable alternative to unsupervised multilingual pre-training for cross-lingual transfer, with the largest gains on low-resource languages.

What carries the argument

The central object is the Massively Multilingual Translation Encoder (MMTE): the encoder of a Transformer-Big NMT model (6 layers, 16 attention heads, 8192 hidden units, roughly 190M parameters) trained with a single shared vocabulary of 64k sentence-piece tokens to translate 102 languages to and from English. The mechanism that makes it transferable is the target-language token prepended to each source sentence, which lets one shared encoder-decoder handle all language pairs, plus temperature-based sampling (T=5) that up-weights low-resource pairs so the encoder sees diverse languages. During fine-tuning, token-level encoder outputs are fed to task heads, with max-pooling for sentence tasks and first-subword pooling for tagging. The paper's analysis shows that scaling language count from 4 to 103 degrades zero-shot transfer, and that removing the target token from the encoder input during pre-training improves downstream transfer, implicating both model capacity and the pre-training input format as the levers that carry the effect.

What would settle it

Run a fully controlled comparison on a fixed set of languages: train two encoders with identical architecture, vocabulary, and fine-tuning protocol, one on a translation objective and one on masked language modeling, using pre-training corpora of matched size and noise level. If MMTE's zero-shot advantage over mBERT on POS tagging and intent classification does not survive, or if its NER deficit persists, the paper's attribution of cross-lingual effectiveness to the translation objective is falsified.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central claim is that a translation-trained encoder transfers across languages at least as effectively as a masked-language-model encoder. MMTE, the encoder of a 103-language NMT system trained with a plain negative-log-likelihood translation loss on 25 billion parallel sentence pairs, is fine-tuned end-to-end on downstream tasks with a small task head. In zero-shot transfer it outperforms mBERT on average on XNLI (67.5 vs 66.3 accuracy), intent classification (94.2 vs 70.6), and POS tagging (75.43 vs 74.87 F1), edges ahead on MLDoc document classification (74.7 vs 74.5), and trails substantially on NER (60.6 vs 78.5 F1). The paper reports that this establishes a new state of the art for zero-shot POS tagging and intent classification, and that the gains concentrate in low-resource languages. The NER failure is attributed to the entity-poor, noisy web text used for translation pre-training.

Load-bearing premise

The comparison between MMTE and mBERT is treated as a controlled test of representation quality, but the two models differ in pre-training data (noisy parallel web text versus clean Wikipedia), vocabulary, architecture depth and width, and the provenance of the mBERT baselines (two tasks from a prior paper, three from the authors' own fine-tuning), so if these differences rather than the translation objective drive the 4-of-5 zero-shot advantage, the central claim is unsupported.

Editorial extensions

If this is right

  • Translation pre-training can serve as a competitive alternative to masked-language-model pre-training for zero-shot cross-lingual classification and sequence labeling, so task labels in English can be projected to dozens of languages without target-language annotations.
  • Token-level representations from a multilingual NMT encoder are usable for sequence tagging tasks such as POS and NER, not just for sentence-level classification.
  • The gains on low-resource languages mean that a translation-trained encoder is most useful precisely where labeled target data is scarcest.
  • Scaling the number of pre-trained languages from 4 to 103 degrades zero-shot transfer, indicating a capacity trade-off that future multilingual models must address with larger models or grouped training.
  • Encoding the target-language token separately rather than prepending it to the source improves downstream zero-shot transfer while keeping translation quality comparable, suggesting a cleaner pre-training format.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the two models differ in pre-training data, vocabulary, architecture, and baseline provenance, the cleanest reading of the paper is that a translation objective can produce transferable representations, not necessarily that it beats masked-language modeling by itself; a matched comparison would be needed to separate the objective from these other factors.
  • The NER result suggests a testable extension: supplementing translation data with entity-rich or Wikipedia-style text during pre-training should recover most of the 18-point zero-shot NER gap while preserving the POS and intent gains.
  • The language-count ablation implies that the optimal design for cross-lingual transfer may be a many-small-models strategy, for example one encoder per language family or script group, rather than a single encoder for all 100+ languages.
  • Mixing the translation objective with a masked-language-model objective, which the paper itself flags as future work, is the most direct next experiment; the target-token ablation already indicates how the translation input should be formatted for that hybrid to work.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper evaluates representations from the encoder of a massively multilingual NMT system (MMTE), pre-trained on 25B parallel sentence pairs covering 103 languages, on five downstream tasks: XNLI, MLDoc document classification, multilingual intent classification, UD v2.3 POS tagging, and CoNLL NER. It compares MMTE against multilingual BERT (mBERT) in zero-shot, few-shot, fine-tuning, and feature-extraction settings, reporting that MMTE outperforms mBERT in zero-shot transfer on four of five tasks and claiming new state-of-the-art results on POS tagging and intent classification. The paper also presents ablations on the number of pre-training languages and on the target-language token.

Significance. If the empirical claims are supported, the paper would be a useful contribution: it shows that a supervised translation objective at massive scale can produce token-level, cross-lingually transferable representations competitive with mBERT on several public benchmarks, and it broadens the evidence beyond sentence-level pooled embeddings. The strengths are the breadth of tasks and languages, the inclusion of several practically relevant transfer settings (zero-shot, few-shot, feature-based, joint training), and the use of publicly available evaluation datasets. The ablations on language count and the target-language token are also informative. The evaluation is against external benchmarks, so the central metrics are not circular. The main weakness is experimental control: the mBERT comparison mixes baselines from different sources and protocols, and all headline numbers are single runs without variance, which weakens the central 4-of-5 claim.

major comments (4)
  1. [§3, Tables 2–6] The central comparison between MMTE and mBERT is not a controlled test of representation quality. For XNLI, MLDoc, and NER, the mBERT numbers are quoted from Wu and Dredze (2019), while for intent classification and POS tagging they are from the authors' own implementation using the public mBERT checkpoint. The paper does not state whether the fine-tuning protocol—task-specific head, pooling, optimizer schedule, early stopping, and seed selection—was matched between MMTE and mBERT in either case. This is load-bearing because the abstract's claim of gains in 4 of 5 tasks and the SOTA claims rely directly on these comparisons. The largest reported margins (e.g., zero-shot intent classification: 93.6 vs. 69.2 for Spanish and 89.6 vs. 43.4 for Thai in Table 4) are precisely where differences in fine-tuning recipe can dominate. The authors should re-run mBERT with the identical pipeline used for MMTE, including the same task heads and hyperparameters, and report those numbers as the primary comparison.
  2. [§3, Tables 2–6] All main results are reported as single runs without variance. Several of the claimed advantages are small: +1.2 average accuracy on XNLI (Table 2), +0.2 average zero-shot accuracy on MLDoc (Table 3), and +0.56 average zero-shot F1 on POS (Table 5). These margins are within the range of run-to-run variation commonly observed when fine-tuning large pre-trained transformers, especially with different random seeds. The authors should report means and standard deviations over multiple seeds, or at least demonstrate that the qualitative pattern of 4-of-5 wins is stable across seeds, before drawing conclusions about the relative effectiveness of MMTE and mBERT.
  3. [§3.4, Table 5] The claim of state-of-the-art zero-shot cross-lingual POS tagging is computed on the development set, not on a held-out test set. Table 5 reports F1 scores on the dev set of the UD v2.3 data, and the abstract and contributions state SOTA on this dataset without this qualification. Test-set scores can differ materially from dev-set scores, and comparisons to prior work usually refer to test sets. The authors should either report test-set results or explicitly and prominently restrict the SOTA claim to the dev set.
  4. [§2.2, §6] The comparison conflates the pre-training objective with data scale and data type. MMTE is pre-trained on 25 billion parallel sentence pairs from noisy web crawls, while mBERT is pre-trained on Wikipedia text in 104 languages. Even with perfectly matched fine-tuning, the observed differences between MMTE and mBERT cannot be attributed solely to the translation objective versus the masked-language-model objective. The conclusion in Section 6 that "the translation objective is competitive with specialized approaches to learn cross-lingual embeddings" is therefore stronger than the experimental design supports. The authors should either soften this attribution or add an ablation that controls for data source and scale, for example by pre-training a translation model on Wikipedia-scale parallel data or by comparing to an mBERT variant trained on the same parallel corpus.
minor comments (4)
  1. [§3.1] The reference to "Appedix Table 1" should be corrected to the actual appendix table number (Table 12 in the supplementary material).
  2. [§4, Tables 8–11] The few-shot subsection refers to "results ... in 8" without a table number, and Tables 8, 9, 10, and 11 lack descriptive captions; adding captions would improve readability.
  3. [Table 5] The table reports a "48L" average row but does not define whether the average is macro-averaged over languages or weighted by dataset size; this should be stated in the caption.
  4. [§5] The sentence "Further, neither concern themselves with the performance of the actual translation task whereas we our mNMT model performs comparable to bilingual baselines" contains a grammatical error ("we our") and should be rewritten.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: MMTE is evaluated against external public benchmarks, and the cross-lingual effectiveness claim is an empirical comparison rather than a derivation from fitted inputs.

full rationale

This paper is an empirical evaluation, not a derivation. The central claim that MMTE encoder representations outperform mBERT in zero-shot transfer on 4 of 5 tasks is supported by measured accuracies and F1 scores on external datasets: XNLI, MLDoc, intent classification, Universal Dependencies POS tagging, and CoNLL NER. No downstream parameter is fitted to the reported test quantities in a way that forces the comparisons; the task-specific heads are trained on English training splits and then evaluated on unseen target languages, so zero-shot transfer scores are genuine measurements. The self-citations (Aharoni et al., 2019; Johnson et al., 2017; Chen et al., 2018; Shen et al., 2019) describe the pre-training architecture, data sampling strategy, and infrastructure, but the downstream outcomes are not defined by those papers. The state-of-the-art comparisons use external baselines such as Wu and Dredze (2019), Schuster et al. (2018), and Lample and Conneau (2019). The concern that mBERT numbers come from different fine-tuning protocols or single runs is a question of experimental validity and reproducibility, not definitional circularity. Likewise, reporting the POS result on the dev set is a benchmark-protocol detail, not a circular step. The paper is self-contained against external benchmarks, so no circularity is found.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

This is an empirical comparison paper with no closed-form derivation. The free parameters are hand-chosen hyperparameters that influence the reported numbers. The axioms are the comparability, data, and evaluation assumptions on which the central claim rests. No new entities are postulated.

free parameters (6)
  • Data sampling temperature T = 5
    Controls the balance between high- and low-resource language pairs during mNMT pre-training (Section 2.2). Chosen by hand and directly affects which languages the encoder sees more often.
  • XNLI fine-tuning learning rate schedule = (0.2, 90k)
    Peak learning rate 0.2 with 90k warmup steps, used for the XNLI classifier (Section 3.1).
  • MLDoc fine-tuning learning rate schedule = (0.2, 5k)
    Peak learning rate 0.2 with 5k warmup steps, used for document classification (Section 3.2).
  • Intent classification learning rate schedule = (0.1, 100k)
    Peak learning rate 0.1 with 100k warmup steps, used for intent classification (Section 3.3).
  • POS and NER learning rate schedule = (0.1, 40k)
    Peak learning rate 0.1 with 40k warmup steps, used for POS tagging and NER (Sections 3.4 and 3.5).
  • Transformer Big encoder configuration = 6 layers, 16 heads, 8192 hidden
    Larger Transformer architecture used for mNMT pre-training (Section 2.1); a scaling choice not derived from first principles.
assumptions (4)
  • ad hoc to paper mBERT baseline numbers from different sources are directly comparable to MMTE numbers obtained with the authors' fine-tuning protocol.
    Tables 2, 3, and 6 use mBERT numbers from Wu and Dredze (2019), while Tables 4 and 5 use the authors' own implementation. Fine-tuning details and task heads may differ, so differences could reflect protocol rather than representation quality.
  • domain assumption The proprietary 25B sentence-pair web-crawled corpus is an appropriate pre-training resource for learning cross-lingual representations.
    The corpus is not released or described in detail (Section 2.2); its quality and composition are assumed to be suitable.
  • domain assumption Zero-shot evaluation with English-only training and target-language testing measures cross-lingual transfer fairly.
    All zero-shot setups in Section 3 use English-only training and evaluate on target languages; this assumes no target-language contamination and that the task setup is equivalent across languages.
  • standard math The Transformer and Adafactor implementations in Lingvo are correct and used with the stated hyperparameters.
    The paper relies on the open-source Lingvo framework (Shen et al., 2019) with specified settings (Section 2.1).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Evaluating the Cross-Lingual Effectiveness of Massively Multilingual Neural Machine Translation." pith.science (2026). https://pith.science/paper/Y7NZMU5A

@misc{pith2026190900437,
  author       = {Pith},
  title        = {Pith review of: Evaluating the Cross-Lingual Effectiveness of Massively Multilingual Neural Machine Translation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Y7NZMU5A}},
  note         = {Machine review of arXiv:1909.00437}
}
read the original abstract

The recently proposed massively multilingual neural machine translation (NMT) system has been shown to be capable of translating over 100 languages to and from English within a single model. Its improved translation performance on low resource languages hints at potential cross-lingual transfer capability for downstream tasks. In this paper, we evaluate the cross-lingual effectiveness of representations from the encoder of a massively multilingual NMT model on 5 downstream classification and sequence labeling tasks covering a diverse set of over 50 languages. We compare against a strong baseline, multilingual BERT (mBERT), in different cross-lingual transfer learning scenarios and show gains in zero-shot transfer in 4 out of these 5 tasks.

Figures

Figures reproduced from arXiv: 1909.00437 by the authors.

Figure 1
Figure 1. Per language pair data distribution of the [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 13 canonical work pages

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Roee Aharoni, Melvin Johnson, and Orhan Firat. 2019. Massively multilingual neural machine translation. arXiv preprint arXiv:1903.00089

    arXiv 2019

  4. [4]

    Mikel Artetxe and Holger Schwenk. 2018. Massively multilingual sentence embeddings for zero-shot cross-lingual transfer and beyond. arXiv preprint arXiv:1812.10464

    arXiv 2018

  5. [5]

    Mia Xu Chen, Orhan Firat, Ankur Bapna, Melvin Johnson, Wolfgang Macherey, George Foster, Llion Jones, Mike Schuster, Noam Shazeer, Niki Parmar, Ashish Vaswani, Jakob Uszkoreit, Lukasz Kaiser, Zhifeng Chen, Yonghui Wu, and Macduff Hughes. 2018. http://www.aclweb.org/anthology/P18-1008 The best of both worlds: Combining recent advances in neural machine tra...

    2018

  6. [6]

    Bowman, Holger Schwenk, and Veselin Stoyanov

    Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel R. Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. Xnli: Evaluating cross-lingual sentence representations. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics

    2018

  7. [7]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

    arXiv 2018

  8. [8]

    Akiko Eriguchi, Melvin Johnson, Orhan Firat, Hideto Kazawa, and Wolfgang Macherey. 2018. Zero-shot cross-lingual classification using multilingual neural machine translation. arXiv preprint arXiv:1809.04686

    arXiv 2018

Show all 35 references
  1. [9]

    Manaal Faruqui and Chris Dyer. 2014. Improving vector space word representations using multilingual correlation. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 462--471

    2014

  2. [10]

    Orhan Firat, Kyunghyun Cho, and Yoshua Bengio. 2016. Multi-way, multilingual neural machine translation with a shared attention mechanism. arXiv preprint arXiv:1601.01073

    2016 arXiv

  3. [11]

    Jeremy Howard and Sebastian Ruder. 2018. Universal language model fine-tuning for text classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 328--339

    2018

  4. [12]

    Melvin Johnson, Mike Schuster, Quoc V Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Vi \'e gas, Martin Wattenberg, Greg Corrado, et al. 2017. Google's multilingual neural machine translation system: Enabling zero-shot translation. Transactions of the Asso...

    2017

  5. [13]

    Taku Kudo and John Richardson. 2018. Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 66--71

    2018

  6. [14]

    Guillaume Lample and Alexis Conneau. 2019. Cross-lingual language model pretraining. arXiv preprint arXiv:1901.07291

    2019 arXiv

  7. [15]

    Bryan McCann, James Bradbury, Caiming Xiong, and Richard Socher. 2017. Learned in translation: Contextualized word vectors. In Advances in Neural Information Processing Systems, pages 6294--6305

    2017

  8. [16]

    Phoebe Mulcaire, Jungo Kasai, and Noah Smith. 2019. Polyglot contextual representations improve crosslingual transfer. arXiv preprint arXiv:1902.09697

    2019 arXiv

  9. [17]

    Joakim Nivre, Mitchell Abrams, Z eljko Agi \'c , Lars Ahrenberg, Lene Antonsen, Maria Jesus Aranzabe, Gashaw Arutie, Masayuki Asahara, Luma Ateyah, Mohammed Attia, et al. 2018. Universal dependencies 2.2

    2018

  10. [18]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics, pages 311--318. Association for Computational Linguistics

    2002

  11. [19]

    Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: H...

    2018

  12. [20]

    Telmo Pires, Eva Schlinger, and Dan Garette. 2019. How multilingual is multilingual bert ? In Proceedings of Association of Computational Linguistics (ACL)

    2019

  13. [21]

    Improving language understanding by generative pre-training

    Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. Improving language understanding by generative pre-training

  14. [22]

    Sebastian Ruder, Ivan Vuli \'c , and Anders S gaard. 2017. A survey of cross-lingual word embedding models. arXiv preprint arXiv:1706.04902

    2017 arXiv

  15. [23]

    Tjong Kim Sang

    Erik F. Tjong Kim Sang. 2002. http://arxiv.org/abs/cs.CL/0209010 Introduction to the conll-2002 shared task: Language-independent named entity recognition . CoRR, cs.CL/0209010

    2002

  16. [24]

    Erik F Tjong Kim Sang and Fien De Meulder. 2003. Introduction to the conll-2003 shared task: Language-independent named entity recognition. In Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003

    2003

  17. [25]

    Sebastian Schuster, Sonal Gupta, Rushin Shah, and Mike Lewis. 2018. Cross-lingual transfer learning for multilingual task oriented dialog. arXiv preprint arXiv:1810.13327

    2018 arXiv

  18. [26]

    Holger Schwenk and Xian Li. 2018. A corpus for multilingual document classification in eight languages. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Paris, France. European Language Resources Association (ELRA)

    2018

  19. [27]

    Noam Shazeer and Mitchell Stern. 2018. Adafactor: Adaptive learning rates with sublinear memory cost. arXiv preprint arXiv:1804.04235

    2018 arXiv

  20. [28]

    Jonathan Shen, Patrick Nguyen, Yonghui Wu, and Zhifeng Chen et al. 2019. Lingvo: a modular and scalable framework for sequence-to-sequence modeling. arXiv preprint arXiv:1902.08295

    2019 arXiv

  21. [29]

    cloze procedure

    Wilson L Taylor. 1953. “cloze procedure”: A new tool for measuring readability. Journalism Bulletin, 30(4):415--433

    1953

  22. [30]

    Shyam Upadhyay, Manaal Faruqui, Chris Dyer, and Dan Roth. 2016. Cross-lingual models of word embeddings: An empirical comparison. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 1661--1670

    2016

  23. [31]

    Jakob Uszkoreit, Jay M Ponte, Ashok C Popat, and Moshe Dubiner. 2010. Large scale parallel document mining for machine translation. In Proceedings of the 23rd International Conference on Computational Linguistics, pages 1101--1109. Association for Computational Linguistics

    2010

  24. [32]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, pages 5998--6008

    2017

  25. [33]

    Shijie Wu and Mark Dredze. 2019. Beto, bentz, becas: The surprising cross-lingual effectiveness of bert. arXiv preprint arXiv:1904.09077

    2019 arXiv

  26. [34]

    Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. 2016. Google's neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144

    2016 arXiv

  27. [35]

    Daniel Zeman, Jan Haji c , Martin Popel, Martin Potthast, Milan Straka, Filip Ginter, Joakim Nivre, and Slav Petrov. 2018. Conll 2018 shared task: Multilingual parsing from raw text to universal dependencies. In Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing f...

    2018

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.