Pith. sign in

REVIEW 4 cited by

A Primer on Pretrained Multilingual Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2107.00676 v2 pith:ZCV36EI5 submitted 2021-07-01 cs.CL

Sumanth Doddapaneni , Gowtham Ramesh , Mitesh M. Khapra , Anoop Kunchukuttan , Pratyush Kumar This is my paper · ORCID
classification cs.CL
keywords mllmslanguagescoveringlanguagelargeemergedmodelsmultilingual
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Multilingual Language Models (\MLLMs) such as mBERT, XLM, XLM-R, \textit{etc.} have emerged as a viable option for bringing the power of pretraining to a large number of languages. Given their success in zero-shot transfer learning, there has emerged a large body of work in (i) building bigger \MLLMs~covering a large number of languages (ii) creating exhaustive benchmarks covering a wider variety of tasks and languages for evaluating \MLLMs~ (iii) analysing the performance of \MLLMs~on monolingual, zero-shot cross-lingual and bilingual tasks (iv) understanding the universal language patterns (if any) learnt by \MLLMs~ and (v) augmenting the (often) limited capacity of \MLLMs~ to improve their performance on seen or even unseen languages. In this survey, we review the existing literature covering the above broad areas of research pertaining to \MLLMs. Based on our survey, we recommend some promising directions of future research.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. How Programming Concepts and Neurons Are Shared in Code Language Models

    cs.CL 2025-06 conditional novelty 6.0 of 10

    In Llama-based code models, programming languages are represented through an English-like intermediate token space, with language-specific neurons concentrated in bottom layers and exclusive PL neurons in top layers; ...

  2. Can MLLMs Generalize to Multi-Party dialog? Exploring Multilingual Response Generation in Complex Scenarios

    cs.CL 2025-01 reject novelty 5.0 of 10

    On the new XMP parallel multi-party dialogue dataset, current multilingual LLMs generate low-quality responses; adding more or mixed-language training data yields only marginal, often negative, cross-lingual transfer.

  3. Breaking Physical and Linguistic Borders: Multilingual Federated Prompt Tuning for Low-Resource Languages

    cs.CL 2025-07 conditional novelty 4.0 of 10

    Federated averaging of prompt embeddings from a frozen multilingual model improves accuracy on some low-resource tasks (XNLI) but not consistently on others (MasakhaNEWS).

  4. Duluth at SemEval-2025 Task 7: TF-IDF with Optimized Vector Dimensions for Multilingual Fact-Checked Claim Retrieval

    cs.CL 2025-05 conditional novelty 2.0 of 10

    Word-level TF-IDF with a 15,000-term vocabulary reaches 0.69 average success@10 on the SemEval-2025 Task 7 multilingual test set, far behind the top system's 0.96.

Pith tools