Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

Extending LLMs to New Languages: A Case Study of Llama and Persian Adaptation

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Bilingual embedding alignment, not heavy pretraining, is what lets Llama classify new-language tasks, and it leaves English mostly intact.

desk verdict The staged PEFT recipe is useful, but Table 5 undermines the paper's central claim that bilingual alignment drives the Persian classification gains, and the 'no adverse impact' claim is contradicted by the paper's own generation results. read the letter →

arxiv 2412.13375 v2 pith:I77FVO55 submitted 2024-12-17 cs.CL

Samin Mahdizadeh Sani , Pouya Sadeghi , Thuy-Trang Vu , Yadollah Yaghoobzadeh , Gholamreza Haffari This is my paper · ORCID
classification cs.CL
keywords Persianlanguageadaptationparameter-efficientfine-tuningcross-lingualtransferembeddingalignmentinstructiontuninglow-resourcelanguagesclassificationtextgeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks what it takes to teach a largely English-only large language model a new, low-resource language using parameter-efficient fine-tuning, with Persian as the test case. It argues that embedding-level bilingual alignment is the decisive ingredient for classification: aligning Persian embeddings with English ones through parallel data is enough to lift Persian classification accuracy, without hurting English and sometimes improving it. For generation, additional monolingual Persian pretraining helps, but it costs some English summarization quality. The paper also argues that cross-lingual transfer from English to Persian without Persian instruction data is shallow, helping mostly simple tasks like sentiment analysis.

What carries the argument

The central mechanism is a staged parameter-efficient pipeline: first, vocabulary expansion by merging 20,000 SentencePiece Persian tokens into Llama-2's 32,000-token vocabulary; second, embedding alignment, where all transformer layers are frozen and only embeddings and the LM head are trained on monolingual Persian and parallel English-Persian next-token prediction; third, text-generation pretraining with low-rank adaptation (LoRA, rank 8 and alpha 32) on monolingual Persian; and fourth, instruction tuning on Persian and English instructions, including English-Persian translation instructions. The load-bearing step is the second one, bilingual embedding alignment, which the paper shows is sufficient for classification gains and preserves English ability.

What would settle it

Take the same training recipe and evaluate on Persian tasks whose instruction templates and instances come from a source completely disjoint from those used in instruction tuning; if the classification gains from bilingual alignment vanish, the gains are format familiarity rather than Persian understanding.

Watch

Extended reading notes

Core claim

The paper's central claim is that when extending a largely English-only LLM to Persian with parameter-efficient fine-tuning, the stage that matters most for classification is embedding-level bilingual alignment: training only the embedding and head layers on monolingual Persian plus English-Persian parallel text yields Persian classification accuracy equal to or better than further full-weight LoRA pretraining, with no English degradation and sometimes English gains. For generation tasks, deeper monolingual pretraining helps, but at a cost of English summarization performance. Cross-lingual transfer from English to Persian without Persian instructions is weak, succeeding mainly on simple sentiment classification and failing on textual entailment, and with very limited Persian instruction data the model's initial Persian knowledge outweighs alignment.

Load-bearing premise

The comparison assumes that the Persian classification scores reflect language ability, not the model's familiarity with instruction templates and examples that overlap between training and test sets.

Editorial extensions

If this is right

  • Classification adaptation to a new language can be achieved by aligning embeddings with bilingual parallel data; additional monolingual pretraining is not necessary for classification.
  • Adding a language through embedding alignment does not degrade English classification and sometimes improves it, though further Persian pretraining can lower English summarization quality.
  • When only limited instruction data is available, the model's initial target-language comprehension matters more than cross-lingual alignment, so base-model choice dominates.
  • Knowledge transfer from English to Persian without Persian instruction data is limited: it helps simple classification tasks like sentiment analysis but not complex tasks like textual entailment.
  • Translating Persian inputs to English improves classification accuracy for the adapted models but does not help summarization, suggesting generation is more sensitive to input language.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because all English evaluation tasks come from the same instruction-tuning source, the measured classification gains may partly reflect instruction-format familiarity rather than general Persian ability; a truly held-out Persian task format would test this.
  • The alignment benefit may scale with base-model quality: a stronger base model with more Persian tokens could show deeper transfer than Llama-2 demonstrates, so the shallow-transfer result may be specific to weak initial Persian support.
  • This bilingual-alignment recipe, applied with only embedding and head updates, may generalize to other low-resource languages with non-Latin scripts, where embedding alignment is the primary bottleneck.
  • In practical deployment, translating inputs to English is a cheaper alternative to target-language adaptation for classification, but not for generation, where the adapted model retains an advantage in summarization.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper studies how to extend Llama-2-7B to Persian under parameter-efficient constraints. It proposes a multi-stage pipeline: extending the tokenizer with 20,000 Persian subword tokens, aligning newly added embeddings via frozen-transformer next-token prediction on monolingual and parallel English-Persian data, continued pretraining with LoRA on Persian text, and bilingual instruction tuning. The authors evaluate four model variants (Llama-2, Llama-2-noLoRA, Em-aligned, Fa-pretrained) on Persian and English classification tasks (multiple-choice QA, sentiment, entailment) and generation tasks (summarization, translation), and additionally study knowledge transfer with English-only plus translation instructions, input translation, limited instruction data, and comparisons with multilingual and state-of-the-art models. The central claims are that bilingual embedding alignment improves Persian classification without adverse impact on English tasks, that cross-lingual alignment provides minimal benefit in low-resource settings, and that knowledge transfer from English to Persian is marginal and mostly helps simple classification tasks.

Significance. If the central claims were firmly established, the paper would provide practical guidance for adapting English-centric LLMs to lower-resource languages under PEFT constraints, and its systematic stage-by-stage evaluation would be a useful reference. The paper has genuine strengths: it releases code, covers a non-Latin low-resource language, evaluates both classification and generation, includes unseen-task generalization checks, and compares against several recent open and closed models. However, the headline attribution of classification gains to bilingual alignment is directly undermined by the paper's own Table 5, and the scoping of the 'no adverse impact' claim is internally inconsistent with the reported English summarization drops and with the conclusion section. These issues are load-bearing rather than cosmetic, so the central claims need correction or additional experimental support before the paper can be accepted.

major comments (4)
  1. [§5.2, Table 5] The abstract's first claim, that bilingual data alignment enhances Persian classification accuracy, is not supported by Table 5. For Persian tasks, average classification accuracy is 0.56 for Llama-2 (no vocabulary expansion, no alignment), 0.56 for Em-aligned, and 0.56 for Fa-pretrained; the improvement over random (0.36) is shared by the non-aligned instruction-tuned baseline. This indicates that instruction tuning with Persian instructions, rather than the alignment stages, drives the reported gain. The paper needs either an explicit control that isolates the alignment stages while holding instruction tuning fixed, or a reframed claim that attributes the classification improvement to instruction tuning with Persian data.
  2. [§5.2, §7, Table 5] The claim of 'no adverse impact and sometimes even improvements on English tasks' is contradicted by the paper's own numbers: English summarization drops from 0.35 (Llama-2) to 0.32 (Em-aligned) and 0.23 (Fa-pretrained). Section 7 explicitly concedes that 'this alignment negatively impacts the model's English text generation.' The abstract and Section 1 should therefore scope the 'no adverse impact' claim to classification tasks, or the conflict with Section 7 should be resolved.
  3. [§3.2, §5.1, §6] The evaluation setup has a train/test overlap confound that affects both the classification-gain claim and the limited-transfer claim. English evaluation tasks are selected from SuperNatural-Instructions, the same source used for instruction tuning, and Persian multiple-choice, entailment, and translation evaluations also come from SuperNatural-Instructions. Section 6 acknowledges that observed improvements 'may be influenced by similarities between the training and test sets, such as instruction formats.' The authors should evaluate on held-out sources or explicitly verify that the specific test instances and templates were not present in the instruction-tuning data.
  4. [Tables 5–9] All comparative claims rest on single-run point estimates without error bars, confidence intervals, or significance testing. Many differences are within one or two points of accuracy (e.g., Persian averaged classification 0.56 vs. 0.56, English averaged classification 0.52 vs. 0.56), and generation scores are reported without variance. Without repeated runs or statistical testing, the claims of 'sometimes even improvements' and 'minimal benefits' are not robust; the paper should report variance or at least multiple seeds for the central comparisons.
minor comments (6)
  1. [Table 1] The header 'Perian sourceType' appears to be a typo and should read 'Persian sourceType'; also consider aligning the table columns for readability.
  2. [§4.2.1] The phrase 'only the heads and embeddings are trained' is ambiguous; please specify whether 'heads' means the LM head, attention heads, or both, and clarify what is frozen in each stage.
  3. [§4.3, Table 4] Table 4 reports the same parameter counts for pretraining and instruction-tuning (6.20%), but Section 4.2.2 says LoRA weights are updated during pretraining and later during instruction tuning; a sentence clarifying which components are trainable in each stage would prevent confusion.
  4. [§5.1] The description of the Llama-2-noLoRA model says it has 'limited Persian tokens' but the model description for Llama-2 says the same; since noLoRA has the original 32k vocabulary, please clarify whether Llama-2-noLoRA uses the expanded vocabulary or not.
  5. [Figure 2] The text refers to 'the orange box' in Figure 2, but the figure is not in color in this format; consider describing the location concretely (e.g., 'the highlighted box').
  6. [§7] The conclusion says alignment 'negatively impacts the model's English text generation,' which contradicts the abstract's 'no adverse impact' phrasing; align the wording across abstract, introduction, and conclusion after the major revisions are made.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the paper's claims are direct empirical comparisons with no fitted-parameter or self-citation chain that forces the conclusions.

full rationale

The paper contains no mathematical derivation, no fitted constants, and no uniqueness theorem. Its claims are empirical measurements of models at different training stages (Tables 5-9). The only self-citation, Abaskohi et al. (2024), is used in the introduction as background motivation ('Persian has not fully benefited... largely due to limited annotated data and research efforts') and is not load-bearing for any result. No claim reduces to its inputs by construction: the reported classification gains are not obtained by fitting a parameter and then predicting the same quantity, and the aligned model is not defined in terms of the evaluation metric. The relevant validity caveat is benchmark overlap: Section 5.1 states that for English, 'all tasks are selected from the SuperNatural-Instructions dataset,' the same corpus used for instruction tuning, and Section 6 concedes that improvements 'may be influenced by similarities between the training and test sets, such as instruction formats.' This is a contamination/attribution risk, not circular reasoning; indeed, Table 5 shows the same Persian classification average (0.56) for Llama-2, Em-aligned, and Fa-pretrained, which undercuts rather than supports the alignment attribution, but that is a correctness issue outside the circularity definition. Therefore no circular step is exhibited.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

No new physical or formal entities are introduced. The central claim rests on empirical assumptions about representational alignment, evaluation validity, and the base model's limitations, plus a set of hand-chosen hyperparameters. The most fragile entry is the independence of evaluation from instruction-tuning data.

free parameters (4)
  • LoRA rank r = 8
    Chosen in Section 4.3 to limit memory usage on eight V100 GPUs; the authors note a higher rank could lead to stronger models.
  • LoRA alpha = 32
    Set alongside rank in Section 4.3 with weight-update coefficient alpha/r = 4; hand-chosen, not tuned.
  • New Persian vocabulary size = 20,000
    Number of SentencePiece tokens added in Section 4.1; approximately 10% overlapped with existing tokens, resulting in 49,816 total tokens.
  • Persian-probability threshold in preprocessing = 70%
    Section 3.1 keeps sentences with at least 70% probability of being Persian; chosen deliberately to retain some English text in the monolingual corpus.
assumptions (4)
  • domain assumption Llama-2 has limited understanding of Persian and therefore is an appropriate base model for adaptation.
    Stated in the Introduction and Section 4; the whole study assumes the base model needs Persian adaptation.
  • domain assumption Bilingual next-token prediction on parallel English-Persian data aligns newly added Persian embeddings with English representations.
    Invoked in Section 4.2.1 as the mechanism for embedding alignment; no independent evidence is given that alignment actually occurs.
  • domain assumption Accuracy and BLEU on the selected downstream tasks are valid measures of language adaptation and knowledge transfer.
    Section 5.1 uses these metrics without analyzing their reliability on small or noisy Persian test sets.
  • domain assumption Evaluation tasks drawn from SuperNatural-Instructions are independent enough of the instruction-tuning data to measure generalization.
    Section 5.1 selects English tasks from SuperNatural-Instructions, which is also used for instruction tuning; Section 6 later concedes possible training/test similarity.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Extending LLMs to New Languages: A Case Study of Llama and Persian Adaptation." pith.science (2026). https://pith.science/paper/I77FVO55

@misc{pith2026241213375,
  author       = {Pith},
  title        = {Pith review of: Extending LLMs to New Languages: A Case Study of Llama and Persian Adaptation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/I77FVO55}},
  note         = {Machine review of arXiv:2412.13375}
}
read the original abstract

Large language models (LLMs) have made great progress in classification and text generation tasks. However, they are mainly trained on English data and often struggle with low-resource languages. In this study, we explore adding a new language, i.e., Persian, to Llama (a model with a limited understanding of Persian) using parameter-efficient fine-tuning. We employ a multi-stage approach involving pretraining on monolingual Persian data, aligning representations through bilingual pretraining and instruction datasets, and instruction-tuning with task-specific datasets. We evaluate the model's performance at each stage on generation and classification tasks. Our findings suggest that incorporating the Persian language, through bilingual data alignment, can enhance classification accuracy for Persian tasks, with no adverse impact and sometimes even improvements on English tasks. Additionally, the results highlight the model's initial strength as a critical factor when working with limited training data, with cross-lingual alignment offering minimal benefits for the low-resource language. Knowledge transfer from English to Persian has a marginal effect, primarily benefiting simple classification tasks.

Figures

Figures reproduced from arXiv: 2412.13375 by the authors.

Figure 1
Figure 1. Statistics of datasets used for pre-training and instruction-tuning. [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Generated text before and after LoRA fine [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. An example of the model conversation. The results are the best generations from multiple attempts. [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Model lack of Consistency [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]
Figure 5
Figure 5. Figure 5: Model Hallucination [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: An example of model repetition [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: An example of model conversation [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Model lack of Consistency [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 9
Figure 9. Figure 9: Model Hallucination (The answer about the birthplace of Molana is incorrect.) [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: An example of model repetition [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Prompt, Translate, Fine-Tune, Re-Initialize, or Instruction-Tune? Adapting LLMs for In-Context Learning in Low-Resource Languages

    cs.CL 2025-06

Reference graph

Works this paper leans on

49 extracted references · 8 canonical work pages · cited by 1 Pith paper

  1. [1]

    Amirhossein Abaskohi, Sara Baruni, Mostafa Masoudi, Nesa Abbasi, Mohammad Hadi Babalou, Ali Edalat, Sepehr Kamahi, Samin Mahdizadeh Sani, Nikoo Naghavian, Danial Namazifard, Pouya Sadeghi, and Yadollah Yaghoobzadeh. 2024. https://arxiv.org/abs/2404.02403 Benchmarking large language models for persian: A preliminary study focusing on chatgpt . Preprint, ar...

    arXiv 2024

  2. [2]

    Mohammad Amin Abbasi, Arash Ghafouri, Mahdi Firouzmandi, Hassan Naderi, and Behrouz Minaei Bidgoli. 2023. Persianllama: Towards building first persian large language model. arXiv preprint arXiv:2312.15713

    arXiv 2023

  3. [3]

    Hadi Abdi Khojasteh, Ebrahim Ansari, and Mahdi Bohlouli. 2020. https://aclanthology.org/2020.lrec-1.776 LSCP : Enhanced large scale colloquial P ersian language understanding . In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 6323--6327, Marseille, France. European Language Resources Association

    2020

  4. [4]

    Anton Alexandrov, Veselin Raychev, Mark Niklas Müller, Ce Zhang, Martin Vechev, and Kristina Toutanova. 2024. https://arxiv.org/abs/2407.08699 Mitigating catastrophic forgetting in language transfer via model merging . Preprint, arXiv:2407.08699

    arXiv 2024

  5. [5]

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, Peng Wang, Shijie Wang, Wei Wang, Shengguang Wu, Benfeng X...

    arXiv 2023

  6. [6]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gr...

    2020

  7. [7]

    Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzm \'a n, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. https://doi.org/10.18653/v1/2020.acl-main.747 Unsupervised cross-lingual representation learning at scale . In Proceedings of the 58th Annual Meeting of the Association for Comp...

    doi:10.18653/v1/2020.acl-main.747 2020

  8. [8]

    Mike Conover, Matt Hayes, Ankit Mathur, Xiangrui Meng, Jianwei Xie, Jun Wan, Ali Ghodsi, Patrick Wendell, and Matei Zaharia. 2023. https://www.databricks.com/blog/2023/03/24/hello-dolly-democratizing-magic-chatgpt-open-models.html Hello dolly: Democratizing the magic of chatgpt with open models

    work page 2023

Show all 49 references
  1. [9]

    Zoltan Csaki, Pian Pawakapan, Urmish Thakker, and Qiantong Xu. 2023. https://arxiv.org/abs/2311.05741 Efficiently adapting pretrained language models to new languages . Preprint, arXiv:2311.05741

    2023 arXiv

  2. [10]

    Yiming Cui, Ziqing Yang, and Xin Yao. 2023. Efficient and effective text encoding for chinese llama and alpaca. arXiv preprint arXiv:2304.08177

    2023 arXiv

  3. [11]

    Raj Dabre, Himani Shrotriya, Anoop Kunchukuttan, Ratish Puduppully, Mitesh Khapra, and Pratyush Kumar. 2022. https://doi.org/10.18653/v1/2022.findings-acl.145 I ndic BART : A pre-trained model for indic natural language generation . In Findings of the Association for Computati...

    2022 doi

  4. [12]

    Mehrdad Farahani, Mohammad Gharachorloo, and Mohammad Manthouri. 2021. Leveraging parsbert and pretrained mt5 for persian abstractive text summarization. In 2021 26th International computer conference, computer society of Iran (CSICC), pages 1--6. IEEE

    2021

  5. [13]

    Gurpreet Gosal, Yishi Xu, Gokul Ramakrishnan, Rituraj Joshi, Avraham Sheinin, Zhiming, Chen, Biswajit Mishra, Natalia Vassilieva, Joel Hestness, Neha Sengupta, Sunil Kumar Sahu, Bokang Jia, Onkar Pandit, Satheesh Katipomu, Samta Kamboj, Samujjwal Ghosh, Rahul Pal, Parvez Mulla...

    2024 arXiv

  6. [14]

    Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. https://openreview.net/forum?id=nZeVKeeFYf9 Lo RA : Low-rank adaptation of large language models . In International Conference on Learning Representations

    2022

  7. [15]

    Yunjie Ji, Yan Gong, Yong Deng, Yiping Peng, Qiang Niu, Baochang Ma, and Xiangang Li. 2023. Towards better instruction following language models for chinese: Investigating the impact of training data and evaluation. arXiv preprint arXiv:2304.07854

    2023 arXiv

  8. [16]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas ...

    2023 arXiv

  9. [17]

    Akbar Karimi, Ebrahim Ansari, and Bahram Sadeghi Bigham. 2018. https://aclanthology.org/L18-1549 Extracting an E nglish- P ersian parallel corpus from comparable corpora . In Proceedings of the Eleventh International Conference on Language Resources and Evaluation ( LREC 2018)...

    2018

  10. [18]

    Omid Kashefi. 2018. https://api.semanticscholar.org/CorpusID:141068289 MizĀn : A large persian-english parallel corpus omid kashefi

    2018

  11. [19]

    o pf, Yannic Kilcher, Dimitri von R \

    Andreas K \"o pf, Yannic Kilcher, Dimitri von R \"u tte, Sotiris Anagnostidis, Zhi Rui Tam, Keith Stevens, Abdullah Barhoum, Duc Nguyen, Oliver Stanley, Rich \'a rd Nagyfi, et al. 2024. Openassistant conversations-democratizing large language model alignment. Advances in Neura...

    2024

  12. [20]

    Taku Kudo and John Richardson. 2018. https://doi.org/10.18653/v1/D18-2012 S entence P iece: A simple and language independent subword tokenizer and detokenizer for neural text processing . In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processin...

    2018 doi

  13. [21]

    Xianhang Li, Haoqin Tu, Mude Hui, Zeyu Wang, Bingchen Zhao, Junfei Xiao, Sucheng Ren, Jieru Mei, Qing Liu, Huangjie Zheng, Yuyin Zhou, and Cihang Xie. 2024. https://arxiv.org/abs/2406.08478 What if we recaption billions of web images with llama-3? Preprint, arXiv:2406.08478

    2024 arXiv

  14. [22]

    Ibraheem Muhammad Moosa, Mahmud Elahi Akhter, and Ashfia Binte Habib. 2023. https://doi.org/10.18653/v1/2023.findings-eacl.50 Does transliteration help multilingual language modeling? In Findings of the Association for Computational Linguistics: EACL 2023, pages 670--685, Dubr...

    2023 doi

  15. [23]

    Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng Xin Yong, Hailey Schoelkopf, Xiangru Tang, Dragomir Radev, Alham Fikri Aji, Khalid Almubarak, Samuel Albanie, Zaid Alyafeai, Albert Webson, Edward ...

    2023 doi

  16. [24]

    Benjamin Muller, Antonios Anastasopoulos, Beno \^ t Sagot, and Djam \'e Seddah. 2021. https://doi.org/10.18653/v1/2021.naacl-main.38 When being unseen from m BERT is just the beginning: Handling new languages with multilingual language models . In Proceedings of the 2021 Confe...

    2021 doi

  17. [25]

    OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff ...

    2024 arXiv

  18. [26]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311--318

    2002

  19. [27]

    Mohammad Taher Pilehvar, Heshaam Faili, and Abdol Hamid Pilehvar. 2011. https://api.semanticscholar.org/CorpusID:14612068 Tep: Tehran english-persian parallel corpus . In Conference on Intelligent Text Processing and Computational Linguistics

    2011

  20. [28]

    Telmo Pires, Eva Schlinger, and Dan Garrette. 2019. https://doi.org/10.18653/v1/P19-1493 How multilingual is multilingual BERT ? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4996--5001, Florence, Italy. Association for Compu...

    2019 doi

  21. [29]

    Sukannya Purkayastha, Sebastian Ruder, Jonas Pfeiffer, Iryna Gurevych, and Ivan Vuli \'c . 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.538 R omanization-based large-scale adaptation of multilingual language models . In Findings of the Association for Computational Li...

    2023 doi

  22. [30]

    Jirui Qi, Raquel Fern \'a ndez, and Arianna Bisazza. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.658 Cross-lingual consistency of factual knowledge in multilingual language models . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing...

    2023 doi

  23. [31]

    Libo Qin, Qiguang Chen, Yuhang Zhou, Zhi Chen, Yinghui Li, Lizi Liao, Min Li, Wanxiang Che, and Philip S. Yu. 2024. https://arxiv.org/abs/2404.04925 Multilingual large language model: A survey of resources, taxonomy and frontiers . Preprint, arXiv:2404.04925

    2024 arXiv

  24. [32]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1--67

    2020

  25. [33]

    Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. 2020. Zero: Memory optimizations toward training trillion parameter models. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, pages 1--16. IEEE

    2020

  26. [34]

    Leonardo Ranaldi and Giulia Pucci. 2023. https://doi.org/10.18653/v1/2023.mrl-1.14 Does the E nglish matter? elicit cross-lingual abilities of large language models . In Proceedings of the 3rd Workshop on Multi-lingual Representation Learning (MRL), pages 173--183, Singapore. ...

    2023 doi

  27. [35]

    Leonardo Ranaldi, Giulia Pucci, and Andre Freitas. 2023. Empowering cross-lingual abilities of instruction-tuned large language models by translation-following demonstrations. arXiv preprint arXiv:2308.14186

    2023 arXiv

  28. [36]

    Pedram Rostami, Ali Salemi, and Mohammad Javad Dousti. 2024. Persianmind: A cross-lingual persian-english large language model. arXiv preprint arXiv:2401.06466

    2024 arXiv

  29. [37]

    Mortazavi Najafabadi, and Amir Vaheb

    Behnam Sabeti, Hossein Abedi Firouzjaee, Ali Janalizadeh Choobbasti, S.H.E. Mortazavi Najafabadi, and Amir Vaheb. 2018. https://aclanthology.org/L18-1188 M iras T ext: An automatically generated text corpus for P ersian . In Proceedings of the Eleventh International Conference...

    2018

  30. [38]

    Hashimoto

    Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford alpaca: An instruction-following llama model. https://github.com/tatsu-lab/stanford_alpaca

    2023

  31. [39]

    Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. 2024 a . Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530

    2024 arXiv

  32. [40]

    Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, L \'e onard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ram \'e , et al. 2024 b . Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118

    2024 arXiv

  33. [41]

    Atula Tejaswi, Nilesh Gupta, and Eunsol Choi. 2024. Exploring design choices for building language-specific llms. arXiv preprint arXiv:2406.14670

    2024 arXiv

  34. [42]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, W...

    2023 arXiv

  35. [43]

    Bibek Upadhayay and Vahid Behzadan. 2023. Taco: Enhancing cross-lingual transfer for low-resource languages in llms through translation-assisted chain-of-thought processes. arXiv preprint arXiv:2311.10797

    2023 arXiv

  36. [44]

    Tu Vu, Aditya Barua, Brian Lester, Daniel Cer, Mohit Iyyer, and Noah Constant. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.630 Overcoming catastrophic forgetting in zero-shot cross-lingual generation . In Proceedings of the 2022 Conference on Empirical Methods in Natural...

    2022 doi

  37. [45]

    Yizhong Wang, Hamish Ivison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Chandu, David Wadden, Kelsey MacMillan, Noah A Smith, Iz Beltagy, et al. 2023. How far can camels go? exploring the state of instruction tuning on open resources. Advances in Neural Information Proc...

    2023

  38. [46]

    Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, et al. 2022. Super-naturalinstructions:generalization via declarative instructions on 1600+ tasks. In EMNLP

    2022

  39. [47]

    Wenhao Zhu, Yunzhe Lv, Qingxiu Dong, Fei Yuan, Jingjing Xu, Shujian Huang, Lingpeng Kong, Jiajun Chen, and Lei Li. 2023. Extrapolating large language models to non-english by aligning languages. arXiv preprint arXiv:2308.04948

    2023 arXiv

  40. [48]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  41. [49]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.