Pith. sign in

REVIEW 3 major objections 7 minor 1 cited by

Arabic Stable LM: Adapting Stable LM 2 1.6B to Arabic

T0 review · 3 major / 7 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A 1.6-billion-parameter Arabic model outperforms Arabic LLMs up to eight times its size on cloze-format benchmarks.

desk verdict Solid empirical contribution with a real gap: no contamination check, so the 'beats 8x larger models' headline is unsecured even though the models are released and the recipe is credible. read the letter →

arxiv 2412.04277 v1 pith:232WRDJN submitted 2024-12-05 cs.CL

Zaid Alyafeai , Michael Pieler , Hannah Teufel , Jonathan Tow , Marco Bellagente , Duy Phung , Nikhil Pinnaparaju , Reshinth Adithyan
show 3 more authors
Paulo Rocha Maksym Zhuravinskyi Carlos Riquelme
This is my paper · ORCID
classification cs.CL
keywords ArabicLLMlow-resourceNLPcontinuedpre-traininginstructiontuningsyntheticdataclozeformatevaluationmodelefficiencyMMLU
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Arabic Stable LM 1.6B is an attempt to answer a concrete question: can a small Arabic-focused language model compete with much larger ones? The authors fine-tune the 1.6B Stable LM 2 model on roughly 114 billion Arabic tokens, then instruction-tune it with a mix of existing datasets and 183k synthetic question-answer dialogues generated from its own training documents. Their central claim is that the resulting chat model, ar-stablelm-2-chat, beats every model they compare against on ArabicMMLU in cloze format, including 13B models, and sets the best average on AlGhafa. If correct, this matters because it suggests strong Arabic NLP performance does not require 7-13B parameters, cutting hardware cost and inference latency.

What carries the argument

The load-bearing mechanism is continued pre-training plus synthetic instruction tuning. Starting from Stable LM 2 1.6B, the authors train for 500k steps on a mixture of 619B English and 115B Arabic tokens (sampled so Arabic gets 82% of steps), with a warm-up, cosine/inverse-square-root schedule, and an early linear cool-down that they show improves ArabicMMLU. The Arabic cleaning pipeline keeps 114B of 158B raw tokens. For the chat model they add 677,746 instruction samples, of which 182,505 come from using Qwen2-7B-Instruct to rephrase training documents into dialogues and multiple-choice tests; the MCQ portion alone raises the average score by 1.5 points. Evaluation uses the cloze format with normalized accuracy, which the paper argues is more robust than letter-based multiple choice.

What would settle it

Compute exact or near-duplicate n-gram overlap between the test questions in ArabicMMLU, AlGhafa, ACVA, and CIDAR and the 114B cleaned training tokens; if a substantial fraction of questions (or their answer strings) appear in the crawl, the benchmark superiority could come from memorization rather than general capability. A cleaner test would re-run the comparisons on a newly written Arabic benchmark released after training.

Watch

Extended reading notes

Core claim

The paper's central discovery is that a 1.64B chat model, ar-stablelm-2-chat, achieves the highest average accuracy among all compared models on ArabicMMLU (cloze format) at 45.5%, ahead of the best 13B model (jais-family-13b) at 41.9%, and scores 50.1% on AlGhafa versus 45.1% for AceGPT-7B-chat. The base model also finishes near or above most larger models. The authors attribute the gains to continued pre-training on 114B cleaned Arabic tokens from CulturaX, SANAD news, and an Arabic e-book corpus, a learning-rate cool-down schedule, and instruction tuning that includes synthetic dialogues with multiple-choice questions. They also report that the cloze format gives more reliable and more favorable measurements for their models than the multiple-choice-letter format.

Load-bearing premise

The paper performs no contamination analysis, so the central claim rests on the assumption that ArabicMMLU, AlGhafa, ACVA, and CIDAR test items were not present in the crawled training data.

Editorial extensions

If this is right

  • If the result holds, Arabic NLP can be served with a 1.6B model, lowering GPU memory and inference latency relative to 7-13B Arabic models.
  • The cloze-format evaluation protocol, with normalized accuracy, becomes the recommended way to compare Arabic MCQ models, since letter-based formats penalize models that do not follow the letter-answer convention.
  • Synthetic instruction data generated by rephrasing the model's own training documents is a cheap way to build large Arabic chat-training sets.
  • The fine-tuning recipe (continued pre-training, cool-down schedule, MCQ rephrasing) transfers to other base models and possibly to larger Stable LM versions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next experiment, not run here, is tokenizer transfer: replacing the high-fertility Stable LM tokenizer with an Arabic-trained one would likely cut per-token cost and raise inference speed.
  • Because the MCQ-format gap is large for several baselines, leaderboard rankings for Arabic models may depend heavily on evaluation format; this suggests future Arabic benchmarks should report both CF and MCF.
  • The rephrasing pipeline, which uses Qwen2-7B-Instruct to generate dialogues from arbitrary documents, should transfer to other low-resource languages with small adaptation cost.
  • The absence of a decontamination study means the headline numbers should be treated as upper bounds until overlap checks are published.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper presents Arabic Stable LM 1.6B, a base and chat model obtained by continuing pretraining of Stable LM 2 1.6B on 114B Arabic tokens (CulturaX, SANAD, e-books) mixed with English, followed by instruction tuning on a synthetic rephrased dialogue dataset, Instar-500k, and Aya. The authors evaluate on ArabicMMLU in cloze format (CF), CIDAR-MCQ-100, ACVA, and AlGhafa, and report that the chat model achieves the highest average scores, outperforming models with up to 13B parameters on ArabicMMLU and AlGhafa.

Significance. If the reported results are contamination-free and reproducible, the paper would make a useful contribution: it demonstrates that a 1.6B model can be competitive with 7-13B Arabic-centric LLMs after continued pretraining on a moderately sized Arabic corpus, and that synthetic MCQ-style instruction data provides a measurable boost. The release of the models and the detailed ablations (learning-rate cool-down, synthetic data) are valuable. However, the central superiority claim is currently unsecured due to absent decontamination analysis, under-specified evaluation prompts, and lack of statistical uncertainty quantification.

major comments (3)
  1. [§3.2, §4, §5] No contamination analysis is reported. The pretraining corpus includes 114B tokens of web-crawled text from CulturaX (Table 3), and the instruction-tuning dataset is generated by rephrasing documents from that same corpus (§4, Appendix B). All four evaluation benchmarks are public datasets (Table 12). Because web crawls are known to contain benchmark questions, and because the MCQ rephrasing pipeline converts document text into multiple-choice items, a leaked benchmark item could appear in both pretraining and instruction tuning. Table 8 shows that adding MCQ synthetic data improves the average by 1.4 points, which indicates that the evaluation is sensitive to exactly the kind of data that would be contaminated. The paper must provide an n-gram overlap analysis (e.g., 8-13 gram hits) between the training corpora and the evaluation sets, remove any overlapping items, and re-report Tables 5 and 6 before the 'beats up to 8x' claim can be interpreted as a capability result.
  2. [§5] The evaluation protocol is under-specified. The text states that ACVA uses 5-shot and F1 macro, but for ArabicMMLU, AlGhafa, and CIDAR-MCQ-100 no few-shot count or prompt template is given. This matters because cloze-format accuracy can vary substantially with prompt wording and example choices. In addition, CIDAR-MCQ-100 contains only 100 items, and no confidence intervals or significance tests are reported anywhere; differences of 1-2 percentage points on that set (e.g., 46.0 vs 45.0 in Table 6) are within sampling noise. The authors should specify all evaluation prompts and shot counts, report variance (e.g., bootstrap CIs) or repeated runs, and avoid claiming superiority over baselines whose margins are not statistically distinguishable.
  3. [§6, Appendix A] The headline claim is format-dependent, but the paper does not give this caveat enough prominence. In the cloze format, ar-stablelm-2-chat beats larger models (Tables 5-6), but in the multiple-choice format it is the second-best small model, not a leader (Figure 6). The authors justify CF as more robust (Appendix A), but because most prior Arabic benchmark reports use MCF, the comparison to the literature is oblique. The claim 'beating multiple models with up to 8x the parameters' in the abstract should be explicitly qualified as 'in cloze format on ArabicMMLU and AlGhafa' and ideally accompanied by MCF results for all evaluated models, so readers can judge whether the advantage is specific to the answer format.
minor comments (7)
  1. [§3.2] 'around 74K documents' contradicts Table 3, which lists 74.0M documents for CulturaX before cleaning.
  2. [§3.3] 'Gropher' should be 'Gopher'.
  3. [Table 11] The column headers list 't5' twice; the sequence should probably be t1-t9.
  4. [§6] 'a 4 % higher accuracy on average' is a 3.6 percentage-point difference from Table 5; please rephrase.
  5. [Table 8] The caption says 'with and without MCQ synthetic data,' but the 'w/o MCQ' model still includes standard (non-MCQ) rephrased data; please clarify.
  6. [Appendix B] Figures 7 and 8 are reproduced at a size that makes the Arabic text nearly illegible; please enlarge or provide text versions.
  7. [References] The Koto et al. entry has stray quotation marks from the source file.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation; central benchmark results are external measurements, with only minor non-load-bearing self-citations.

full rationale

The paper is an empirical fine-tuning study; there is no mathematical derivation whose outputs equal its inputs. The central claims (45.5% ArabicMMLU CF vs 41.9% for jais-family-13b; 50.1% AlGhafa vs 45.1% for AceGPT-7B-chat) are measurements on external benchmarks, not quantities fit from the model or from the rephrased-data pipeline. Self-citations to Stable LM 2 1.6B (Bellagente et al. 2024), the rephrasing recipe (Pieler et al. 2024), and co-authorship of ArabicMMLU/CIDAR do not substitute for a derivation: the base model is an external checkpoint, the rephrasing procedure is described in Appendix B, and the central benchmarks are public datasets. The only notable limitation is the absence of a contamination audit between the CulturaX web crawl used for pretraining/rephrasing and the public evaluation benchmarks; this threatens the validity of the superiority claim but is not circularity, because no benchmark item or score is used as a training signal or as a fitted parameter. Accordingly no circular step can be exhibited, and the score is limited to 2 for the minor, non-load-bearing self-citations.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claims are empirical results from training and evaluation, so there are no derived parameters. The free parameters listed are hand-chosen hyperparameters and filtering thresholds that shape the model and its scores. The axioms capture the evaluation assumptions and reliance on an external teacher model for synthetic data. No new entities are introduced.

free parameters (4)
  • English-to-Arabic sampling ratio = 18% English / 82% Arabic (tokens) with 4.6:1 upsampling of Arabic
    Chosen in Section 3.2 to compensate for the Arabic tokenizer's high fertility; directly shapes the learned mix and is a hand-set hyperparameter.
  • Learning rate schedule = max 5e-4, min 2.5e-6, cosine + inverse sqrt, then linear cool-down
    Set in Section 3.4 following the original Stable LM 2 report; the early vs late cool-down choice affects final benchmark scores (Figure 4).
  • Instruction-tuning epochs = 3 epochs over 677,746 samples
    Reported in Section 4; chosen by hand with no ablation on epoch count.
  • Rephrasing filtering thresholds = 183k conversations kept from 285k generated
    Section 4 and Appendix B; hand-designed format filters determine the synthetic instruction dataset size and content.
assumptions (4)
  • domain assumption The four benchmarks (ArabicMMLU, CIDAR-MCQ-100, ACVA, AlGhafa) are valid indicators of Arabic language understanding and cultural alignment.
    The evaluation section assumes these datasets measure the claimed capabilities; this is not derived.
  • domain assumption Cloze format is superior to multiple-choice format for comparing models.
    Section 5 and Appendix A argue this from prior literature (Alzahrani et al., Madaan et al., Wang et al.), but the choice remains a modeling assumption that favors the authors' model.
  • domain assumption Baseline models are evaluated under identical conditions (same prompts, normalization, few-shot counts).
    The paper does not state explicitly how baseline numbers were produced; if baseline scores were taken from prior papers with different setups, the comparison is invalid. This assumption is load-bearing.
  • ad hoc to paper The rephrased synthetic dialogues generated by Qwen2-7B-Instruct are high-quality and do not introduce systematic errors.
    Section 4 relies on Qwen2-7B-Instruct for dialogue generation; only simple format filters are applied, and no human quality evaluation is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Arabic Stable LM: Adapting Stable LM 2 1.6B to Arabic." pith.science (2026). https://pith.science/paper/232WRDJN

@misc{pith2026241204277,
  author       = {Pith},
  title        = {Pith review of: Arabic Stable LM: Adapting Stable LM 2 1.6B to Arabic},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/232WRDJN}},
  note         = {Machine review of arXiv:2412.04277}
}
read the original abstract

Large Language Models (LLMs) have shown impressive results in multiple domains of natural language processing (NLP) but are mainly focused on the English language. Recently, more LLMs have incorporated a larger proportion of multilingual text to represent low-resource languages. In Arabic NLP, several Arabic-centric LLMs have shown remarkable results on multiple benchmarks in the past two years. However, most Arabic LLMs have more than 7 billion parameters, which increases their hardware requirements and inference latency, when compared to smaller LLMs. This paper introduces Arabic Stable LM 1.6B in a base and chat version as a small but powerful Arabic-centric LLM. Our Arabic Stable LM 1.6B chat model achieves impressive results on several benchmarks beating multiple models with up to 8x the parameters. In addition, we show the benefit of mixing in synthetic instruction tuning data by augmenting our fine-tuning data with a large synthetic dialogue dataset.

Figures

Figures reproduced from arXiv: 2412.04277 by the authors.

Figure 1
Figure 1. Fertility scores of multiple tokenizers on the [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Different learning rate schedulers. The late [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. ArabicMMLU benchmark differences between the cloze format (CF) and the multiple-choice format [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: ArabicMMLU results for the early and late [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: ArabicMMLU benchmark results of the Ara [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: ArabicMMLU benchmark results of our Arabic Stable LM 1.6B base and chat models [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: An example dialogue created by using the standard rephrasing template. [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: An example dialogue created by using the multiple-choice question (MCQ) rephrasing template. [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Histogram of the rephrased multi-turn dialogue data for the standard and MCQ templates. The MCQ [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Histogram of the enumeration symbols for each type of multiple choice question (MCQ) data. [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]
Figure 11
Figure 11. Figure 11: Example conversation with the Arabic Stable LM chat model about a seasonal Arabic holiday. [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Multimodal Large Language Models for Image, Text, and Speech Data Augmentation: A Survey

    cs.CV 2025-01 conditional novelty 4.0 of 10

    A literature review cataloging LLM-based augmentation methods across image, text, and speech, with a taxonomy of techniques, limitations, and suggested fixes.

Reference graph

Works this paper leans on

48 extracted references · 13 canonical work pages · cited by 1 Pith paper

  1. [1]

    Julien Abadji , Pedro Ortiz Suarez , Laurent Romary , and Beno \^ t Sagot . 2022. https://arxiv.org/abs/2201.06642 Towards a Cleaner Document-Oriented Multilingual Crawled Corpus . arXiv e-prints, arXiv:2201.06642

    arXiv 2022

  2. [2]

    Asaad Alghamdi, Xinyu Duan, Wei Jiang, Zhenhai Wang, Yimeng Wu, Qingrong Xia, Zhefeng Wang, Yi Zheng, Mehdi Rezagholizadeh, Baoxing Huai, et al. 2023. Aramus: Pushing the limits of data and model scale for arabic natural language processing. arXiv preprint arXiv:2306.06800

    work page Pith review arXiv 2023

  3. [3]

    Ebtesam Almazrouei, Ruxandra Cojocaru, Michele Baldo, Quentin Malartic, Hamza Alobeidli, Daniele Mazzotta, Guilherme Penedo, Giulia Campesan, Mugariya Farooq, Maitha Alhammadi, Julien Launay, and Badreddine Noune. 2023. https://doi.org/10.18653/v1/2023.arabicnlp-1.21 A l G hafa evaluation benchmark for A rabic language models . In Proceedings of ArabicNLP...

    doi:10.18653/v1/2023.arabicnlp-1.21 2023

  4. [4]

    Zaid Alyafeai, Khalid Almubarak, Ahmed Ashraf, Deema Alnuhait, Saied Alshahrani, Gubran A. Q. Abdulrahman, Gamil Ahmed, Qais Gawah, Zead Saleh, Mustafa Ghaleb, Yousef Ali, and Maged S. Al-Shaibani. 2024. https://arxiv.org/abs/2402.03177 CIDAR: Culturally Relevant Instruction Dataset For Arabic . Preprint, arXiv:2402.03177

    arXiv 2024

  5. [5]

    Norah Alzahrani, Hisham Abdullah Alyahya, Yazeed Alnumay, Sultan Alrashed, Shaykhah Alsubaie, Yusef Almushaykeh, Faisal Mirza, Nouf Alotaibi, Nora Altwairesh, Areeb Alowisheq, et al. 2024. When benchmarks are targets: Revealing the sensitivity of large language model leaderboards. arXiv preprint arXiv:2402.01781

    arXiv 2024

  6. [6]

    Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. Aragpt2: Pre-trained transformer for arabic language generation. arXiv preprint arXiv:2012.15520

    arXiv 2020

  7. [7]

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. 2023. Qwen technical report. arXiv preprint arXiv:2309.16609

    arXiv 2023

  8. [8]

    M Saiful Bari, Yazeed Alnumay, Norah A Alzahrani, Nouf M Alotaibi, Hisham A Alyahya, Sultan AlRashed, Faisal A Mirza, Shaykhah Z Alsubaie, Hassan A Alahmed, Ghadah Alabduljabbar, et al. 2024. Allam: Large language models for arabic and english. arXiv preprint arXiv:2407.15390

    arXiv 2024

Show all 48 references
  1. [9]

    Marco Bellagente, Jonathan Tow, Dakota Mahan, Duy Phung, Maksym Zhuravinskyi, Reshinth Adithyan, James Baicoianu, Ben Brooks, Nathan Cooper, Ashish Datta, et al. 2024. Stable lm 2 1.6 b technical report. arXiv preprint arXiv:2402.17834

    2024 arXiv

  2. [10]

    Hasna Chouikhi, Manel Aloui, Cyrine Ben Hammou, Ghaith Chaabane, Haithem Kchaou, and Chehir Dhaouadi. 2024. Llamar & gemmar: Enhancing llms through arabic instruction-tuning. arXiv preprint arXiv:2407.02147

    2024 arXiv

  3. [11]

    Omar Einea, Ashraf Elnagar, and Ridhwan Al Debsi. 2019. Sanad: Single-label arabic news articles dataset for automatic text categorization. Data in brief, 25:104076

    2019

  4. [12]

    Leo Gao. 2021. https://blog.eleuther.ai/multiple-choice-normalization/ Multiple choice normalization in lm evaluation

    2021

  5. [13]

    Abbas Ghaddar, Yimeng Wu, Sunyam Bagga, Ahmad Rashid, Khalil Bibi, Mehdi Rezagholizadeh, Chao Xing, Yasheng Wang, Duan Xinyu, Zhefeng Wang, et al. 2022. Revisiting pre-trained language models and their evaluation for arabic natural language understanding. arXiv preprint arXiv:...

    2022 arXiv

  6. [14]

    Vipul Gupta, David Pantoja, Candace Ross, Adina Williams, and Megan Ung. 2024. Changing answer order can decrease mmlu accuracy. arXiv preprint arXiv:2406.19470

    2024 arXiv

  7. [15]

    Jan Hajic, Otakar Smrz, Petr Zem \'a nek, Jan S naidauf, and Emanuel Be s ka. 2004. Prague arabic dependency treebank: Development in data and tools. In Proc. of the NEMLAR Intern. Conf. on Arabic Language Resources and Tools, volume 1

    2004

  8. [16]

    Hallberg

    A. Hallberg. 2024. The arabic e-book corpus (version 1) [data set]. DOI: Available after publication

    2024

  9. [17]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300

    2020 arXiv

  10. [18]

    Huang Huang, Fei Yu, Jianqing Zhu, Xuening Sun, Hao Cheng, Dingjie Song, Zhihong Chen, Abdulmohsen Alharthi, Bang An, Ziche Liu, et al. 2023. Acegpt, localizing large language models in arabic. arXiv preprint arXiv:2309.12053

    2023 arXiv

  11. [19]

    Inception. 2024. https://huggingface.co/inceptionai/jais-family-30b-16k-chat/blob/main/README.md Jais family model card

    2024

  12. [20]

    "2024". https://arxiv.org/abs/

    "Fajri Koto, Haonan Li, Sara Shatanawi, Jad Doughman, Abdelrahman Boda Sadallah, Aisha Alraeesi, Khalid Almubarak, Zaid Alyafeai, Neha Sengupta, Shady Shehata, Nizar Habash, Preslav Nakov, and Timothy Baldwin". "2024". https://arxiv.org/abs/"2402.12840" "arabicmmlu: Assessing ...

    2024 arXiv

  13. [21]

    Imad Lakim, Ebtesam Almazrouei, Ibrahim Abualhaol, Merouane Debbah, and Julien Launay. 2022. A holistic assessment of the carbon footprint of noor, a very large arabic language model. In Proceedings of BigScience Episode\# 5--Workshop on Challenges & Perspectives in Creating L...

    2022

  14. [22]

    Jeffrey Li, Alex Fang, Georgios Smyrnis, Maor Ivgi, Matt Jordan, Samir Gadre, Hritik Bansal, Etash Guha, Sedrick Keh, Kushal Arora, et al. 2024. Datacomp-lm: In search of the next generation of training sets for language models. arXiv preprint arXiv:2406.11794

    2024 arXiv

  15. [23]

    Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Marc Marone, Christopher Akiki, Jia Li, Jenny Chim, et al. 2023. Starcoder: may the source be with you! arXiv preprint arXiv:2305.06161

    2023 arXiv

  16. [24]

    Tiedong Liu and Bryan Kian Hsiang Low. 2023. Goat: Fine-tuned llama outperforms gpt-4 on arithmetic tasks. arXiv preprint arXiv:2305.14201

    2023 arXiv

  17. [25]

    Lovish Madaan, Aaditya K Singh, Rylan Schaeffer, Andrew Poulton, Sanmi Koyejo, Pontus Stenetorp, Sharan Narang, and Dieuwke Hupkes. 2024. Quantifying variance in evaluation benchmarks. arXiv preprint arXiv:2406.10229

    2024 arXiv

  18. [26]

    Pratyush Maini, Skyler Seto, He Bai, David Grangier, Yizhe Zhang, and Navdeep Jaitly. 2024. Rephrasing the web: A recipe for compute and data-efficient language modeling. arXiv preprint arXiv:2401.16380

    2024 arXiv

  19. [27]

    Benjamin Minixhofer, Edoardo Maria Ponti, and Ivan Vuli \'c . 2024. Zero-shot tokenizer transfer. arXiv preprint arXiv:2405.07883

    2024

  20. [28]

    El Moatez Billah Nagoudi, AbdelRahim Elmadany, and Muhammad Abdul-Mageed. 2021. Arat5: Text-to-text transformers for arabic language generation. arXiv preprint arXiv:2109.12068

    2021 arXiv

  21. [29]

    Thuat Nguyen, Chien Van Nguyen, Viet Dac Lai, Hieu Man, Nghia Trung Ngo, Franck Dernoncourt, Ryan A Rossi, and Thien Huu Nguyen. 2023. Culturax: A cleaned, enormous, and multilingual dataset for large language models in 167 languages. arXiv preprint arXiv:2309.09400

    2023 arXiv

  22. [30]

    Michael Pieler, Marco Bellagente, Hannah Teufel, Duy Phung, Nathan Cooper, Jonathan Tow, Paulo Rocha, Reshinth Adithyan, Zaid Alyafeai, Nikhil Pinnaparaju, Maksym Zhuravinskyi, and Carlos Riquelme. 2024. https://arxiv.org/abs/2410.20796 Rephrasing natural text data with differ...

    2024 arXiv

  23. [31]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9

    2019

  24. [32]

    Jack W Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, et al. 2021. Scaling language models: Methods, analysis & insights from training gopher. arXiv preprint arXiv:2112.11446

    2021 arXiv

  25. [33]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2019. https://arxiv.org/abs/1910.10683 Exploring the limits of transfer learning with a unified text-to-text transformer . arXiv e-prints

    2019 arXiv

  26. [34]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1--67

    2020

  27. [35]

    Neha Sengupta, Sunil Kumar Sahu, Bokang Jia, Satheesh Katipomu, Haonan Li, Fajri Koto, Osama Mohammed Afzal, Samta Kamboj, Onkar Pandit, Rahul Pal, et al. 2023. Jais and jais-chat: Arabic-centric foundation and instruction-tuned open generative large language models. arXiv pre...

    2023 arXiv

  28. [36]

    Walelign Tewabe Sewunetie, Atnafu Lambebo Tonja, Tadesse Destaw Belay, Hellina Hailu Nigatu, Gashaw Kidanu, Zewdie Mossie, Hussien Seid, Eshete Derb, and Seid Muhie Yimam. 2024. Evaluating gender bias in machine translation for low-resource languages. In 5th Workshop on Africa...

    2024

  29. [37]

    Shivalika Singh, Freddie Vargus, Daniel Dsouza, Börje F. Karlsson, Abinaya Mahendiran, Wei-Yin Ko, Herumb Shandilya, Jay Patel, Deividas Mataciunas, Laura OMahony, Mike Zhang, Ramith Hettiarachchi, Joseph Wilson, Marina Machado, Luisa Souza Moura, Dominik Krzemiński, Hakimeh F...

    2024 arXiv

  30. [38]

    Pedro Javier Ortiz Su \'a rez, Beno \^ t Sagot, and Laurent Romary. 2019. Asynchronous pipeline for processing huge corpora on medium to low resource infrastructures. In 7th Workshop on the Challenges in the Management of Large Corpora (CMLC-7). Leibniz-Institut f \"u r Deutsc...

    2019

  31. [39]

    Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivi \`e re, Mihir Sanjay Kale, Juliette Love, et al. 2024. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08295

    2024 arXiv

  32. [40]

    Silma Team. 2024. https://www.silma.ai Silma

    2024

  33. [41]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  34. [42]

    Ahmet \"U st \"u n, Viraat Aryabumi, Zheng-Xin Yong, Wei-Yin Ko, Daniel D'souza, Gbemileke Onilude, Neel Bhandari, Shivalika Singh, Hui-Lee Ooi, Amr Kayid, et al. 2024. Aya model: An instruction finetuned open-access multilingual language model. arXiv preprint arXiv:2402.07827

    2024 arXiv

  35. [43]

    Haochun Wang, Sendong Zhao, Zewen Qiang, Bing Qin, and Ting Liu. 2024. Beyond the answers: Reviewing the rationality of multiple choice question answering for the evaluation of large language models. arXiv preprint arXiv:2402.01349

    2024 arXiv

  36. [44]

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. 2024. Qwen2 technical report. arXiv preprint arXiv:2407.10671

    2024 arXiv

  37. [45]

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2024. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36

    2024

  38. [46]

    Second language (arabic) acquisition of llms via progressive vocabulary expansion

    Jianqing Zhu, Huang Huang, Zhihang Lin, Juhao Liang, Zhengyang Tang, Khalid Almubarak, Abdulmohsen Alharthi, Bang An, Juncai He, Xiangbo Wu, et al. Second language (arabic) acquisition of llms via progressive vocabulary expansion

  39. [47]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  40. [48]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.