Pith. sign in

REVIEW 1 cited by

AraMUS: Pushing the Limits of Data and Model Scale for Arabic Natural Language Processing

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2306.06800 v1 pith:FQVFYG4E submitted 2023-06-11 cs.CL

Asaad Alghamdi , Xinyu Duan , Wei Jiang , Zhenhai Wang , Yimeng Wu , Qingrong Xia , Zhefeng Wang , Yi Zheng
show 4 more authors
Mehdi Rezagholizadeh Baoxing Huai Peilun Cheng Abbas Ghaddar
This is my paper · ORCID
classification cs.CL
keywords arabicaramuslanguagedatanaturalplmsprocessingtasks
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Developing monolingual large Pre-trained Language Models (PLMs) is shown to be very successful in handling different tasks in Natural Language Processing (NLP). In this work, we present AraMUS, the largest Arabic PLM with 11B parameters trained on 529GB of high-quality Arabic textual data. AraMUS achieves state-of-the-art performances on a diverse set of Arabic classification and generative tasks. Moreover, AraMUS shows impressive few-shot learning abilities compared with the best existing Arabic PLMs.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Arabic Stable LM: Adapting Stable LM 2 1.6B to Arabic

    cs.CL 2024-12 conditional novelty 5.0 of 10

    A 1.6B Arabic language model trained with synthetic multiple-choice instruction data outperforms 7B-13B models on several Arabic multiple-choice benchmarks.

Pith tools