The reviewed record of science sign in
Pith

arxiv: 2407.13657 · v1 · pith:WHSFZROQ · submitted 2024-07-18 · cs.CL

FuLG: 150B Romanian Corpus for Language Model Pretraining

Vlad-Andrei B\u{a}doiu , Mihai-Valentin Dumitru , Alexandru M. Gherghescu , Alexandru Agache , Costin Raiciu This is my paper

Reviewed by Pithpith:WHSFZROQopen to challenge →

classification cs.CL
keywords fulgromaniancorporacorpuslanguagemanymodelspretraining
0
0 comments X
read the original abstract

Research in the field of language models is rapidly evolving, with many open models being released to the public. Openly available pretraining corpora usually focus on only a handful of languages, with many others either missing completely or extremely underrepresented. In this report, we introduce FuLG, a hundred-fifty-billion-token Romanian corpus extracted from CommonCrawl. We present our methodology for filtering FuLG and compare it via ablation studies against existing Romanian corpora.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.