Pith. sign in

REVIEW 3 cited by

Sailor: Open Language Models for South-East Asia

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2404.03608 v1 pith:YKNGRXN5 submitted 2024-04-04 cs.CL cs.AI

Longxu Dou , Qian Liu , Guangtao Zeng , Jia Guo , Jiahui Zhou , Wei Lu , Min Lin This is my paper
classification cs.CLcs.AI
keywords modelslanguagesailorcasesdataincludinglanguagesmodel
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

We present Sailor, a family of open language models ranging from 0.5B to 7B parameters, tailored for South-East Asian (SEA) languages. These models are continually pre-trained from Qwen1.5, a great language model for multilingual use cases. From Qwen1.5, Sailor models accept 200B to 400B tokens, primarily covering the languages of English, Chinese, Vietnamese, Thai, Indonesian, Malay, and Lao. The training leverages several techniques, including BPE dropout for improving the model robustness, aggressive data cleaning and deduplication, and small proxy models to optimize data mixture. Experimental results on four typical tasks indicate that Sailor models demonstrate strong performance across different benchmarks, including commonsense reasoning, question answering, reading comprehension and examination. Embracing the open-source spirit, we share our insights through this report to spark a wider interest in developing large language models for multilingual use cases.

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. In-Place Tokenizer Expansion for Pre-trained LLMs

    cs.CL 2026-07 conditional novelty 6.0 of 10

    Continuing a model's own BPE merges and training only new embedding rows preserves quality while cutting token counts 2.4–4× for previously under-tokenized languages.

  2. Thunder-LLM: Efficiently Adapting LLMs to Korean with Minimal Resources

    cs.CL 2025-06 conditional novelty 6.0 of 10

    A cost-effective recipe consisting of tokenizer extension, continual pretraining, FP8 training, and SFT/DPO post-training yields Korean-English bilingual 8B models with top Korean benchmark scores.

  3. RefineX: Learning to Refine Pre-training Data at Scale from Expert-Guided Programs

    cs.CL 2025-07 conditional novelty 5.0 of 10

    A deletion-only program refiner, trained on expert end-to-end edits converted via minimum edit distance, improves LLM pretraining data and downstream accuracy.

Pith tools