REVIEW 4 cited by
SambaLingo: Teaching Large Language Models New Languages
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Despite the widespread availability of LLMs, there remains a substantial gap in their capabilities and availability across diverse languages. One approach to address these issues has been to take an existing pre-trained LLM and continue to train it on new languages. While prior works have experimented with language adaptation, many questions around best practices and methodology have not been covered. In this paper, we present a comprehensive investigation into the adaptation of LLMs to new languages. Our study covers the key components in this process, including vocabulary extension, direct preference optimization and the data scarcity problem for human alignment in low-resource languages. We scale these experiments across 9 languages and 2 parameter scales (7B and 70B). We compare our models against Llama 2, Aya-101, XGLM, BLOOM and existing language experts, outperforming all prior published baselines. Additionally, all evaluation code and checkpoints are made public to facilitate future research.
Forward citations
Cited by 4 Pith papers
-
Mangosteen: An Open Thai Corpus for Language Model Pretraining
An open 47B-token Thai pre-training corpus and a Thai-adapted data cleaning pipeline, with ablations showing quality gains and an 8B model that improves on Thai benchmarks.
-
ChatTime: A Unified Multimodal Time Series Foundation Model Bridging Numerical and Textual Data
ChatTime tokenizes time series as discrete foreign-language words, continually pre-trains and instruction-tunes LLaMA-2-7B via LoRA, and shows competitive zero-shot forecasting plus bimodal text and time series tasks.
-
Prune or Retrain: Optimizing the Vocabulary of Multilingual Models for Estonian
Pruning unused tokens from a multilingual encoder's vocabulary for Estonian preserves named-entity recognition F1 and cuts model size, while a retrained 32K tokenizer degrades performance under the tested training budget.
-
The Rise and Down of Babel Tower: Investigating the Evolution Process of Multilingual Code Large Language Model
A code LLM learning a new language first translates through its dominant language's internal system, then builds a separate system; this pattern can guide optimal training data mixing.
Discussion (0). Continue with ORCID to comment.