REVIEW 5 cited by
Democratizing LLMs for Low-Resource Languages by Leveraging their English Dominant Abilities with Linguistically-Diverse Prompts
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Large language models (LLMs) are known to effectively perform tasks by simply observing few exemplars. However, in low-resource languages, obtaining such hand-picked exemplars can still be challenging, where unsupervised techniques may be necessary. Moreover, competent generative capabilities of LLMs are observed only in high-resource languages, while their performances among under-represented languages fall behind due to pre-training data imbalance. To elicit LLMs' ability onto low-resource languages without any supervised data, we propose to assemble synthetic exemplars from a diverse set of high-resource languages to prompt the LLMs to translate from any language into English. These prompts are then used to create intra-lingual exemplars to perform tasks in the target languages. Our unsupervised prompting method performs on par with supervised few-shot learning in LLMs of different sizes for translations between English and 13 Indic and 21 African low-resource languages. We also show that fine-tuning a 7B model on data generated from our method helps it perform competitively with a 175B model. In non-English translation tasks, our method even outperforms supervised prompting by up to 3 chrF++ in many low-resource languages. When evaluated on zero-shot multilingual summarization, our method surpasses other English-pivoting baselines by up to 4 ROUGE-L and is also favored by GPT-4.
Forward citations
Cited by 5 Pith papers
-
QUPID: Quantified Understanding for Enhanced Performance, Insights, and Decisions in Korean Search Engines
A fine-tuned ensemble of a generative small language model and an embedding model outperformed zero-shot LLMs on Korean search relevance labeling, with reported Cohen's kappa of 0.646 versus 0.387 and 60x lower latency.
-
Multilingual LLMs Inherently Reward In-Language Time-Sensitive Semantic Alignment for Low-Resource Languages
A translated temporal reasoning dataset plus a cross-lingual example retriever that outperforms semantic-alignment baselines on low-resource language temporal questions.
-
Breaking Language Barriers: Equitable Performance in Multilingual Language Models
Fine-tuning Llama-3-8B on synthetic Hindi-English code-switched CommonSenseQA data raises Hindi accuracy from 54% to 85.6% while English accuracy rises from 78% to 90.4%.
-
AI-Driven Generation of Old English: A Framework for Low-Resource Languages
A LoRA-fine-tuned 8B language model with backtranslation and a dual-agent pipeline generates Old English text, with reported English-to-Old English BLEU rising from 25.94 to 65.41.
-
Can Large Language Models Predict the Outcome of Judicial Decisions?
Fine-tuning a small LLaMA model on a new Arabic legal dataset yields near-par performance with a larger model, but the generalization claim is tested on the same instructions used during training.
Discussion (0). Continue with ORCID to comment.