REVIEW 9 cited by
How do Large Language Models Handle Multilingualism?
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
abstract
Large language models (LLMs) have demonstrated impressive capabilities across diverse languages. This study explores how LLMs handle multilingualism. Based on observed language ratio shifts among layers and the relationships between network structures and certain capabilities, we hypothesize the LLM's multilingual workflow ($\texttt{MWork}$): LLMs initially understand the query, converting multilingual inputs into English for task-solving. In the intermediate layers, they employ English for thinking and incorporate multilingual knowledge with self-attention and feed-forward structures, respectively. In the final layers, LLMs generate responses aligned with the original language of the query. To verify $\texttt{MWork}$, we introduce Parallel Language-specific Neuron Detection ($\texttt{PLND}$) to identify activated neurons for inputs in different languages without any labeled data. Using $\texttt{PLND}$, we validate $\texttt{MWork}$ through extensive experiments involving the deactivation of language-specific neurons across various layers and structures. Moreover, $\texttt{MWork}$ allows fine-tuning of language-specific neurons with a small dataset, enhancing multilingual abilities in a specific language without compromising others. This approach results in an average improvement of $3.6\%$ for high-resource languages and $2.3\%$ for low-resource languages across all tasks with just $400$ documents.
Forward citations
Cited by 9 Pith papers
-
Multilingual Self-Taught Faithfulness Evaluators
STEMF trains multilingual faithfulness evaluators from synthetic data alone, and English-only training yields the best average results across languages.
-
Disentangling Language and Culture for Evaluating Multilingual Large Language Models
A new dual-axis evaluation framework shows multilingual LLMs answer culture-specific questions best when the question language matches the cultural context, with partial neuron-level evidence for the effect.
-
Paths Not Taken: Understanding and Mending the Multilingual Factual Recall Pipeline
LLMs recall facts through an English-centric internal path and then translate the answer; injecting a translation vector and a recall vector raises accuracy by over 35 percentage points in the weakest language.
-
What Language(s) Does Aya-23 Think In? How Multilinguality Affects Internal Language Representations
Aya-23-8B appears to activate multiple related languages internally and concentrate code-mixing neurons in final layers, but the paper's own limitations undercut the claim that these are language-specific neurons.
-
Seed-X: Building Strong Multilingual Translation LLM with 7B Parameters
A 7B open-weight translation model matches or outperforms far larger commercial systems across 28 languages in automatic and human evaluations.
-
Pruning General Large Language Models into Customized Expert Models
Cus-Prun identifies and removes neurons that are irrelevant to a user's target language, domain, and task, producing specialized expert models without post-training.
-
Exploring the Impact of Occupational Personas on Domain-Specific QA
Profession-based personas slightly improve LLM accuracy on science QA, while occupational personality personas often reduce it, even when semantically related.
-
Unveiling Instruction-Specific Neurons & Experts: An Analytical Framework for LLM's Instruction-Following Capabilities
Activation-frequency analysis identifies sparse units in LLMs that respond to instructions; same-category instructions share more of these units than different-category ones, and fine-tuning measurably changes the sets.
-
Small Language Models in the Real World: Insights from Industrial Text Classification
For 1B to 3B models, prompting alone is near random, while training a small classification head on frozen weights is the most accurate and VRAM-efficient path, with data volume and pretraining domain as the main bottlenecks.
Discussion (0). Sign in to comment.