REVIEW 6 cited by
The Semantic Hub Hypothesis: Language Models Share Semantic Representations Across Languages and Modalities
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Modern language models can process inputs across diverse languages and modalities. We hypothesize that models acquire this capability through learning a shared representation space across heterogeneous data types (e.g., different languages and modalities), which places semantically similar inputs near one another, even if they are from different modalities/languages. We term this the semantic hub hypothesis, following the hub-and-spoke model from neuroscience (Patterson et al., 2007) which posits that semantic knowledge in the human brain is organized through a transmodal semantic "hub" which integrates information from various modality-specific "spokes" regions. We first show that model representations for semantically equivalent inputs in different languages are similar in the intermediate layers, and that this space can be interpreted using the model's dominant pretraining language via the logit lens. This tendency extends to other data types, including arithmetic expressions, code, and visual/audio inputs. Interventions in the shared representation space in one data type also predictably affect model outputs in other data types, suggesting that this shared representations space is not simply a vestigial byproduct of large-scale training on broad data, but something that is actively utilized by the model during input processing.
Forward citations
Cited by 6 Pith papers
-
Cross-Lingual Bias in Large Language Models: A Comparative Analysis of English and Swahili
Bias in GPT-5.2 and Gemini 2.5 Flash changes rather than transfers between English and Swahili, with GPT-5.2 refusal behavior appearing only in English.
-
Inference-Time Steering for Cross-Lingual Factual Consistency in LLMs
Persona prompting beats CAA steering and DPO adapters at making an English-prompted LLM match its own German, Spanish, and Bulgarian answer distributions, and transfers better to cultural scenarios.
-
The Emergence of Abstract Thought in Large Language Models Beyond Any Language
Across 20 open LLMs, shared multilingual neurons grow in number and per-neuron importance over release generations, which the authors interpret as evidence of language-agnostic abstract thought and use to guide neuron...
-
Paths Not Taken: Understanding and Mending the Multilingual Factual Recall Pipeline
LLMs recall facts through an English-centric internal path and then translate the answer; injecting a translation vector and a recall vector raises accuracy by over 35 percentage points in the weakest language.
-
How and Where to Translate? The Impact of Translation Strategies in Cross-lingual LLM Prompting
For multilingual RAG intent classification, the best translation strategy depends on the model and language; translating instructions into the user's language helps some models, while making the model answer in low-re...
-
Language Matters: How Do Multilingual Input and Reasoning Paths Affect Large Reasoning Models?
Large reasoning models default to English or Chinese as internal 'reasoning hubs', and forcing non-hub reasoning lowers math accuracy, especially for low-resource languages, while sometimes improving safety and cultur...
Discussion (0). Continue with ORCID to comment.