REVIEW 7 cited by
Large Language Models as Superpositions of Cultural Perspectives
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large Language Models (LLMs) are often misleadingly recognized as having a personality or a set of values. We argue that an LLM can be seen as a superposition of perspectives with different values and personality traits. LLMs exhibit context-dependent values and personality traits that change based on the induced perspective (as opposed to humans, who tend to have more coherent values and personality traits across contexts). We introduce the concept of perspective controllability, which refers to a model's affordance to adopt various perspectives with differing values and personality traits. In our experiments, we use questionnaires from psychology (PVQ, VSM, IPIP) to study how exhibited values and personality traits change based on different perspectives. Through qualitative experiments, we show that LLMs express different values when those are (implicitly or explicitly) implied in the prompt, and that LLMs express different values even when those are not obviously implied (demonstrating their context-dependent nature). We then conduct quantitative experiments to study the controllability of different models (GPT-4, GPT-3.5, OpenAssistant, StableVicuna, StableLM), the effectiveness of various methods for inducing perspectives, and the smoothness of the models' drivability. We conclude by examining the broader implications of our work and outline a variety of associated scientific questions. The project website is available at https://sites.google.com/view/llm-superpositions .
Forward citations
Cited by 7 Pith papers
-
A Dual-Layered Evaluation of Geopolitical and Cultural Bias in LLMs
A multilingual two-phase evaluation shows LLMs lean on query language for factual questions and on training-country perspective for territorial and historical disputes.
-
EtiCor++: Towards Understanding Etiquettical Bias in LLMs
A new English etiquette corpus and bias metrics show that LLMs over-prefer Western norms and under-predict etiquettes from low-resource regions.
-
Do Language Models Think Consistently? A Study of Value Preferences Across Varying Response Lengths
Value preferences inferred from short-form LLM responses correlate only weakly (r around 0.05 to 0.25) with preferences inferred from long-form arguments, and alignment gives only modest consistency gains.
-
Meta-Cultural Competence: Climbing the Right Hill of Cultural Awareness
The paper argues that LLMs should be evaluated and built for meta-cultural competence rather than static knowledge of specific cultures, and gives a first, illustrative measurement of one component.
-
Reading between the Lines: Can LLMs Identify Cross-Cultural Communication Gaps?
A user study of 57 preselected Goodreads reviews finds culture-specific comprehension gaps in most texts, while GPT-4o identifies the relevant spans with 0.49 precision and 0.65 recall across India, Mexico, and the USA.
-
User Behavior Prediction as a Generic, Robust, Scalable, and Low-Cost Evaluation Strategy for Estimating Generalization in LLMs
The authors introduce an entropy-based framework that uses user behavior prediction as a measure of LLM generalization, and find GPT-4o outperforms GPT-4o-mini and Llama-3.1 on movie and music recommendation tasks.
-
Bridging the Gap: In-Context Learning for Modeling Human Disagreement
Across four open-source LLMs and three subjective-task datasets, in-context learning with multi-perspective prompts improves aggregated-label predictions in zero-shot, but disaggregated hard and soft label predictions...
Discussion (0). Continue with ORCID to comment.