REVIEW 5 cited by
Large Language Models Assume People are More Rational than We Really are
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
In order for AI systems to communicate effectively with people, they must understand how we make decisions. However, people's decisions are not always rational, so the implicit internal models of human decision-making in Large Language Models (LLMs) must account for this. Previous empirical evidence seems to suggest that these implicit models are accurate -- LLMs offer believable proxies of human behavior, acting how we expect humans would in everyday interactions. However, by comparing LLM behavior and predictions to a large dataset of human decisions, we find that this is actually not the case: when both simulating and predicting people's choices, a suite of cutting-edge LLMs (GPT-4o & 4-Turbo, Llama-3-8B & 70B, Claude 3 Opus) assume that people are more rational than we really are. Specifically, these models deviate from human behavior and align more closely with a classic model of rational choice -- expected value theory. Interestingly, people also tend to assume that other people are rational when interpreting their behavior. As a consequence, when we compare the inferences that LLMs and people draw from the decisions of others using another psychological dataset, we find that these inferences are highly correlated. Thus, the implicit decision-making models of LLMs appear to be aligned with the human expectation that other people will act rationally, rather than with how people actually act.
Forward citations
Cited by 5 Pith papers
-
Will Scaling Improve Social Simulation with LLMs?
Using 85 controlled and 35 public LLMs, the authors show social-simulation accuracy generally improves with compute, but some behavioral and low-resource tasks do not scale.
-
Recovering Event Probabilities from Large Language Model Embeddings via Axiomatic Constraints
A sign-flip constraint on one latent dimension of a VAE trained on LLM embeddings yields complementary event probabilities that sum to near one and track true probabilities on held-out dice events.
-
Are Large Language Models Reliable AI Scientists? Assessing Reverse-Engineering of Black-Box Systems
LLMs struggle to use passive observations for reverse engineering, but active intervention improves performance, largely through the process of generating queries rather than the data obtained.
-
Steering Risk Preferences in Large Language Models by Aligning Behavioral and Neural Representations
Behavioral and neural representations of risk are aligned via regression to produce steering vectors that reliably shift LLM risk preferences across tasks.
-
Revisiting Rogers' Paradox in the Context of Human-AI Interaction
A simulation of Rogers' Paradox with an AI agent that learns the population average shows that cheap AI alone does not improve collective world understanding, while critical appraisal and independent AI learning can.
Discussion (0). Continue with ORCID to comment.