REVIEW 16 cited by
Machine Psychology
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large language models (LLMs) show increasingly advanced emergent capabilities and are being incorporated across various societal domains. Understanding their behavior and reasoning abilities therefore holds significant importance. We argue that a fruitful direction for research is engaging LLMs in behavioral experiments inspired by psychology that have traditionally been aimed at understanding human cognition and behavior. In this article, we highlight and summarize theoretical perspectives, experimental paradigms, and computational analysis techniques that this approach brings to the table. It paves the way for a "machine psychology" for generative artificial intelligence (AI) that goes beyond performance benchmarks and focuses instead on computational insights that move us toward a better understanding and discovery of emergent abilities and behavioral patterns in LLMs. We review existing work taking this approach, synthesize best practices, and highlight promising future directions. We also highlight the important caveats of applying methodologies designed for understanding humans to machines. We posit that leveraging tools from experimental psychology to study AI will become increasingly valuable as models evolve to be more powerful, opaque, multi-modal, and integrated into complex real-world settings.
Forward citations
Cited by 16 Pith papers
-
The yes-no bias of large language models reflects answer order and wording, not shifts in moral judgment
LLM yes-no bias on moral dilemmas is an order-plus-lexical surface artifact, not a moral shift; models have a nearly format-invariant graded stance that the standard binary readout confounds.
-
The inattentional gap in task conditioned AI models that omit otherwise reportable safety critical signals
Task conditioning suppresses safety-critical signal reporting in language and vision models that unconstrained versions report at higher rates, creating an inattentional gap that decouples benchmark safety from real-w...
-
Strategic Intelligence in Large Language Models: Evidence from evolutionary Game Theory
Frontier LLMs survive and often thrive in evolutionary Prisoner's Dilemma tournaments, and each model family shows a distinct, context-dependent cooperation fingerprint.
-
From Representations to Behaviors: Exploring the Person-Situation-Behavior Triad in LLMs
SAE features recovered from matched high–low trait behaviors can be steered to bidirectionally shift situational personality expression and produce human-like social benefit–cost patterns in an 8B LLM.
-
Mixture of Cognitive Experts in Large Vision-Language Models
Routing CV experts into atomic evidence then Bloom-staged verbalization improves LVLM benchmarks and yields measurable query-conditioned reasoning traces.
-
Some Large Language Models Exhibit Consistent Risk Attitudes
Most of six LLMs show stable, cross-domain risk attitudes—consistent mappings from perceived risk to decisions—though these cluster in a narrower range than human risk preferences.
-
Large language models replicate and predict human cooperation across experiments in game theory
Llama-3.1-8B with a multi-step reasoning-and-filter prompt reproduces human cooperation rates across 121 dyadic games (MSD=0.031, r=0.89), outperforming Nash-equilibrium predictions (MSD=0.096, r=0.78).
-
The Mechanistic Emergence of Symbol Grounding in Language Models
Symbol grounding emerges in Transformers and state-space models through middle-layer 'aggregate' attention heads that connect environmental cues to words, but not in unidirectional LSTMs.
-
Quantifying Data Contamination in Psychometric Evaluations of LLMs
Across 21 LLMs and four standard psychology questionnaires, models recognize the items, know which trait each item measures, and can choose responses to hit a specified target score.
-
From Monolingual to Bilingual: Investigating Language Conditioning in Large Language Models for Psycholinguistic Tasks
Prompted language identity changes both the outputs and the internal layer representations of Llama-3.3-70B and Qwen2.5-72B on sound symbolism and word valence tasks.
-
Structured Prompting and Automated Evaluation in Fixed Synthetic Japanese-Language Counseling Dialogues
In a fixed set of Japanese AI-to-AI counseling dialogues, expert ratings favored a structured prompt over a minimal one, while automated LLM ratings were reproducible but systematically more lenient.
-
Large Language Models are Near-Optimal Decision-Makers with a Non-Human Learning Behavior
Across uncertainty, risk, and set-shifting tasks, LLMs generally outperformed humans and neared optimality while exhibiting distinctly non-human decision-making processes.
-
Humans are more gullible than LLMs in believing common psychological myths
Four LLMs believed 8 to 24 percent of 50 psychological myths, versus 51 to 63 percent for human students, and RAG generally, but not always, lowered belief rates.
-
A Conceptual Framework for AI Capability Evaluations
A descriptive conceptual framework with seven elements (target, task, subject, inputs, instance, measurement, result analysis) for systematizing analysis of AI capability evaluations.
-
Psychologically Enhanced AI Agents
MBTI personality prompts measurably change how LLM agents write stories and play strategic games, with self-reflection before communication supporting cooperative behavior.
- Social Pressure Breaks Majority Voting in LLM Safety Panels
Discussion (0). Sign in to comment.