REVIEW 6 major objections 6 minor 1 cited by
How Personality Traits Shape LLM Risk-Taking Behaviour
T0 review · 6 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash
Pith's one-line read The paper claims that GPT-4o reproduces the human link between Openness personality and risk-taking—higher Openness predicts more risk-seeking for gains and more risk aversion for losses—while defaulting to near-rational risk-neutral…
desk verdict A promising but under-validated study linking Big Five prompts to LLM risk parameters; the Openness finding needs a manipulation check before it can be taken at face value. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
Cumulative prospect theory (CPT) — a behavioural-economics model that turns probabilities and outcomes into subjective values through parameters $\alpha, \beta, \lambda, \gamma$ — is the measurement skeleton, and the Big Five personality framework supplies the intervention tool, with trait levels induced through adjective-marker prompts. The novel mechanism is a direct-reporting certainty-equivalent prompt ("What is the least positive amount of money that you would accept instead of taking the gamble?"), which yields more stable estimates than the prior adaptive questioning method, especially for loss prospects.
What would settle it
Re-running the same protocol on the same prospects with semantically equivalent but differently phrased gamble descriptions (such as "50% chance of $100, otherwise $0" versus "win $100 with 50% probability") and checking whether the fitted CPT parameters and the Openness-risk correlations stay within the paper's reported confidence intervals would settle whether the measurements are stable or prompt artifacts.
Extended reading notes
Core claim
The central claim is that GPT-4o's decision-making under risk can be described by cumulative prospect theory with parameters close to rational expectations, and that this behavior is systematically modulated by induced Big Five personalities. Specifically, the paper reports a correlation of $\rho = 0.63$ between induced Openness and risk-taking for gains, and $\rho = 0.44$ between Openness and risk aversion for losses, with Openness the most influential trait. The authors argue this mirrors human findings, while GPT-4-Turbo fails to generalize the relationship globally, showing monotonic agreement only for subsets of trait markers.
Load-bearing premise
The paper assumes that the single dollar amount GPT-4o reports in response to the certainty-equivalent question is a stable, faithful signal of its underlying risk preference, and that personality prompts change only the intended trait rather than affecting response style or the perceived meaning of the gamble.
Editorial extensions
If this is right
- Default GPT-4o should not be expected to show human loss aversion; personality prompts are needed to move its risk appetite in a human-like direction.
- Openness-level prompting provides a practical lever to shift GPT-4o's risk propensity in a predictable way, which could support tailored risk profiles in agentic financial systems.
- The model-specific failure of GPT-4-Turbo implies that the personality-risk mapping must be revalidated for each LLM rather than assumed to transfer across versions.
- The direct certainty-equivalent prompting protocol offers a simpler, more stable benchmark for measuring LLM risk preferences than the earlier adaptive method.
Reading between the lines
- A testable extension not stated in the paper: if the Openness-risk link reflects a genuine psychological mechanism, the same Openness prompts should also shift GPT-4o's choices in other economic tasks (e.g., auction bids or portfolio allocations), not just certainty equivalents.
- Because the study uses a single model snapshot, the reported correlations may drift with future GPT-4o updates; re-running the protocol on later versions would reveal whether the personality-risk mapping is stable across model versions.
- The method's reliance on one numeric answer leaves room for format effects, such as anchoring to round numbers; comparing with a multiple-choice or binary-accept/reject format would test whether the effect is robust to response format.
- The paper focuses on Openness, but its own table shows strong negative correlations between Agreeableness and loss aversion; a fuller trait-by-domain grid could reveal additional personality-risk relations worth mapping.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies whether GPT-4o's risk preferences, measured through cumulative prospect theory (CPT) parameters, are affected by induced Big Five personality traits, and whether those effects mirror human findings. The authors introduce a direct certainty-equivalent elicitation method, report CPT parameters for GPT-4o and GPT-4-Turbo with and without personality interventions, and compare the results with human baselines and with the earlier method of Ross et al. (2024). The headline empirical claim is that Openness interventions correlate positively with risk-taking for gains (ρ = 0.63) and with risk aversion for losses (ρ = 0.44), in agreement with human studies, and that Openness is the most influential trait for GPT-4o. The paper also claims GPT-4o acts as a risk-neutral rational agent, and that GPT-4-Turbo shows an inconsistent or local personality-risk mapping.
Significance. If the headline finding holds, it would be a useful step toward understanding and controlling risk behavior of LLM agents in financial simulations: the paper provides a transparent CPT estimation pipeline, uses external human CPT parameters as benchmarks, draws on out-of-sample prospects from choices13k, and reports per-seed variability. The comparison with Ross et al. (2024) and the inclusion of both gain- and loss-domain prospects are also strengths. However, the central Openness–risk correlation rests on a prompt-intervention manipulation that is not validated for Openness, and the certainty-equivalent measure is a single open-ended numeric answer that could be sensitive to response-style priming. These issues make the paper's main conclusion plausible but not yet established.
major comments (6)
- [§3.3, Figure 7, Table 5] The Openness intervention is the load-bearing manipulation for the headline ρ = 0.63 result, but it is never validated. The only intervention validation shown in Appendix A.2 (Figure 7) is for Conscientiousness, and even that figure shows strong inverse movement in Neuroticism alongside the intended Conscientiousness change. There is no equivalent IPIP-NEO-300 validation for Openness, so the reader cannot tell whether the adjective lists in Table 5 actually move Openness scores monotonically while leaving the other four traits approximately fixed. The markers include words like 'spontaneous' and 'socially progressive' that could plausibly change numeric response style rather than the target trait. Please add per-trait validation for all five interventions (target-level vs. measured trait scores, including leakage to other traits) and report the correlation between intervention level and measured Openness.
- [§3.2, Figure 1] The certainty equivalent is elicited as a single open-ended numeric answer ('least positive/most negative amount'), and the user prompt includes 'Lets think about this step by step.' With temperature set to 1.0 and a single numeric response, the measured CE can be affected by anchoring and by the personality adjectives in the system prompt without any change in underlying risk preference. The paper does not report the distribution of CE responses across the 15 runs, does not compare the open-ended elicitation with a choice-based elicitation for the personality-intervened models, and does not include control prompts containing non-personality numeric primes. Please report per-prospect CE distributions, add an irrelevant-adjective control condition, and ideally compare open-ended and choice-based CEs for the same personality interventions.
- [§4.1 and Figure 5a] The abstract and Section 5 describe GPT-4o as a 'risk-neutral rational agent,' but Section 4.1 states that for mixed prospects α < 1, which is risk aversion in the gain domain under CPT. The α point estimate and its confidence interval are not reported in the text; the claim that all parameters are statistically indistinguishable from unity explicitly excludes α. This is an overstatement of the evidence. Please report α with a bootstrapped confidence interval, test the null α = 1 directly, and either soften the 'risk-neutral rational agent' claim or show formally that the deviation from α = 1 is negligible for the decisions studied.
- [Table 1 and §4.3] The statistical basis of Table 1 is underspecified. It is not stated how many observations the Pearson correlations are computed over (five personality levels, possibly pooled over ten runs), whether the CPT parameters are treated as measured values without error, or how the t-statistics are derived. The table header also says 'over non-mixed prospects the dataset DB,' but DB as defined in Table 7 contains mixed prospects; if a different subset was used, it should be described precisely. Please state n, the pooling scheme, the propagation of CPT fitting uncertainty into the correlations, and correct the dataset description.
- [§4.4, Figure 6, Table 2] The claim that GPT-4-Turbo shows a localized but not global Openness-to-risk mapping is based on visual inspection of Figure 6. Table 2 reports negative weights for both α and β for GPT-4-Turbo, which the text itself describes as contradicting the human pattern, so the 'localized agreement' needs a formal statistical test. A piecewise regression with a breakpoint between the low-level (1–3) and high-level (5–9) marker groups, or an interaction test, would make the claim falsifiable. As written, the global-versus-local distinction is not quantitatively supported.
- [§3.2 and Appendix A.4] The claimed superiority of the new CE elicitation over Ross et al. (2024) rests on visual inspection of Figure 10a/10b. No quantitative metric is reported (e.g., outlier counts, median absolute deviation from expected value, or variance of CE estimates). Since contribution 2 depends on this comparison, please report a numeric stability and accuracy comparison for the two methods over the same prospects and the same number of runs.
minor comments (6)
- [§4.1] Typo: 'risk neural agents' should be 'risk-neutral agents.'
- [§4.2] The text refers to 'α in Table 5a' when the CPT parameter estimates are in Figure 5a; please fix the cross-reference.
- [Contribution list, §1] Contribution 5 says GPT-4-Turbo 'does generalise the personality-risk relationship,' which contradicts Section 4.4 and the abstract; the intended wording is presumably 'does not generalize' or 'shows inconsistent generalization.'
- [§4.3] The sentence 'We not observe any statistically significant correlations' is ungrammatical; it should be 'We do not observe...'.
- [Table 1] The significance levels are denoted α = 0.05/0.025/0.001, which conflicts notationally with the CPT gain-sensitivity parameter α; consider using p < 0.05 etc.
- [§3.2] The text says certainty equivalents are inferred 'over 15 runs in each experiment,' while Section 4.1 reports '10 random seeds'; please reconcile the number of runs throughout.
Circularity Check
No circular derivation: CPT parameters are fitted to GPT-4o responses and compared against external human baselines; the personality intervention is cited external work, not the authors' own result.
full rationale
The derivation chain is: (i) elicit certainty equivalents from GPT-4o using the prompt in Figures 1–2; (ii) fit CPT parameters θ by minimizing Eq. 4 against those certainty equivalents; (iii) induce Big Five levels using the Serapio-García et al. (2023) markers in Table 5; (iv) correlate intervention levels with the fitted parameters. No step feeds the target conclusion back into its own premises. The human-like Openness–risk claim is checked against external human results (Tversky & Kahneman 1992; Rustichini et al. 2016; Highhouse et al. 2022), not defined by them. The two self-citations (Hamill et al. 2025; Hartley et al. 2023) are contextual and not load-bearing. A genuine construct-validity concern exists—the Openness markers include risk-related adjectives such as 'spontaneous', and the appendix validates only the Conscientiousness intervention—but that is a response-style/validity threat, not a circular reduction: the Openness level is not defined in terms of the risk outcome, and the reported correlation is an empirical result rather than an identity. Therefore no circular step meets the quoted-reduction bar.
Assumptions & free parameters
free parameters (5)
- α (gain sensitivity) =
estimated below 1; exact value not stated
- β (loss sensitivity) =
estimated near 1
- λ (loss aversion) =
estimated near 1 for GPT-4o; human reference 2.25
- ϕ+ and ϕ− (probability weighting for gains and losses) =
estimated near 1
- Openness-to-CPT linear weights ω in Table 2 =
GPT-4o: α 0.039, β 0.052, λ -0.43; GPT-4-Turbo: α -0.096, β -0.092, λ -0.67
assumptions (5)
- domain assumption Cumulative prospect theory with power value function and single-parameter probability weighting adequately represents LLM risk preferences.
- domain assumption Self-reported IPIP-NEO-300 responses measure the LLM's personality traits.
- domain assumption Bipolar adjective marker prompts from Serapio-García et al. (2023) induce the intended trait independently of other traits.
- domain assumption Human comparison samples (Johnson 2005 for personality, Tversky & Kahneman 1992 for CPT) are appropriate baselines.
- domain assumption The LLM's stated certainty equivalent reflects a stable underlying preference rather than arbitrary text generation.
Cite this review
Pith. "Pith review of How Personality Traits Shape LLM Risk-Taking Behaviour." pith.science (2026). https://pith.science/paper/D4LLZY3Y
@misc{pith2026250304735,
author = {Pith},
title = {Pith review of: How Personality Traits Shape LLM Risk-Taking Behaviour},
year = {2026},
howpublished = {\url{https://pith.science/paper/D4LLZY3Y}},
note = {Machine review of arXiv:2503.04735}
}
read the original abstract
Large Language Models (LLMs) are increasingly deployed as autonomous agents, necessitating a deeper understanding of their decision-making behaviour under risk. This study investigates the relationship between LLMs' personality traits and risk propensity, employing cumulative prospect theory (CPT) and the Big Five personality framework. We focus on GPT-4o, comparing its behaviour to human baselines and earlier models. Our findings reveal that GPT-4o exhibits higher Conscientiousness and Agreeableness traits compared to human averages, while functioning as a risk-neutral rational agent in prospect selection. Interventions on GPT-4o's Big Five traits, particularly Openness, significantly influence its risk propensity, mirroring patterns observed in human studies. Notably, Openness emerges as the most influential factor in GPT-4o's risk propensity, aligning with human findings. In contrast, legacy models like GPT-4-Turbo demonstrate inconsistent generalization of the personality-risk relationship. This research advances our understanding of LLM behaviour under risk and elucidates the potential and limitations of personality-based interventions in shaping LLM decision-making. Our findings have implications for the development of more robust and predictable AI systems such as financial modelling.
Figures
Figures from the paper (9 more)
Forward citations
Cited by 1 Pith paper
-
NextFund: A Unified Performance Tracking Platform for Agentic Portfolio Management
NextFund unifies live multi-market data, multi-agent portfolio reasoning, and end-to-end decision traces with an interactive Trading Arena for fairer LLM agent benchmarking.
Reference graph
Works this paper leans on
-
[2]
e xt rem el y { l o w _ m a r k e r }
B.3 Datasets In this section we provide a comprehensive description of the datasets used in our work. Figure 12 shows the summary statistics for the datasets DA and DB. 20 Table 5: Low and high markers for the Big Five personality traits (Serapio-García et al., 2023). Trait Low Level Marker (levels 1–3) High Level Marker (levels 5–9) Conscientiousness uns...
work page 2023
-
[10]
Generative agent simulations of 1,000 people
Park, Joon Sung, Zou, Carolyn Q, Shaw, Aaron, Hill, Benjamin Mako, Cai, Carrie, Morris, Mered- ith Ringel, Willer, Robb, Liang, Percy, and Bernstein, Michael S. Generative agent simulations of 1,000 people. arXiv preprint arXiv:2411.10109,
-
[13]
Röttger, Paul, Hofmann, Valentin, Pyatkin, Valentina, Hinck, Musashi, Kirk, Hannah Rose, Schütze, Hinrich, and Hovy, Dirk. Political compass or spinning arrow? towards more meaningful eval- uations for values and opinions in large language models. arXiv preprint arXiv:2402.16786 ,
-
[15]
Llama: Open and efficient foundation language models
Touvron, Hugo, Lavril, Thibaut, Izacard, Gautier, Martinet, Xavier, Lachaux, Marie-Anne, Lacroix, Timothée, Rozière, Baptiste, Goyal, Naman, Hambro, Eric, Azhar, Faisal, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971,
-
[16]
Cultural evolution of cooperation among llm agents
Vallinder, Aron and Hughes, Edward. Cultural evolution of cooperation among llm agents. arXiv preprint arXiv:2412.10270,
-
[18]
Agentless: Demystifying llm-based software engineering agents
Xia, Chunqiu Steven, Deng, Yinlin, Dunn, Soren, and Zhang, Lingming. Agentless: Demystifying llm-based software engineering agents. arXiv preprint arXiv:2407.01489,
-
[19]
Zhang, Chong, Liu, Xinyi, Jin, Mingyu, Zhang, Zhongmou, Li, Lingyao, Wang, Zhengting, Hua, Wenyue, Shu, Dong, Zhu, Suiyuan, Jin, Xiaobo, et al. When ai meets finance (stockagent): Large language model-based stock trading in simulated real-world environments. arXiv preprint arXiv:2407.18957,
-
[20]
16 A Supplementary results A.1 Median certainty equivalents for non-mixed gambles Table 3 shows the median certainty equivalents for GPT-4o, human and rational agents estimated over the non-mixed prospects in the dataset DA. The certainty equivalents for GPT-4o and rational agents identical except for a small minority of values. This result strongly sugge...
work page 1992
Show all 22 references
-
[22]
CE is the median certainty equivalent estimated from 25 graduate students
0 5 10 15 20 25 30 Count Dataset DB DA (a) The distribution of probabilities associated with Outcome 2 for prospects in datasets DA, DB -400-100 -10 0 10 100 400 [x] 0.000 0.005 0.010 0.015 0.020 Density Dataset DB DA (b) The distribution expected values for prospects in datas...
1992
-
[1979]
Estimating the personality of white-box language models
Karra, Saketh Reddy, Nguyen, Son The, and Tulabandhula, Theja. Estimating the personality of white-box language models. arXiv preprint arXiv:2204.12000,
-
[1989]
Large language model agent in financial trading: A survey
Ding, Han, Li, Yinheng, Wang, Junhao, and Chen, Hang. Large language model agent in financial trading: A survey. arXiv preprint arXiv:2408.06361,
-
[1992]
Who is gpt-3? an exploration of personality, values and demographics
Miotto, Marilù, Rossberg, Nicola, and Kleinberg, Bennett. Who is gpt-3? an exploration of personality, values and demographics. arXiv preprint arXiv:2209.14338,
-
[1996]
Personality traits in large language models
Serapio-García, Greg, Safdari, Mustafa, Crepy, Clément, Sun, Luning, Fitz, Stephen, Romero, Peter, Abdulhai, Marwa, Faust, Aleksandra, and Matari ´c, Maja. Personality traits in large language models. arXiv preprint arXiv:2307.00184,
-
[1999]
Machine psychology: Investigating emergent capabilities and behavior in large language models using psychological methods
Hagendorff, Thilo. Machine psychology: Investigating emergent capabilities and behavior in large language models using psychological methods. arXiv preprint arXiv:2303.13988,
-
[2008]
Llm economicus? mapping the behavioral biases of llms via utility theory
Ross, Jillian, Kim, Yoon, and Lo, Andrew W. Llm economicus? mapping the behavioral biases of llms via utility theory. arXiv preprint arXiv:2408.02784,
-
[2009]
Turning large language models into cognitive models
Binz, Marcel and Schulz, Eric. Turning large language models into cognitive models. arXiv preprint arXiv:2306.03917, 2023a. Binz, Marcel and Schulz, Eric. Using cognitive psychology to understand gpt-3. Proceedings of the National Academy of Sciences, 120(6):e2218523120, 2023b...
-
[2012]
The rise and potential of large language model based agents: A survey
Xi, Zhiheng, Chen, Wenxiang, Guo, Xin, He, Wei, Ding, Yiwen, Hong, Boyang, Zhang, Ming, Wang, Junzhe, Jin, Senjie, Zhou, Enyu, et al. The rise and potential of large language model based agents: A survey. arXiv preprint arXiv:2309.07864,
-
[2013]
Simulating financial market via large language model based agents
Gao, Shen, Wen, Yuntao, Zhu, Minghang, Wei, Jianing, Cheng, Yuhan, Zhang, Qunzi, and Shang, Shuo. Simulating financial market via large language model based agents. arXiv preprint arXiv:2406.19966,
-
[2016]
Sparks of artificial general intelligence: Early experiments with gpt-4
Bubeck, Sébastien, Chandrasekaran, Varun, Eldan, Ronen, Gehrke, Johannes, Horvitz, Eric, Kamar, Ece, Lee, Peter, Lee, Yin Tat, Li, Yuanzhi, Lundberg, Scott, et al. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712,
-
[2021]
Communicative agents for software development
Qian, Chen, Cong, Xin, Yang, Cheng, Chen, Weize, Su, Yusheng, Xu, Juyuan, Liu, Zhiyuan, and Sun, Maosong. Communicative agents for software development. arXiv preprint arXiv:2307.07924, 6,
-
[2023]
Inducing anxiety in large language models increases exploration and bias
Coda-Forno, Julian, Witte, Kristin, Jagadish, Akshay K, Binz, Marcel, Akata, Zeynep, and Schulz, Eric. Inducing anxiety in large language models increases exploration and bias. arXiv preprint arXiv:2304.11111,
-
[2024]
Does gpt- 3 demonstrate psychopathy? evaluating large language models from a psychological perspective
Li, Xingxuan, Li, Yutong, Joty, Shafiq, Liu, Linlin, Huang, Fei, Qiu, Lin, and Bing, Lidong. Does gpt- 3 demonstrate psychopathy? evaluating large language models from a psychological perspective. arXiv preprint arXiv:2212.10529,
Reviewed August 9, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.