pith. sign in

arxiv: 2606.07513 · v1 · pith:L7MSFZKUnew · submitted 2026-06-05 · 💻 cs.CL

Agentopia: Long-Term Life Simulation and Learning in Agent Societies

Xintao Wang , Sirui Zheng , Hongqiu Wu , Weiyuan Li , Jen-tse Huang , Minghao Zhu , Can Zu , Qi Deng
show 5 more authors
Jiawei Wang Qianyu He Heng Wang Xiaojian Wu Yunzhe Tao
This is my paper

Pith reviewed 2026-06-27 21:57 UTC · model grok-4.3

classification 💻 cs.CL
keywords long-term simulationagent societieslife rewardLLM trainingemergent behaviorsrole-playing benchmarkssocial intelligence
0
0 comments X

The pith

Long-term agent society simulations train LLMs on life-reward trajectories to boost social intelligence and role-playing performance.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper asks whether LLMs can acquire better social understanding by learning from years of simulated life in multi-agent societies rather than from static data alone. It builds Agentopia, a setup with 100 agents that autonomously pursue growth, form relationships, and meet needs across 10 simulated years. A life reward that mirrors human well-being selects successful trajectories for rejection-sampling training of the base LLM. If this works, the trained models produce agents with higher well-being inside the simulation and transfer the gains to external role-playing tasks.

Core claim

Running 10-year simulations of 100 autonomous agents in Agentopia and training LLMs via rejection sampling on trajectories chosen by a life reward that mirrors human well-being produces models whose enhanced social capabilities raise measured agent well-being inside the simulation and deliver a 15.6 percent gain on downstream role-playing benchmarks.

What carries the argument

Life reward, a scalar defined to mirror human well-being, that selects simulation trajectories for rejection-sampling updates to the underlying LLM.

If this is right

  • Agents exhibit rich emergent social behaviors that only appear across multi-year timescales.
  • Life-reward training directly raises agent well-being scores inside the ongoing simulation.
  • The resulting LLM generalizes beyond the simulation to produce a 15.6 percent lift on separate role-playing benchmarks.
  • Shorter simulations cannot generate the depth of interaction needed for the observed learning effect.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same life-reward selection method could be applied to other long-horizon simulated environments such as scientific collaboration or creative production.
  • Scaling agent count or simulation length beyond the reported 100 agents and 10 years might surface additional stable social structures.
  • If the life reward can be validated against real human outcome data, the approach offers a route to social alignment that relies on generated experience rather than curated human labels.

Load-bearing premise

The life reward accurately captures human well-being so that the selected trajectories supply a useful training signal for general social intelligence.

What would settle it

A test in which LLMs fine-tuned on the life-reward trajectories show no gain over the base model on the role-playing benchmarks or in which independent human raters judge the simulated agent lives as no better aligned with well-being than random trajectories.

read the original abstract

Humans learn from social life. Simulating this process with LLM-powered agents represents a promising research direction, raising a natural question: whether LLMs can learn from such simulated social experience to better understand and replicate human behavior. However, prior agent society simulations typically operate at the scale of days, limiting the depth of social interactions and long-term growth. In this paper, we study long-term life simulation and LLM learning in agent societies, with two goals: (1) investigating social behaviors that emerge from life-long simulation, and (2) developing anthropomorphic capabilities in LLMs, particularly intelligence in social life, through years of simulated social experience. Specifically, we present Agentopia, a comprehensive framework for long-term life simulation in multi-agent societies, where 100 agents autonomously pursue personal growth, develop social relationships, and fulfill their needs and goals over 10 simulated years. We define life reward to mirror human well-being, and leverage this reward to train LLMs via rejection sampling. Extensive experiments show that agents exhibit rich emergent social behaviors. Furthermore, life reward training effectively enhances the underlying LLM, which leads to improved agent well-being in simulation, and generalizes to downstream role-playing benchmarks with +15.6% improvement.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper introduces Agentopia, a framework for long-term (10-year) multi-agent life simulation involving 100 LLM-powered agents that autonomously pursue personal growth, form relationships, and fulfill needs. It defines a 'life reward' intended to mirror human well-being and applies it via rejection sampling to train the underlying LLM. Experiments report emergent social behaviors in simulation, improved agent well-being from the training, and a +15.6% gain on downstream role-playing benchmarks.

Significance. If the life reward is shown to be a non-circular, externally validated proxy for well-being and the benchmark gains are robust, the work would meaningfully extend agent-society research beyond short-horizon simulations and provide evidence that simulated long-term social experience can improve LLM social capabilities. The scale (100 agents, 10 years) and the dual focus on emergence plus LLM improvement are distinctive.

major comments (2)
  1. [Abstract] Abstract (and visible text): the central claim that 'life reward training effectively enhances the underlying LLM' and yields +15.6% on role-playing benchmarks rests on the unshown functional form of the life reward and its correlation with independent human well-being judgments on the same trajectories. Without this, measured gains may reflect optimization to the authors' heuristic rather than genuine capability growth.
  2. [Abstract] Abstract: no error bars, ablation on the life-reward definition, or description of how the +15.6% figure was obtained (e.g., which benchmarks, baseline, number of runs) are supplied, rendering the quantitative claim unverifiable from the provided text.
minor comments (1)
  1. [Abstract] The abstract states the reward 'mirrors human well-being' without clarifying whether this is a claim or a design goal; consistent terminology would help.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments highlighting the need for greater transparency in the abstract regarding the life reward and quantitative results. We agree these details are essential for verifiability and will revise the abstract to incorporate them while preserving its length constraints.

read point-by-point responses

  1. Referee: [Abstract] Abstract (and visible text): the central claim that 'life reward training effectively enhances the underlying LLM' and yields +15.6% on role-playing benchmarks rests on the unshown functional form of the life reward and its correlation with independent human well-being judgments on the same trajectories. Without this, measured gains may reflect optimization to the authors' heuristic rather than genuine capability growth.

    Authors: The functional form of the life reward is specified in Section 3.2 of the full manuscript as a composite score combining normalized metrics for physiological needs, safety, social belonging, esteem, and self-actualization, each weighted according to Maslow-inspired priorities and updated daily based on agent state. A post-hoc human evaluation on 200 sampled trajectories yielded a Pearson correlation of 0.68 with independent annotator well-being ratings. We will add a one-sentence description of this definition and the correlation result to the abstract to make the grounding explicit. revision: yes

  2. Referee: [Abstract] Abstract: no error bars, ablation on the life-reward definition, or description of how the +15.6% figure was obtained (e.g., which benchmarks, baseline, number of runs) are supplied, rendering the quantitative claim unverifiable from the provided text.

    Authors: The +15.6% figure represents the mean improvement on three role-playing benchmarks (SocialIQA, PersonaChat, and a held-out long-horizon dialogue set) relative to the untuned base model, computed as macro-averaged accuracy/F1 over five independent runs with standard deviation ±2.1%. Ablations on individual life-reward components appear in Appendix C. We will insert these specifics (benchmarks, baseline, run count, and error bars) into the abstract. revision: yes

Circularity Check

0 steps flagged

No significant circularity detected in derivation chain

full rationale

The provided abstract defines a life reward to mirror human well-being and applies it via rejection sampling to train the LLM, claiming resulting gains in simulated well-being and +15.6% on downstream benchmarks. No equations, fitting procedures, or self-citations are quoted that reduce any prediction or central claim to its own inputs by construction. The derivation chain relies on the external definition of the reward and reported experimental outcomes rather than self-referential loops or imported uniqueness results, making the paper self-contained on the evidence given.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review supplies no equations, no parameter lists, and no explicit assumptions; ledger entries are therefore empty by necessity.

pith-pipeline@v0.9.1-grok · 5786 in / 1150 out tokens · 16512 ms · 2026-06-27T21:57:22.162337+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

13 extracted references · 5 canonical work pages · 2 internal anchors

  1. [1]

    DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

    doi: 10.2307/2223319. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shi- rong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025. Bruce Headey. Hedonic adaptation. In Alex C. Michalos, editor,Encyclopedia of Quality...

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.2307/2223319 2025

  2. [2]

    https://thinkingmachines.ai/blog/on-policy-distillation

    doi: 10.64434/tml.20251026. https://thinkingmachines.ai/blog/on-policy-distillation. Abraham Harold Maslow. A theory of human motivation.Psychologicalreview, 50(4):370, 1943. Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd. The pagerank citation ranking: Bringing order to the web. InStanford InfoLab TechnicalReport, 1999. Joon Sung Park, Jo...

    work page doi:10.64434/tml.20251026 1943

  3. [3]

    Proximal Policy Optimization Algorithms

    Association for Computing Machinery. Yiting Ran, Xintao Wang, Tian Qiu, Jiaqing Liang, Yanghua Xiao, and Deqing Yang. BOOKWORLD: From novels to interactive agent societies for story creation. In Wanxiang Che, Joyce Nabende, Eka- terina Shutova, and Mohammad Taher Pilehvar, editors,Proceedings of the 63rd Annual Meeting ofthe AssociationforComputationalLin...

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.18653/v1/2025.acl-long.773 2025

  4. [4]

    Rui Xu, Xintao Wang, Jiangjie Chen, Siyu Yuan, Xinfeng Yuan, Jiaqing Liang, Zulong Chen, Xiaoqing Dong, and Yanghua Xiao

    URLhttps://aclanthology.org/2023.emnlp-demo.15. Rui Xu, Xintao Wang, Jiangjie Chen, Siyu Yuan, Xinfeng Yuan, Jiaqing Liang, Zulong Chen, Xiaoqing Dong, and Yanghua Xiao. Character is destiny: Can large language models simulate persona-driven decisions in role-playing?ArXiv preprint, abs/2404.12138, 2024. URLhttps://arxiv.org/ abs/2404.12138. An Yang, Anfe...

    work page doi:10.18653/v1/ 2023

  5. [5]

    High”, skill “Proficient

    Association for Computational Linguistics. doi: 10.18653/v1/2024.emnlp-industry.107. URL https://aclanthology.org/2024.emnlp-industry.107/. 21 Agentopia: Long-Term Life Simulation and Learning in Agent Societies A. World and Character Creation A.1. Pipeline Overview Each simulation world is designed as a self-contained community resembling a small town, w...

    work page doi:10.18653/v1/2024.emnlp-industry.107 2024

  6. [6]

    Cancel processing: if the proposer issued acancel_joint_activity, the proposal is voided and all invitees are notified

  7. [7]

    Response deduplication: for each invitee, only the lastrespond_invitation to a given proposal is kept; earlier responses are discarded

  8. [8]

    The priority order is: existing joint activity (from earlier weeks)> newly proposed joint activity > newly accepted joint activity> existing public/encounter activity

    Time conflict resolution: for each agent, if multiple schedules fall on the same day, the system keeps the one with the highest priority and automatically sets the rest to"no" with a reason attached. The priority order is: existing joint activity (from earlier weeks)> newly proposed joint activity > newly accepted joint activity> existing public/encounter...

  9. [9]

    yes", and(c)at least one invitee responded

    Activity creation: for each non-canceled proposal, the activity is created only if(a)the proposer has not been removed by a time conflict,(b)allrequired_participants responded"yes", and(c)at least one invitee responded"yes". 27 Agentopia: Long-Term Life Simulation and Learning in Agent Societies B.4. Details of Joint Activities This section describes the ...

    2025

  10. [10]

    Born with characters.Each character comes with an initial position as part of its background, assigned during the character creation process (§ A.1)

  11. [11]

    World initialization.After character creation and before the simulation starts, the environment model is provided with all positions initially held by characters and designs additional positions to enrich the world’s occupational structure. To ensure balance, the system imposes constraints on total capacity and diversity: total capacity across all positio...

  12. [12]

    New positions require minimum skill levels above the current highest among all agents, ensuring they serve as growth targets that agents must work toward

    Yearly growth.At the beginning of each year, the system addsmax(2,⌊𝑃/10⌋) new positions, where 𝑃 is the initial position count. New positions require minimum skill levels above the current highest among all agents, ensuring they serve as growth targets that agents must work toward. Income for new positions scales with skill requirements, and capacity is l...

    2025

  13. [13]

    Herbologyisherstrongestsubject...Thisisliterallyhercomfort zone

    and mixed into the training data as general-purpose samples. The training mixture consists of 50% role-playing data and 50% general-purpose data (measured by output tokens). A learning rate of1×10 −5 with a minimum learning rate of1×10 −6 is used, with a training batch size of 256. The model is fine-tuned for 1 epoch on 30 nodes of 8×H100 80GB GPUs. C.2. ...

    2025