Pith. sign in

REVIEW 4 major objections 4 minor 55 references

SimuPanel: A Novel Immersive Multi-Agent System to Simulate Interactive Expert Panel Discussion

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SimuPanel claims that LLM agents running a full reasoning chain — knowledge recall, analysis, evaluation, and strategy-selecting inference — can simulate academic panel discussions that learners find in-depth and engaging.

desk verdict A plausible systems contribution whose central evaluation claim currently rests on an unvalidated LLM judge and opaque scoring; deserve review after a major revision. read the letter →

arxiv 2506.16010 v1 pith:XN3ZW6VA submitted 2025-06-19 cs.HC

Xiangyang He , Jiale Li , Jiahao Chen , Yang Yang , Mingming Fan This is my paper
classification cs.HC
keywords SimuPanelmulti-agentsimulationdiscussionlargelanguagemodelsrole-playingagentsmultimedialearningLLM-as-judgeevaluationablationstudy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper presents SimuPanel, a system that stages academic panel discussions with LLM agents as expert panelists, a host agent as moderator, and an immersive 3D interface for the learner-audience. The central claim is that depth of simulated discussion comes from an integrated reasoning architecture: an agent that recalls relevant knowledge, analyzes and evaluates it, and then chooses a discourse strategy before speaking produces materially deeper dialogue than any stripped-down variant. The evidence is an ablation study in which the Full Reasoning-chain outscored five ablations on an LLM-judged ELO comparison (38.17 total points versus 31.00 for the runner-up), plus a user study with ten graduate students in which eight of ten reported new perspectives and the system scored 88 on the System Usability Scale. If the claim holds, the pedagogical benefit of watching experts think aloud — currently limited to conference attendees — becomes an on-demand multimedia learning format for any topic with sufficient source material. The paper also surfaces tension in its own data: self-rated learning effectiveness was moderate, and some participants noticed repetitive dialogue patterns and wanted more methodological detail.

What carries the argument

The load-bearing mechanism is the modular reasoning pipeline of each expert agent — recall, analysis, evaluation, inference — combined with a two-layer persona (a low-level layer of domain knowledge retrieved from public academic materials by retrieval-augmented generation, and a high-level layer of research interests and beliefs summarized from those materials) and a host agent whose internal state machine walks the panel through opening, discussion, convergence, and closing stages. The host decides CONTINUE, TRANSITION, or REDIRECT on each turn, with bounded turn management enforcing between $\tau_{\min}=3$ and $\tau_{\max}=6$ exchanges per segment, which keeps open-ended dialogue coherent without scripting it. Inside the pipeline, the strategy-selection rule $s^{*}=\arg\max_{s\in S}\{\mathrm{educational\_value}(s)+\mathrm{belief\_alignment}(s,P.\mathrm{beliefs})\}$ is what converts retrieved content into interactive moves such as questioning, agreeing, or critiquing a colleague's claim. The paper's ablation study isolates the contribution of this machinery by testing six configurations that progressively add blocks of the pipeline, and the user study tests whether the resulting dialogues support learning and engagement in the 3D interface.

What would settle it

Rate the same generated dialogues — the paper generated two runs per strategy per topic, so re-rating is feasible — with human experts familiar with the five panel topics, using the six published criteria, and compare the human ordering with DeepSeek-R1's ELO ordering. If humans do not place the Full Reasoning-chain first, or if substituting a different LLM judge reverses the winner, the central claim that the full reasoning chain produces the most in-depth discussions fails.

Watch

Extended reading notes

Core claim

On the paper's own terms, expert simulation is not a matter of giving a language model a biography; it emerges from a cognitive architecture in which each agent is a tuple of persona, knowledge base, reasoning process, and utterance generator. The reasoning process is the pipeline $\mathcal{R}(q,K,H)=\mathcal{R}_{\mathrm{inference}}\circ\mathcal{R}_{\mathrm{evaluate}}\circ\mathcal{R}_{\mathrm{analysis}}\circ\mathcal{R}_{\mathrm{recall}}(q,K,H)$, and the decisive module is the last one: the agent selects a discourse strategy from question, answer, scholarly agreement, constructive critique, and synthesis by maximizing the sum of educational value and alignment with the persona's beliefs, $s^{*}=\arg\max_{s\in S}\{\mathrm{educational\_value}(s)+\mathrm{belief\_alignment}(s,P.\mathrm{beliefs})\}$. The ablation results are the paper's main quantitative evidence: the Full Reasoning-chain scored 38.17 against 31.00 for Guided Discourse, 25.00 for Critical Analysis, 22.49 for Strategic Inference, 20.00 for Basic Recall, and 6.67 for a one-shot baseline, which the authors read as proof that analytical processing and strategic inference work synergistically rather than additively. A user study with ten graduate students found that eight of ten gained new insights, with a moderate self-reported learning-effectiveness score (M = 3.40) and high usability (SUS mean 88), and the design discussion positions the system for learners with some background who seek inspiration and new questions rather than learners seeking step-by-step answers.

Load-bearing premise

The ranking of the six dialogue-generation strategies rests entirely on comparative judgments made by the DeepSeek-R1 language model, and the paper reports no human validation, inter-rater reliability, or calibration of that judge; if the judge favors verbose, argumentatively structured output, the Full Reasoning-chain's win could be an artifact of the judge's style rather than a measure of discussion quality.

Editorial extensions

If this is right

  • If the Full Reasoning-chain claim holds, any institution can offer on-demand simulated panel discussions on any topic with sufficient source material, removing the geographic, financial, and scheduling barriers that keep most learners out of live academic panels.
  • The ablation ordering gives system builders a concrete recipe: connect recall, analysis, evaluation, and strategy inference in that order before generating an utterance, since each added module improves the ELO total (6.67 to 38.17 across the six configurations).
  • The host-agent state machine with bounded interventions is a portable control mechanism for keeping open-ended multi-agent dialogue focused without pre-scripting the content.
  • For learners, the intended use case is discovery: participants with some background knowledge used the panels to surface new questions and research directions, while participants seeking concrete methodologies were less satisfied.
  • The system reframes expert agents as epistemic lenses rather than knowledge repositories: what learners gain is not facts but exposure to how a persona with given beliefs processes, evaluates, and responds to another expert's position.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The headline ELO gap is only as trustworthy as the judge: an editorial reading suggests the decisive check is a human re-rating of the six strategy outputs, or at minimum a second LLM judge, before the 38.17-versus-31.00 ordering is treated as a property of the dialogues.
  • Nothing in the architecture binds it to academia; the same host-expert, staged-reasoning design should transfer to policy briefings, clinical second-opinion discussions, or multi-perspective news analysis, where structured expert disagreement carries similar learning value.
  • The paper reports a per-utterance latency of 'xx seconds' (the value is left unspecified in the manuscript), which suggests live use is currently slow; a testable extension is whether running only the inference module on a smaller model preserves most of the full-chain advantage.
  • The immersive 3D layer and the dialogue quality are confounded in the user study, so a text-only condition using the same Full Reasoning-chain dialogues would isolate whether the visual environment or the content drives engagement and reflection.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper presents SimuPanel, an LLM-based multi-agent system that simulates academic panel discussions. The system combines a host-expert agent architecture, two-layer persona construction grounded in public sources, a RAG-based knowledge base, modular reasoning steps (recall, analysis, evaluation, inference), and a Unity-based 3D interface with note-taking and follow-up Q&A. The authors report a technical evaluation using an LLM judge (DeepSeek-R1) to compare six reasoning configurations across five topics, and a user study with 10 graduate students. The central claim is that the Full Reasoning-chain configuration produces more in-depth discussions and that SimuPanel engages learners; the paper concludes with design implications and limitations.

Significance. If validated, SimuPanel addresses a real accessibility gap in academic panel discussions and offers a plausible use of multi-agent LLMs for multimedia learning. Strengths include the end-to-end system implementation, the two-layer persona grounding from real scholarly sources, the modular reasoning design, and the attempt to evaluate both technical quality and user experience. The manuscript is honest about several limitations. However, the empirical evidence for the headline claim that SimuPanel simulates more in-depth discussions is not yet convincing: the technical evaluation relies on a single unvalidated LLM judge without human calibration, and the user study is a small uncontrolled qualitative study whose self-report measures do not directly assess discussion depth.

major comments (4)
  1. [Section 5.2 and 5.3, Table 1] The entire ablation ranking is produced by DeepSeek-R1 as judge, with no human validation, inter-rater reliability, or calibration of the judge against human preferences. The paper cites AI-Scientist and Chain of Ideas as precedents for LLM-based evaluation, but those works include human-alignment checks; here no such check is reported. Because the judged outputs are themselves LLM-generated and the Full Reasoning-chain is designed to contain more analytical and strategic structure, the judge may be rewarding surface features such as utterance length, discourse markers, or argumentative framing rather than human-perceived depth. Consequently, the claim that FR 'simulates more in-depth discussions' is not yet supported.
  2. [Table 1 and Section 5.2] The values in Table 1 labeled 'ELO' appear to be simple arithmetic sums of the six dimension scores: for Full Reasoning-chain, 7.33+4.17+5.67+6.17+7.33+7.50 = 38.17, and the reported average 6.36 equals 38.17/6. The text says ELO scores were computed via a Round-Robin tournament, but no ELO calculation, number of pairwise comparisons, tie handling, or mapping from pairwise judgments to these totals is described. If the reported numbers are not ELO scores, the description in Section 5.2 is misleading; if they are ELO scores, their construction must be specified. Either way, this affects the load-bearing quantitative comparison in the paper.
  3. [Section 5.1 and 5.3, Table 1] Each configuration was generated only twice per topic, and Table 1 reports no variance, confidence intervals, or significance tests. Differences such as Strategic Inference (22.49) versus Critical Analysis (25.00) are likely within noise for pairwise LLM-judge evaluations on five topics. Without per-topic scores, error bars, or an inferential test, the ranking of intermediate strategies is not established, and even the top-ranked Full Reasoning-chain result should be treated as preliminary.
  4. [Section 6.2] The user study has 10 participants, no control condition, and no inter-rater reliability analysis for the pre/post open-ended responses. The self-reported learning effectiveness scale has a moderate mean of 3.40, and several participants explicitly described formulaic dialogue or difficulty following lengthy utterances (e.g., P5). Therefore the user study cannot serve as independent evidence for the claim that SimuPanel simulates more in-depth discussions; it is more accurately a preliminary usability and user-experience study.
minor comments (4)
  1. [Section 1, Contributions] There is a typo in the second contribution: '3D expert avatars with with multimodal representations' should be '3D expert avatars with multimodal representations.'
  2. [Section 5.3, Guided Discourse description] The sentence describing Guided Discourse is grammatically broken: 'Augments The impact of critical analysis results on utterance output with explicit discourse guidance' should be rewritten.
  3. [Section 7.3] The text contains placeholder values and missing references: 'takes xx seconds to output one utterance' and 'like CrossTalk [?]' should be filled in before publication.
  4. [Figure 2] Figure 2 is hard to read at the resolution provided; the six strategy names and the color-coding of module functions are not clearly legible. A larger, higher-contrast version would improve the paper's accessibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the derivation chain is empirical, with no fitted parameter, definitional shortcut, or self-citation chain forcing the headline result.

full rationale

SimuPanel is a systems-building paper rather than a derivation paper. The headline 'more in-depth discussions' rests on an LLM-judge ablation comparison and an uncontrolled n=10 user study. Neither is circular in the sense the analyzer targets: no fitted parameter is subsequently reported as a prediction; the persona and knowledge inputs come from public expert materials; and the six evaluation criteria are adapted from external prior work (Fu et al.) rather than defined in terms of the system's own modules. The main validity risk is that DeepSeek-R1's pairwise judgments were not calibrated against human ratings, so the FR strategy may be preferred for style or length rather than depth; however, that is an evaluation-validity concern, not an equation-level reduction. No load-bearing self-citation is used: the citations to prior LLM-judge works (ResearchAgent, AI-Scientist, Li et al.) are external references, not uniqueness theorems or fitted outputs. The missing 'CrossTalk' reference and the Table 1 column labeled 'ELO' that numerically equals a sum of averages are reporting issues, not circular reasoning. Under the stated hard rules, no circular step can be exhibited, so the score is 0.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The central claims rest on evaluation and persona-fidelity assumptions, not on fitted constants in a derivation. The only hand-chosen parameters are the host intervention thresholds. No new physical or formal entities are introduced; the software agents (host, expert, audience) are the system under construction, not posited constructs requiring external evidence.

free parameters (1)
  • Host intervention thresholds (tau_min, tau_max) = 3 and 6 exchanges
    Chosen by hand in Section 4.2 to bound turn counts between host interventions. This directly shapes dialogue depth and pacing and is not derived from data.
assumptions (5)
  • domain assumption DeepSeek-R1 comparative judgments are a valid proxy for human assessment of dialogue quality
    Section 5.2 uses DeepSeek-R1 as LLM judge to compute ELO scores without human validation or inter-rater reliability checks.
  • domain assumption Personas built from scraped public sources faithfully represent the target experts' intellectual stances
    Section 4.1 constructs low-level and high-level persona layers from Google Scholar and YouTube materials; no verification of faithfulness is reported.
  • domain assumption Five HCI panels form a representative testbed for evaluating panel simulation
    Section 5.1 selects CHI/CSCW/UIST panels by relevance and material availability, not by a defined sampling frame, yet conclusions are drawn about general dialogue quality.
  • domain assumption Two generations per configuration suffice to account for LLM stochasticity
    Section 5.1 generates each dialogue twice, but no variance, confidence intervals, or statistical comparison are reported, so stability is assumed.
  • domain assumption The objectives educational_value and belief_alignment in Eq. 2 are computable and meaningful via LLM prompting
    The strategy selection formula in Section 4.1 presupposes that these quantities can be scored by the model during generation without external grounding.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SimuPanel: A Novel Immersive Multi-Agent System to Simulate Interactive Expert Panel Discussion." pith.science (2026). https://pith.science/paper/XN3ZW6VA

@misc{pith2026250616010,
  author       = {Pith},
  title        = {Pith review of: SimuPanel: A Novel Immersive Multi-Agent System to Simulate Interactive Expert Panel Discussion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XN3ZW6VA}},
  note         = {Machine review of arXiv:2506.16010}
}
read the original abstract

Panel discussion allows the audience to learn different perspectives through interactive discussions among experts moderated by a host and a Q&A session with the audience. Despite its benefits, panel discussion in the real world is inaccessible to many who do not have the privilege to participate due to geographical, financial, and time constraints. We present SimuPanel, which simulates panel discussions among academic experts through LLM-based multi-agent interaction. It enables users to define topics of interest for the panel, observe the expert discussion, engage in Q&A, and take notes. SimuPanel employs a host-expert architecture where each panel member is simulated by an agent with specialized expertise, and the panel is visualized in an immersive 3D environment to enhance engagement. Traditional dialogue generation struggles to capture the depth and interactivity of real-world panel discussions. To address this limitation, we propose a novel multi-agent interaction framework that simulates authentic panel dynamics by modeling reasoning strategies and personas of experts grounded in multimedia sources. This framework enables agents to dynamically recall and contribute to the discussion based on past experiences from diverse perspectives. Our technical evaluation and the user study with university students show that SimuPanel was able to simulate more in-depth discussions and engage participants to interact with and reflect on the discussions. As a first step in this direction, we offer design implications for future avenues to improve and harness the power of panel discussion for multimedia learning.

Figures

Figures reproduced from arXiv: 2506.16010 by the authors.

Figure 1
Figure 1. SimuPanel Interface Overview. The system simulates academic panel discussions with interactive expert agents. Users can view the topic poster, follow the live multimodal dialogue, and access a scrollable discussion history. Note-taking is supported via keyboard shortcuts with color-coded labels for quick categorization. Abstract Panel discussion allows the audience to learn different perspectives through interactive… view at source ↗
Figure 2
Figure 2. The figure illustrates the reasoning workflows of the six strategies evaluated in our ablation study. We present [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Summary of user study results evaluating the learn [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

55 extracted references · 23 canonical work pages

  1. [1]

    Philip Anastassiou, Jiawei Chen, Jitong Chen, Yuanzhe Chen, Zhuo Chen, Ziyi Chen, Jian Cong, Lelai Deng, Chuang Ding, Lu Gao, et al . 2024. Seed-tts: A family of high-quality versatile speech generation models.arXiv preprint arXiv:2406.02430(2024)

    arXiv 2024

  2. [2]

    Jinheon Baek, Sujay Kumar Jauhar, Silviu Cucerzan, and Sung Ju Hwang. 2024. Researchagent: Iterative research idea generation over scientific literature with large language models.arXiv preprint arXiv:2404.07738(2024)

    arXiv 2024

  3. [3]

    André Bittermann, Danielle McNamara, Bianca A Simonsmeier, and Michael Schneider. 2023. The landscape of research on prior knowledge and learning: A bibliometric analysis.Educational Psychology Review35, 2 (2023), 58

    work page 2023

  4. [4]

    John Brooke et al. 1996. SUS-A quick and dirty usability scale.Usability evaluation in industry189, 194 (1996), 4–7

    1996

  5. [5]

    Ana Paula Chaves and Marco Aurelio Gerosa. 2021. How should my chatbot interact? A survey on social characteristics in human–chatbot interaction design. International Journal of Human–Computer Interaction37, 8 (2021), 729–758

    2021

  6. [6]

    Jiangjie Chen, Xintao Wang, Rui Xu, Siyu Yuan, Yikai Zhang, Wei Shi, Jian Xie, Shuang Li, Ruihan Yang, Tinghui Zhu, et al. 2024. From Persona to Personal- ization: A Survey on Role-Playing Language Agents.Transactions on Machine Learning Research(2024)

    work page 2024

  7. [7]

    Jiangjie Chen, Xintao Wang, Rui Xu, Siyu Yuan, Yikai Zhang, Wei Shi, Jian Xie, Shuang Li, Ruihan Yang, Tinghui Zhu, et al. 2024. From persona to personaliza- tion: A survey on role-playing language agents.arXiv preprint arXiv:2404.18231 (2024)

    arXiv 2024

  8. [8]

    Konstantina Christakopoulou, Shibl Mourad, and Maja Mataric. [n. d.]. Agents Thinking Fast and Slow: A Talker-Reasoner Architecture. InNeurIPS 2024 Work- shop on Open-World Agents

    work page 2024

Show all 55 references
  1. [9]

    Leigh Clark, Nadia Pantidi, Orla Cooney, Philip Doyle, Diego Garaialde, Justin Edwards, Brendan Spillane, Emer Gilmartin, Christine Murad, Cosmin Munteanu, et al. 2019. What makes a good conversation? Challenges in designing truly conversational agents. InProceedings of the 20...

    2019

  2. [10]

    2006.Cognitive apprenticeship

    Allan Collins and Manu Kapur. 2006.Cognitive apprenticeship. Vol. 291. na

    2006

  3. [11]

    Yihong Dong, Xue Jiang, Zhi Jin, and Ge Li. 2024. Self-collaboration code gener- ation via chatgpt.ACM Transactions on Software Engineering and Methodology 33, 7 (2024), 1–38

    2024

  4. [12]

    Jean-Luc Doumont, Laura Grossenbacher, Christina Matta, and Jorge Cham. 2014. English communication for scientists. (2014)

    2014

  5. [13]

    Yilun Du, Shuang Li, Antonio Torralba, Joshua B Tenenbaum, and Igor Mordatch

  6. [14]

    Hassan A El-Sabagh. 2021. Adaptive e-learning environment based on learning styles and its impact on development students’ engagement.International Journal of Educational Technology in Higher Education18, 1 (2021), 53

    2021

  7. [15]

    2018.The Cambridge handbook of expertise and expert performance

    K Anders Ericsson, Robert R Hoffman, Aaron Kozbelt, and A Mark Williams. 2018.The Cambridge handbook of expertise and expert performance. Cambridge University Press

    2018

  8. [16]

    Juan Fernando Flórez Marulanda, Cesar A Collazos, and Julio Ariel Hurtado. 2023. Evaluating an immersive virtual classroom as an augmented reality platform in synchronous remote learning.Information14, 10 (2023), 543

    2023

  9. [17]

    Thomas Foken, Marc Aubinet, John J Finnigan, Monique Y Leclerc, Matthias Mauder, and Kyaw Tha Paw U. 2011. Results of a panel discussion about the energy balance closure correction for trace gases.Bulletin of the American Meteorological Society92, 4 (2011), ES13–ES18

    2011

  10. [18]

    Jinlan Fu, See-Kiong Ng, Zhengbao Jiang, and Pengfei Liu. 2023. Gptscore: Evaluate as you desire.arXiv preprint arXiv:2302.04166(2023)

    2023 arXiv

  11. [19]

    Stephen A Gallo, Karen B Schmaling, Lisa A Thompson, and Scott R Glisson

  12. [20]

    Ilie Gligorea, Marius Cioca, Romana Oancea, Andra-Teodora Gorski, Hortensia Gorski, and Paul Tudorache. 2023. Adaptive learning using artificial intelligence in e-learning: A literature review.Education Sciences13, 12 (2023), 1216

    2023

  13. [21]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al . 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948(2025)

    2025 arXiv

  14. [22]

    Kai Guo, Yuchun Zhong, Danling Li, and Samuel Kai Wah Chu. 2023. Effects of chatbot-assisted in-class debates on students’ argumentation skills and task motivation.Computers & Education203 (2023), 104862

    2023

  15. [23]

    Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V Chawla, Olaf Wiest, and Xiangliang Zhang. 2024. Large language model based multi-agents: A survey of progress and challenges.arXiv preprint arXiv:2402.01680(2024)

    2024 arXiv

  16. [24]

    Tiancheng Hu and Nigel Collier. 2024. Quantifying the persona effect in llm simulations.arXiv preprint arXiv:2402.10811(2024)

    2024 arXiv

  17. [25]

    Paul Huffman and James Hutson. 2024. Enhancing History Education with Google NotebookLM: Case Study of Mary Easton Sibley’s Diary for Multimedia Content and Podcast Creation.ISRG Journal of Arts, Humanities and Social Sciences2, 5 (2024)

    2024

  18. [26]

    Zhao Kaiya, Michelangelo Naim, Jovana Kondic, Manuel Cortes, Jiaxin Ge, Shuy- ing Luo, Guangyu Robert Yang, and Andrew Ahn. 2023. Lyfe agents: Generative agents for low-cost real-time social interactions.arXiv preprint arXiv:2310.02172 (2023)

    2023 arXiv

  19. [27]

    Long Li, Weiwen Xu, Jiayan Guo, Ruochen Zhao, Xingxuan Li, Yuqian Yuan, Boqiang Zhang, Yuming Jiang, Yifei Xin, Ronghao Dang, et al . 2024. Chain of ideas: Revolutionizing research via novel idea development with llm agents. arXiv preprint arXiv:2410.13185(2024)

    2024 arXiv

  20. [28]

    Xiao Ping Lin, Bin Bin Li, Zhen Ning Yao, Zhi Yang, and Mingshu Zhang. 2024. The impact of virtual reality on student engagement in the classroom–a critical review of the literature.Frontiers in Psychology15 (2024), 1360574

    2024

  21. [29]

    Jieyi Long. 2023. Large language model guided tree-of-thought.arXiv preprint arXiv:2305.08291(2023)

    2023 arXiv

  22. [30]

    Chris Lu, Cong Lu, Robert Tjarko Lange, Jakob Foerster, Jeff Clune, and David Ha

  23. [31]

    Richard E Mayer. 2002. Multimedia learning. InPsychology of learning and motivation. Vol. 41. Elsevier, 85–139

    2002

  24. [32]

    Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. InProceedings of the 36th annual acm symposium on user interface software and technology. 1–22

    2023

  25. [33]

    Joon Sung Park, Lindsay Popowski, Carrie Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. 2022. Social simulacra: Creating populated prototypes for social computing systems. InProceedings of the 35th Annual ACM Symposium on User Interface Software and Technol...

    2022

  26. [34]

    Annie Piolat, Thierry Olive, and Ronald T Kellogg. 2005. Cognitive effort during note taking.Applied cognitive psychology19, 3 (2005), 291–312

    2005

  27. [35]

    Chen Qian, Xin Cong, Cheng Yang, Weize Chen, Yusheng Su, Juyuan Xu, Zhiyuan Liu, and Maosong Sun. 2023. Communicative agents for software development. arXiv preprint arXiv:2307.079246, 3 (2023)

    2023 arXiv

  28. [36]

    Justin Reich and José A Ruipérez-Valiente. 2019. The MOOC pivot.Science363, 6423 (2019), 130–131

    2019

  29. [37]

    Ivan Sekulić, Mohammad Alinannejadi, and Fabio Crestani. 2024. Analysing utterances in llm-based user simulation for conversational search.ACM Trans- actions on Intelligent Systems and Technology15, 3 (2024), 1–22

    2024

  30. [38]

    Murray Shanahan, Kyle McDonell, and Laria Reynolds. 2023. Role play with large language models.Nature623, 7987 (2023), 493–498

    2023

  31. [39]

    Yunfan Shao, Linyang Li, Junqi Dai, and Xipeng Qiu. 2023. Character-llm: A trainable agent for role-playing.arXiv preprint arXiv:2310.10158(2023)

    2023 arXiv

  32. [40]

    Zayne Sprague, Fangcong Yin, Juan Diego Rodriguez, Dongwei Jiang, Manya Wadhwa, Prasann Singhal, Xinyu Zhao, Xi Ye, Kyle Mahowald, and Greg Durrett

  33. [41]

    Lawrence Summers. 1991. Panel discussion: price stability: how should long-term monetary policy be determined?Journal of Money, Credit and banking23, 3 (1991), 625–631

    1991

  34. [42]

    Oguzhan Topsakal and Tahir Cetin Akinci. 2023. Creating large language model applications utilizing langchain: A primer on developing llm apps fast. InInterna- tional Conference on Applied Engineering and Natural Sciences, Vol. 1. 1050–1056

    2023

  35. [43]

    Yu-Min Tseng, Yu-Chao Huang, Teng-Yun Hsiao, Wei-Lin Chen, Chao-Wei Huang, Yu Meng, and Yun-Nung Chen. 2024. Two tales of persona in llms: A survey of role-playing and personalization.arXiv preprint arXiv:2406.01171 (2024)

    2024 arXiv

  36. [44]

    To cot or not to cot? chain-of-thought helps mainly on math and symbolic reasoning.arXiv preprint arXiv:2409.12183(2024)

    2024 arXiv

  37. [45]

    Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. 2023. Voyager: An open-ended embodied agent with large language models.arXiv preprint arXiv:2305.16291(2023)

    2023 arXiv

  38. [46]

    Mo Wang, Daniel J Beal, David Chan, Daniel A Newman, Jeffrey B Vancouver, and Robert J Vandenberg. 2017. Longitudinal research: A panel discussion on conceptual issues, research design, and statistical techniques.Work, Aging and Retirement3, 1 (2017), 1–24

    2017

  39. [47]

    Qian Wang, Jiaying Wu, Zhenheng Tang, Bingqiao Luo, Nuo Chen, Wei Chen, and Bingsheng He. 2025. What Limits LLM-based Human Simulation: LLMs or Our Design?arXiv preprint arXiv:2501.08579(2025)

    2025 arXiv

  40. [48]

    Prokopia Vlachogianni and Nikolaos Tselios. 2022. Perceived usability evaluation of educational technology using the System Usability Scale (SUS): A systematic review.Journal of Research on Technology in Education54, 3 (2022), 392–409. Xiangyang He, Jiale Li, Jiahao Chen, Yang...

    2022

  41. [49]

    Martin Weiss, Nasim Rahaman, Manuel Wuthrich, Yoshua Bengio, Li Erran Li, Bernhard Schölkopf, and Christopher Pal. 2024. Rethinking the buyer’s inspection paradox in information markets with language agents. (2024)

    2024

  42. [50]

    Juncheng Wu, Anushka Rajesh, Yu-Ning Huang, Karishma Chhugani, Rajesh Acharya, Kerui Peng, Ruth D Johnson, Andrada Fiscutean, Carla Daniela Robles- Espinoza, Francisco M De La Vega, et al . 2022. Virtual meetings promise to eliminate geographical and administrative barriers an...

    2022

  43. [51]

    Yuzhuang Xu, Shuo Wang, Peng Li, Fuwen Luo, Xiaolong Wang, Weidong Liu, and Yang Liu. 2023. Exploring large language models for communication games: An empirical study on werewolf.arXiv preprint arXiv:2309.04658(2023)

    2023 arXiv

  44. [52]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reason- ing in large language models.Advances in neural information processing systems 35 (2022), 24824–24837

    2022

  45. [2020]

    Grant reviewer perceptions of the quality, effectiveness, and influence of panel discussion.Research Integrity and Peer Review5 (2020), 1–9

    2020

  46. [2023]

    InForty-first International Conference on Machine Learning

    Improving factuality and reasoning in language models through multiagent debate. InForty-first International Conference on Machine Learning

  47. [2024]

    arXiv preprint arXiv:2408.06292(2024)

    The ai scientist: Towards fully automated open-ended scientific discovery. arXiv preprint arXiv:2408.06292(2024)

    2024 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.