Pith. sign in

REVIEW 4 major objections 7 minor 52 references

A Dynamic and High-Precision Method for Scenario-Based HRA Synthetic Data Collection in Multi-Agent Collaborative Environments Driven by LLMs

T0 review · 4 major / 7 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that a fine-tuned LLM can predict the workload of individual control-room roles more accurately than GPT-4, GPT-4o, and Claude-3.5-Sonnet, making automated scenario-driven synthetic HRA data collection feasible.

desk verdict Plausible application with a broken evaluation: no held-out split, a duplicated table, and weak baselines; the core idea is worth a revision, not a rejection of the direction. read the letter →

arxiv 2502.00022 v1 pith:JDDH2HKJ submitted 2025-01-16 cs.AI cs.HC

Xingyu Xiao , Peng Chen , Qianqian Jia , Jiejuan Tong , Jingang Liang , Haitao Wang This is my paper · ORCID
classification cs.AIcs.HC
keywords DynamicWorkloadPredictionLargeLanguageModelsNASATLXSARTsyntheticHRAdatagenerationmethodPersonnelSituationalAwarenessScenario-DrivenAssessment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that supervised fine-tuning of an open-weight language model on real operator survey data and LLM-generated cognitive reasoning chains produces workload estimates accurate enough to serve as synthetic human-reliability-analysis data. The resulting system, WELLA, is evaluated against three commercial LLMs on five operator roles in a high-temperature gas-cooled reactor control room, and the paper reports that WELLA is the only model with positive $R^2$ and explained-variance values across scenarios. If the claim is right, workload labels for new collaborative scenarios could be collected automatically, without expert input or post-hoc questionnaires, addressing a long-standing data-scarcity problem in human reliability analysis. The practical payoff would be a dynamic, role-specific, and auditable source of workload and situational-awareness data for safety training and system design.

What carries the argument

The load-bearing mechanism is the fine-tuned language model that maps a scenario description to workload and situational-awareness scores through an intermediate 'virtual cognitive trajectory.' Real control-room operators first fill out NASA-TLX (a six-dimension subjective workload scale) and SART (a ten-item situational-awareness rating scale) after simulator runs; the commercial model Claude is then used to rewrite those responses as macro-cognitive reasoning chains; and the open-weight model Qwen2.5-7B is fine-tuned on that library with the Llama-factory toolkit. At inference time, five role-specific agents (SO, RO1, RO2, RO3, CO) read a new scenario and emit a cognitive trajectory plus the questionnaire scores, which is what makes the workload estimate dynamic, role-specific, and inspectable.

What would settle it

Hold out a random set of scenarios—say 20% of the 69 collected—before building the cognitive trajectory library, fine-tune WELLA only on the remaining scenarios, and compute $R^2$, RMSE, MAE, and explained variance on the held-out set. If the held-out $R^2$ drops toward zero or negative, the central claim of prediction accuracy is refuted and the reported $R^2 \approx 0.90$ is attributable to memorization of the training scenarios.

Watch

Extended reading notes

Core claim

The paper's central claim is that a fine-tuned Qwen2.5-7B model—trained on a 'virtual cognitive trajectory library' built by having Claude expand real operators' NASA-TLX and SART responses into five-stage macro-cognitive narratives (detection, understanding, decision-making, action execution, inter-team coordination)—can predict operator workload from scenario text more accurately than GPT-4, GPT-4o, and Claude-3.5-Sonnet. In the reported experiments, WELLA achieves $R^2 = 0.9012$, $0.9343$, and $0.9628$ for the three reactor operators, with lower RMSE and MAE than all baselines; the commercial models mostly show negative $R^2$, meaning their predictions are worse than simply predicting the mean. The paper further claims that this method is fully automated, captures dynamic changes in workload across scenarios, and avoids the expert-input burden of the existing HUNTER simulator, thereby opening a new paradigm for scenario-based synthetic HRA data collection.

Load-bearing premise

The load-bearing premise is that the reported accuracy metrics describe prediction of scenarios not seen during fine-tuning; the paper never reports a train/test split, so if the evaluation scenarios overlap with the SFT training library, the high $R^2$ may be memorization rather than prediction.

Editorial extensions

If this is right

  • If the central claim holds, workload labels for new high-temperature gas-cooled reactor scenarios can be generated automatically from scenario text, removing the need for expert knowledge and post-hoc surveys.
  • The same training recipe—survey labels, LLM-generated cognitive trajectories, and supervised fine-tuning—could be transplanted to other multi-agent high-risk domains such as aviation, healthcare, or chemical plants.
  • Because the model emits an intermediate cognitive trajectory alongside each score, the resulting synthetic HRA data carries a transparent reasoning record that can be audited for plausibility.
  • WELLA's additional situational-awareness output gives synthetic HRA datasets a dynamic indicator of risk perception, not just a static workload number.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's own comparison table marks WELLA as 'No' on generalizability, which is at odds with the abstract's claim of adapting to 'various collaborative scenarios'; the most plausible reading is that the high accuracy reflects overlap between the fine-tuning library and the evaluation scenarios rather than transfer to unseen situations.
  • A decisive extension would be to fine-tune on a subset of roles or reactor modules and predict the held-out roles, testing whether the cognitive-trajectory representation generalizes across job functions rather than merely memorizing per-role phrasing.
  • The comparison against commercial LLMs as zero-shot baselines necessarily favours the fine-tuned model; an apples-to-apples baseline would be an open-weight model fine-tuned on the same library without the trajectory augmentation, isolating the contribution of the macro-cognitive intermediate step.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes WELLA, a fine-tuned Qwen2.5-7B model that estimates operator workload (NASA-TLX) and situational awareness (SART-derived SA) in a multi-agent HTGR main-control-room setting. The pipeline consists of collecting real operator workload data on a simulator testbed, using Claude to generate 'virtual cognitive trajectories' based on macrocognitive theory, fine-tuning Qwen2.5-7B on that trajectory library via Llama-Factory, and then comparing WELLA against zero-shot GPT-4, GPT-4o, and Claude-3.5-Sonnet on per-role regression metrics (R², RMSE, MAE, EV). The authors report that WELLA outperforms the commercial baselines, particularly for RO1/RO2/RO3 (R² 0.90–0.96), with weaker results for CO and SO (R² 0.42 and 0.38). The central claim is that this constitutes a high-precision, dynamic, scenario-driven method for synthetic HRA workload data collection.

Significance. The motivation is solid: HRA data scarcity and the static nature of survey-based workload measurements are real problems, and the use of a real HTGR simulator testbed with actual operators is a strength. The idea of grounding fine-tuning data in macrocognitive trajectories is also interesting and could be valuable for dynamic HRA. However, the current evaluation does not establish out-of-sample prediction accuracy: no train/test split is reported, the commercial baselines are not fine-tuned, and Table 6 is numerically identical to Table 5, which invalidates the aggregate comparison. If a proper held-out evaluation were provided, the approach could be relevant to HRA and human-factors communities, but the evidence in this version does not support the paper's headline claim. No code or data are provided to aid reproducibility.

major comments (4)
  1. [§3.3–3.4, §4.2] The paper does not report any train/validation/test split: Section 3.4 fine-tunes Qwen2.5-7B on a cognitive-trajectory library constructed in Section 3.3 from the same real operator workload data collected in Section 3.2, and Section 4.2 evaluates on the same scenario suite (Startup, Shutdown, Accident with 28, 11, and 30 instances). Without a stated held-out set or an overlap audit, the high R² values in Tables 1–5 can be explained by the model memorizing training labels, and the abstract's claim of 'prediction accuracy' is unsupported. A scenario-level holdout or an explicit non-overlap analysis is required.
  2. [Table 6 vs Table 5] Table 6 (ALL data) is numerically identical to Table 5 (SO) for every row and every metric, even though the text describes Table 6 as the combined dataset. This is impossible for a proper aggregation unless the ALL dataset coincides with the SO subset; the discrepancy invalidates the aggregate comparison and must be corrected with the actual combined metrics.
  3. [§4.2, Tables 1–6] The evaluation compares fine-tuned WELLA with zero-shot commercial models (GPT-4, GPT-4o, Claude-3.5-Sonnet). This is not an apples-to-apples comparison: any advantage could come simply from exposure to real operator labels during fine-tuning. The authors should add few-shot baselines for the commercial models or a same-data fine-tuned open-source baseline to support the claim of superior prediction accuracy.
  4. [§4.2, Tables 1–6] The evaluation reports point estimates without error bars, confidence intervals, or significance tests, and the per-role sample sizes are small (11–30). For example, the CO and SO results (R² 0.42 and 0.38) might not be distinguishable from the commercial baselines under resampling. Bootstrap intervals or paired tests should be reported before drawing comparative conclusions.
minor comments (7)
  1. [§3.3] The section title 'Vitural cognitive trajectory Generation' contains a typo: 'Vitural' should be 'Virtual'.
  2. [§3.2] Equation references point to 'equation 3.2' before the equations are numbered; the equations should be numbered and referenced consistently.
  3. [§4.1] The text says 'Detailed implementation can be found in ??'; this unresolved cross-reference should be replaced with an actual appendix or hyperparameter table.
  4. [§4.4, Figure 7] The model is referred to as 'VELLA' in Section 4.4 and Figure 7; this should be 'WELLA'.
  5. [§4.2] The sentence 'The evaluation is conducted for five scenarios: RO1, RO2, RO3, CO, SO, and the aggregated dataset (ALL)' lists six items; rephrase to clarify the five per-role scenarios plus the pooled dataset.
  6. [§3.2] The SART items are said to be detailed in Appendix D, but no appendix appears in the manuscript; provide the appendix or remove the reference.
  7. [Keywords and §4.2] The keywords contain a double comma in 'generation method,, Personnel Workload', and the phrase 'currently unavailable commercial models' in Section 4.2 should be reworded.

Circularity Check

1 steps flagged · score 6.0 of 10

The central 'prediction accuracy' claim is not supported by a held-out evaluation: WELLA is fine-tuned on a virtual trajectory library derived from the same real operator workload data against which it is scored, so the reported R2 values can reduce to in-sample fit.

  1. fitted input called prediction [The relevant sections are 3.2, 3.4, and 4.2; no train/test split is described.]
    "After each simulation, operators were asked to complete the NASA Task Load Index (TLX) ... which were then used to gather performance and workload data. ... Using the Llama-factory framework, we performed supervised fine-tuning (SFT) on our cognitive trajectory library data. ... The evaluation is conducted for five scenarios: RO1, RO2, RO3, CO, SO, and the aggregated dataset (ALL)."

    The real operator workload data collected in Section 3.2 is used to construct the cognitive trajectory library on which WELLA is fine-tuned in Sections 3.3-3.4, and Section 4.2 reports R2/RMSE/MAE/EV against the same five-role scenarios with no held-out split. Therefore the reported metrics are in-sample fits, not out-of-sample predictions; a fine-tuned model reproducing its training labels would produce exactly the reported behavior. The abstract's phrase 'prediction accuracy' renames this in-sample fit as prediction, closing the loop by construction.

full rationale

The derivation chain as written is closed: real operator workload data, Claude-generated virtual cognitive trajectories, SFT of Qwen2.5-7B, and evaluation against the same real operator data. Because the paper never reports a train/validation/test split, the positive R2 values in Tables 1-5 are consistent with the model memorizing its fine-tuning targets and do not demonstrate generalization. Independent evidence that the evaluation block is unreliable is that Table 6 ('ALL data') is numerically identical to Table 5 ('SO'), so the aggregate claim is not actually computed from all roles. The paper also cites the authors' own prior work [1] for the premise that a cognitive trajectory exists, but that self-citation is not the main source of circularity; the missing held-out evaluation is. The comparison against zero-shot GPT-4, GPT-4o, and Claude-3.5 is not apples-to-apples, but that is a benchmarking concern rather than a circularity.

Assumptions & free parameters 1 free parameters · 3 assumptions · 1 invented entities

The ledger shows the central claim rests on an unvalidated ground-truth formula, an unvalidated intermediate representation generated by another LLM, and an unstated train/test separation. No fitted physical constants or external benchmark anchors the result; the only numeric inputs are the TLX weights and the hyperparameters of the SFT run.

free parameters (1)
  • Equal weights in unweighted NASA-TLX workload composite (Eq. 1) = 1/6 for each of the six dimensions
    The ground-truth workload score used in all evaluation tables is computed with hand-chosen equal weights rather than the standard weighted TLX procedure or a fit to physiological measures. All model accuracy numbers depend on this composite.
assumptions (3)
  • domain assumption The unweighted NASA-TLX composite (Eq. 1) is a valid ground-truth measure of operator workload.
    Section 3.2 defines workload with this formula and every reported accuracy number is measured against it; its validity is inherited from the TLX literature, not re-established.
  • ad hoc to paper Claude-generated virtual cognitive trajectories faithfully represent real operator cognition.
    Section 3.3 generates trajectories with Claude but provides no validation against actual cognitive processes, inter-rater agreement, or behavioral data.
  • ad hoc to paper The evaluation scenarios are disjoint from the scenarios used to build the SFT training data.
    No train/test split is stated anywhere in Section 3.4 or 4.2, yet the reported R2 values assume the model generalizes to unseen scenarios.
invented entities (1)
  • Virtual cognitive trajectory library
    purpose: Intermediate representation generated by Claude and used as supervised fine-tuning data to teach WELLA to output workload scores from scenario descriptions.
    No evidence is given that these trajectories correspond to real operator cognition; they are generated from the same real data that defines the prediction targets, so they are a speculative construct with no external handle.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Dynamic and High-Precision Method for Scenario-Based HRA Synthetic Data Collection in Multi-Agent Collaborative Environments Driven by LLMs." pith.science (2026). https://pith.science/paper/JDDH2HKJ

@misc{pith2026250200022,
  author       = {Pith},
  title        = {Pith review of: A Dynamic and High-Precision Method for Scenario-Based HRA Synthetic Data Collection in Multi-Agent Collaborative Environments Driven by LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JDDH2HKJ}},
  note         = {Machine review of arXiv:2502.00022}
}
read the original abstract

HRA (Human Reliability Analysis) data is crucial for advancing HRA methodologies. however, existing data collection methods lack the necessary granularity, and most approaches fail to capture dynamic features. Additionally, many methods require expert knowledge as input, making them time-consuming and labor-intensive. To address these challenges, we propose a new paradigm for the automated collection of HRA data. Our approach focuses on key indicators behind human error, specifically measuring workload in collaborative settings. This study introduces a novel, scenario-driven method for workload estimation, leveraging fine-tuned large language models (LLMs). By training LLMs on real-world operational data from high-temperature gas-cooled reactors (HTGRs), we simulate human behavior and cognitive load in real time across various collaborative scenarios. The method dynamically adapts to changes in operator workload, providing more accurate, flexible, and scalable workload estimates. The results demonstrate that the proposed WELLA (Workload Estimation with LLMs and Agents) outperforms existing commercial LLM-based methods in terms of prediction accuracy.

Figures

Figures reproduced from arXiv: 2502.00022 by the authors.

Figure 1
Figure 1. IDHEAS-ECA HRA Process 3.2. Real-World Data Collection Next, we examine the main control room of a multi-module High-Temperature Gas-Cooled Reactor (HTGR) nuclear power plant. In this setting, the main control room operates under the one operator controls two or more reactors. The control room is staffed by three reactor operators (RO1, RO2, RO3), one secondary loop operator (CO), and one shift supervisor (SO). RO1 … view at source ↗
Figure 2
Figure 2. The High-Temperature Reactor Simulator Operating Test Bench [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. A simplified model of macrocognitive functions. [8] [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: WELLA: Input-Output Dynamics in Scenario Analysis [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: Performance Comparison of WELLA, GPT-4, GPT-4O, and Claude Across [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: Step Module Execution Example Using the Instruction Step [49]. [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: The predictive analysis results of WELLA for RO1 in a specific scenario.. [PITH_FULL_IMAGE:figures/full_fig_p020_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

52 extracted references · 42 canonical work pages

  1. [1]

    X. Xiao, P. Chen, B. Qi, H. Zhao, J. Liang, J. Tong, H. Wang, Krail: A knowledge-driven framework for base human reliability analysis integrat- ing idheas and large language models, arXiv preprint arXiv:2412.18627 (2024)

    arXiv 2024

  2. [2]

    Y. Yu, S. Wu, Y. Fu, X. Liu, Q. Zeng, H. Ding, Y. Pan, Y. Wu, H. Guo, Y. Yang, Human reliability analysis of offshore high integrity pressure protection system based on improved cream and hcr integration method, Ocean Engineering 307 (2024) 118153

    work page 2024

  3. [3]

    Gertman, H

    D. Gertman, H. Blackman, J. Marble, J. Byers, C. Smith, et al., The spar-h human reliability analysis method, US Nuclear Regulatory Com- mission 230 (4) (2005) 35

    2005

  4. [4]

    Hollnagel, Cognitive reliability and error analysis method (CREAM), Elsevier, 1998

    E. Hollnagel, Cognitive reliability and error analysis method (CREAM), Elsevier, 1998

    work page 1998

  5. [5]

    J. Xing, Y. Chang, J. DeJesus, Integrated human event analysis system for event and condition assessment (idheas-eca), US Nuclear Regulatory Commission, Washington, DC (2020). 21

    work page 2020

  6. [6]

    J. Xing, Y. CHANG, J. SEGARRA, Draft—integrated human event analysis system for human reliability data (idheas-data), RIL-2021-XX (2021)

    work page 2021

  7. [7]

    W. Jung, J. Park, Y. Kim, S. Y. Choi, S. Kim, Hurex–a framework of hra data collection from simulators in nuclear power plants, Reliability Engineering & System Safety 194 (2020) 106235

    work page 2020

  8. [8]

    Y. J. Chang, D. Bley, L. Criscione, B. Kirwan, A. Mosleh, T. Madary, R. Nowell, R. Richards, E. M. Roth, S. Sieben, et al., The sacada database for human reliability and human performance, Reliability En- gineering & System Safety 125 (2014) 117–133

    work page 2014

Show all 52 references
  1. [9]

    Boring, M

    R. Boring, M. Rasmussen, Goms-hra: A method for treating subtasks in dynamic human reliability analysis, in: Proceedings of the 2016 Eu- ropean Safety and Reliability Conference, 2016, pp. 956–963

    2016

  2. [10]

    R. L. Boring, T. A. Ulrich, R. Lew, The procedure performance predic- tor (p3): Application of the hunter dynamic human reliability analysis software to inform the development of new procedures (2023)

    2023

  3. [11]

    X. Xiao, J. Liang, J. Tong, H. Wang, Emergency decision support tech- niques for nuclear power plants: Current state, challenges, and future trends, Energies 17 (10) (2024) 2439

    2024

  4. [12]

    R. L. Boring, C. D. Griffith, J. C. Joe, The measure of human error: Direct and indirect performance shaping factors, in: 2007 IEEE 8th Human Factors and Power Plants and HPRCT 13th Annual Meeting, IEEE, 2007, pp. 170–176

    2007

  5. [13]

    Chamon, E

    M. Chamon, E. Kaplan, The iceberg theory of campaign contributions: Political threats and interest group behavior, American Economic Jour- nal: Economic Policy (2013) 1–31

    2013

  6. [14]

    Wright, Towards an empirical test of the iceberg model, Hum

    L. Wright, Towards an empirical test of the iceberg model, Hum. Decis. Mak. Man. Control. EAM2000 EUR 19599 (2000) 145–152

    2000

  7. [15]

    Domurath, M

    J. Domurath, M. Saphiannikova, J. F´ erec, G. Ausias, G. Heinrich, Stress and strain amplification in a dilute suspension of spherical particles based on a bird–carreau model, Journal of Non-Newtonian Fluid Me- chanics 221 (2015) 95–102. 22

    2015

  8. [16]

    B. H. Kantowitz, P. A. Casper, Human workload in aviation, in: Human error in aviation, Routledge, 2017, pp. 123–153

    2017

  9. [17]

    Jou, T.-C

    Y.-T. Jou, T.-C. Yenn, C. J. Lin, C.-W. Yang, C.-C. Chiang, Evaluation of operators’ mental workload of human–system interface automation in the advanced nuclear power plants, Nuclear Engineering and Design 239 (11) (2009) 2537–2542

    2009

  10. [18]

    Bagheri, S

    M. Bagheri, S. D. Power, Eeg-based detection of mental workload level and stress: the effect of variation in each state on classification of the other, Journal of Neural Engineering 17 (5) (2020) 056015

    2020

  11. [19]

    Brouwer, M

    A.-M. Brouwer, M. A. Hogervorst, J. B. Van Erp, T. Heffelaar, P. H. Zimmerman, R. Oostenveld, Estimating workload using eeg spectral power and erps in the n-back task, Journal of neural engineering 9 (4) (2012) 045008

    2012

  12. [20]

    Hassan, M

    J. Hassan, M. S. Reza, S. U. Ahmed, N. H. Anik, M. O. Khan, Eeg workload estimation and classification: a systematic review, Journal of Neural Engineering (2024)

    2024

  13. [21]

    Delliaux, A

    S. Delliaux, A. Delaforge, J.-C. Deharo, G. Chaumet, Mental workload alters heart rate variability, lowering non-linear dynamics, Frontiers in physiology 10 (2019) 565

    2019

  14. [22]

    Marquart, C

    G. Marquart, C. Cabrall, J. De Winter, Review of eye-related measures of drivers’ mental workload, Procedia Manufacturing 3 (2015) 2854– 2861

    2015

  15. [23]

    L. J. Planke, A. Gardi, R. Sabatini, T. Kistan, N. Ezer, Online multi- modal inference of mental workload for cognitive human machine sys- tems, Computers 10 (6) (2021) 81

    2021

  16. [24]

    Y. Xing, C. Lv, D. Cao, H. Wang, Y. Zhao, Driver workload estima- tion using a novel hybrid method of error reduction ratio causality and support vector machine, Measurement 114 (2018) 390–397

    2018

  17. [25]

    R. W. Schvaneveldt, G. B. Reid, R. Gomez, S. Rice, Modeling mental workload, Cognitive Technology 3 (1) (1998) 19–31. 23

    1998

  18. [26]

    F. E. Ritter, F. Tehranchi, J. D. Oury, Act-r: A cognitive architecture for modeling cognition, Wiley Interdisciplinary Reviews: Cognitive Science 10 (3) (2019) e1488

    2019

  19. [27]

    Y. Liu, R. Feyen, O. Tsimhoni, Queueing network-model human proces- sor (qn-mhp) a computational architecture for multitask performance in human-machine systems, ACM Transactions on Computer-Human In- teraction (TOCHI) 13 (1) (2006) 37–70

    2006

  20. [28]

    Paxion, E

    J. Paxion, E. Galy, C. Berthelon, Mental workload and driving, Frontiers in psychology 5 (2014) 1344

    2014

  21. [29]

    Longo, Experienced mental workload, perception of usability, their interaction and impact on task performance, PloS one 13 (8) (2018) e0199661

    L. Longo, Experienced mental workload, perception of usability, their interaction and impact on task performance, PloS one 13 (8) (2018) e0199661

    2018

  22. [30]

    Dehais, A

    F. Dehais, A. Lafont, R. Roy, S. Fairclough, A neuroergonomics ap- proach to mental workload, engagement and human performance, Fron- tiers in neuroscience 14 (2020) 268

    2020

  23. [31]

    K. Wang, Y. Lu, M. Santacroce, Y. Gong, C. Zhang, et al., Adapting llm agents with universal feedback in communication, in: ICML 2024 Workshop on Foundation Models in the Wild, 2024

    2024

  24. [32]

    C. Xie, C. Chen, F. Jia, Z. Ye, K. Shu, A. Bibi, Z. Hu, P. Torr, B. Ghanem, G. Li, Can large language model agents simulate human trust behaviors?, arXiv preprint arXiv:2402.04559 (2024)

    2024 arXiv

  25. [33]

    Sreedhar, L

    K. Sreedhar, L. Chilton, Simulating strategic reasoning: Comparing the ability of single llms and multi-agent systems to replicate human behavior (2025)

    2025

  26. [34]

    Z. Wu, R. Peng, S. Zheng, Q. Liu, X. Han, B. Kwon, M. Onizuka, S. Tang, C. Xiao, Shall we team up: Exploring spontaneous cooperation of competing llm agents, in: Findings of the Association for Computa- tional Linguistics: EMNLP 2024, 2024, pp. 5163–5186

    2024

  27. [35]

    Tjuatja, V

    L. Tjuatja, V. Chen, T. Wu, A. Talwalkwar, G. Neubig, Do llms exhibit human-like response biases? a case study in survey design, Transactions of the Association for Computational Linguistics 12 (2024) 1011–1026. 24

    2024

  28. [36]

    X. Xiao, S. Liu, Z. Zuo, P. Chen, B. Qi, J. Liang, J. Tong, A text intelligence-based approach for automatic generation of fault trees in nuclear power plants, in: International Conference on Nuclear Engi- neering, Vol. 88308, American Society of Mechanical Engineers, 2024, p....

    2024

  29. [37]

    Zhang, Z

    Z. Zhang, Z. Wu, Y. Sun, F. Li, Design aspects of the chinese modular high-temperature gas-cooled reactor htr-pm, Nuclear Engineering and Design 236 (5-6) (2006) 485–490

    2006

  30. [38]

    S. G. Hart, Nasa task load index (tlx) (1986)

    1986

  31. [39]

    R. M. Taylor, Situational awareness rating technique (sart): The devel- opment of a tool for aircrew systems design, in: Situational awareness, Routledge, 2017, pp. 111–128

    2017

  32. [40]

    S. G. Hart, Nasa-task load index (nasa-tlx); 20 years later, in: Pro- ceedings of the human factors and ergonomics society annual meeting, Vol. 50, Sage publications Sage CA: Los Angeles, CA, 2006, pp. 904–908

    2006

  33. [41]

    P. Strydom, The cognitive and metacognitive dimensions of social and political theory, in: Routledge International Handbook of Contemporary Social and Political Theory, Routledge, 2011, pp. 328–338

    2011

  34. [42]

    G. Dong, H. Yuan, K. Lu, C. Li, M. Xue, D. Liu, W. Wang, Z. Yuan, C. Zhou, J. Zhou, How abilities in large language models are affected by supervised fine-tuning data composition, arXiv preprint arXiv:2310.05492 (2023)

    2023 arXiv

  35. [43]

    A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang, H. Wei, et al., Qwen2. 5 technical report, arXiv preprint arXiv:2412.15115 (2024)

    2024 arXiv

  36. [44]

    Zheng, R

    Y. Zheng, R. Zhang, J. Zhang, Y. Ye, Z. Luo, Z. Feng, Y. Ma, Lla- mafactory: Unified efficient fine-tuning of 100+ language models, arXiv preprint arXiv:2403.13372 (2024)

    2024 arXiv

  37. [45]

    X. Liu, J. Zhang, H. Zhang, F. Xue, Y. You, Hierarchical dialogue un- derstanding with special tokens and turn-level attention, arXiv preprint arXiv:2305.00262 (2023). 25

    2023 arXiv

  38. [46]

    Achiam, S

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al., Gpt-4 tech- nical report, arXiv preprint arXiv:2303.08774 (2023)

    2023 arXiv

  39. [47]

    Shahriar, B

    S. Shahriar, B. D. Lund, N. R. Mannuru, M. A. Arshad, K. Hayawi, R. V. K. Bevara, A. Mannuru, L. Batool, Putting gpt-4o to the sword: A comprehensive evaluation of language, vision, speech, and multimodal proficiency, Applied Sciences 14 (17) (2024) 7782

    2024

  40. [48]

    X. Xiao, P. Chen, A hybrid real-time framework for efficient fussell- vesely importance evaluation using virtual fault trees and graph neural networks, arXiv preprint arXiv:2412.10484 (2024)

    2024 arXiv

  41. [49]

    Ulrich, R

    T. Ulrich, R. Boring, J. Park, Y. Heo, J. Ahn, The hunter dynamic human reliability analysis tool: Procedurally driven operator simulation, in: 16th International Conference on Probabilistic Safety Assessment and Management, PSAM 2022, 2022

    2022

  42. [50]

    Preece, Interaction issues for special applications, in: The Human- Computer Interaction Handbook, CRC Press, 2002, pp

    J. Preece, Interaction issues for special applications, in: The Human- Computer Interaction Handbook, CRC Press, 2002, pp. 555–784

    2002

  43. [51]

    P. Chen, X. Wei, Q. Wuwu, X. Wang, X. Xiao, M. Lu, Mixedgaussiana- vatar: Realistically and geometrically accurate head avatar via mixed 2d-3d gaussian splatting, arXiv preprint arXiv:2412.04955 (2024)

    2024

  44. [52]

    P. Chen, X. Wei, M. Lu, Y. Zhu, N. Yao, X. Xiao, H. Chen, Diffu- siontalker: Personalization and acceleration for speech-driven 3d face diffuser, arXiv preprint arXiv:2311.16565 (2023). 26

    2023 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.