Pith. sign in

REVIEW 1 major objections 1 minor 43 references

Can AI Extract Antecedent Factors of Human Trust in AI? An Application of Information Extraction for Scientific Literature in Behavioural and Computer Sciences

T0 review · 1 major / 1 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Prompt-based LLMs cannot yet replace supervised learning for extracting trust-in-AI factors from scientific papers, a new benchmark dataset shows.

desk verdict The dataset is a useful new resource, but the benchmark's test set was used to develop the annotation guidelines, so the headline claim about supervised vs. prompt-based LLMs is not supported as is. read the letter →

arxiv 2412.11344 v1 pith:ESWNYZEZ submitted 2024-12-16 cs.CL cs.AI

Melanie McGrath , Harrison Bailey , Necva Bölücü , Xiang Dai , Sarvnaz Karimi , Cecile Paris This is my paper · ORCID
classification cs.CLcs.AI
keywords trustinAIinformationextractionnamedentityrecognitionrelationlargelanguagemodelsscientificliteratureannotationfew-shotlearningdatasetbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether large language models can extract the factors that drive human trust in AI from the scientific literature, and answers that, at present, they cannot do so reliably through prompting alone. To test this, the authors build a new annotated dataset, called Trust in AI, covering 27 English publications, and define two extraction tasks: recognizing mentions of applications and of human, technological, and contextual trust factors, and determining how those factors relate to the concept of trust. They benchmark supervised models and prompt-based LLMs, finding that supervised models clearly outperform zero- and few-shot LLMs on both tasks. Their central claim is that this problem requires supervised learning, and that prompt-based LLMs are not yet a feasible substitute for expert annotation in this domain. The dataset is meant to be the first step toward structured, queryable knowledge of trust antecedents for researchers building and evaluating AI systems.

What carries the argument

The central object is the Trust in AI dataset and its annotation scheme. Each sentence is paired with its surrounding paragraph as context; entity spans are labeled as application, human factor, technology factor, or context factor; and each relation connects one factor (or a pair of factors) to the concept of trust, which itself is not a mention in the text, so the relation is to a hidden concept. The evaluation is carried by the dataset's split into training, development, and test sets, by a two-stage relation extraction design (first decide whether a relation exists, then classify its type), and by the contrast between supervised fine-tuning and prompt-based LLM inference. The paper's argument stands on the measured gap between these two families of methods on the same annotation scheme.

What would settle it

Re-annotate the test set's relation labels with access to the full article rather than only the sentence and paragraph; if a substantial share of labels change, the dataset's gold relations do not stand. Alternatively, run the same zero-shot GPT-4 prompt with the full article as context; if it matches or beats RoBERTa-Large's 81.7 ROC-AUC on relation identification, the paper's conclusion that prompt-based LLMs are not currently feasible for this problem would be overturned.

Watch

Extended reading notes

Core claim

The paper's central discovery is a negative empirical result with a new resource attached. It constructs the Trust in AI dataset, which annotates 27 English scientific papers at the sentence level with application names and trust-factor mentions (human, technology, context), plus relations linking one or two factors to the hidden concept of trust, labeled unspecified, null, positive, negative, interaction, or no relation. On this dataset, supervised fine-tuned models such as RoBERTa reach substantially higher scores than prompt-based LLMs: for relation identification, RoBERTa-Large reaches a ROC-AUC of 81.7 while zero-shot GPT-4 reaches 52.2, near the random baseline of 50.5, and few-shot in-context learning for entity recognition reaches only 18.3 micro F1 against 61.2 for RoBERTa. The authors interpret this as evidence that extracting trust antecedents from scientific text is a genuinely hard structured-prediction problem, not one that current prompting techniques can solve with a handful of examples.

Load-bearing premise

The gold relations are annotated under the assumption that a factor mention's relationship to trust can be judged reliably from a single sentence and its paragraph, even when the word 'trust' never appears, and the initial 32.3% agreement between annotators on relations shows that judgment is not straightforward.

Editorial extensions

If this is right

  • A practical consequence for behavioural and computer science teams is that building a structured database of trust antecedents from the literature currently requires a human-annotated training corpus, not just a prompt.
  • Because the dataset fixes the task definitions, future supervised models can be compared against the reported baselines, and the claim that prompts are insufficient becomes testable.
  • Relation labels are highly imbalanced and mostly 'no relation', so any practical extraction system needs a reliable relation-identification step before type classification, which is the direct reason the paper adopts its two-step design.
  • The low LLM-human agreement (Cohen's kappa 12.85 for NER and 5.59 for RE) implies that using LLMs to pre-annotate in this domain would still require substantial manual correction.
  • Supervised models themselves remain weak on human and contextual factor types, so the annotation scheme exposes a hard subproblem that is not yet solved even with fine-tuning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves open whether full-document context rather than sentence-plus-paragraph would let prompt-based LLMs close the gap; a testable extension is to prompt with the complete article and the annotation guideline.
  • The relation-to-hidden-trust formulation could generalize to other latent constructs in scientific text, such as risk, fairness, or usability, where the target concept is discussed without being named in every sentence.
  • If supervised learning remains necessary, the cost of annotation in this domain argues for active-learning or human-in-the-loop pipelines that focus annotation effort on the sentences a model is least certain about; this follows from the paper's findings but is not proposed in the paper.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

1 major / 1 minor

Summary. The paper introduces Trust in AI, a new English annotated dataset of scientific publications on antecedents of human trust in AI, and formulates two information extraction tasks: named entity recognition of factor and application spans, and relation extraction between factor mentions and the hidden concept of trust. The authors describe a multi-phase annotation process with two annotators and LLM-guided annotation, and benchmark supervised models (RoBERTa, Seq2seq-BERT, BiaffineNER) against prompt-based LLMs (GPT-3.5 for NER, GPT-4 for RE). Based on these benchmarks, the abstract concludes that this extraction problem requires supervised learning and may not currently be feasible with prompt-based LLMs.

Significance. If the benchmark were valid, the paper would provide a useful new resource for the trust-in-AI community and an informative comparison of supervised and prompt-based IE methods in a specialized scientific domain. The dataset itself, with expert-informed annotation guidelines, is a potentially valuable contribution, and the authors are transparent about the annotation procedure and the LLM-guided component. However, the central empirical claim rests on a test set that is not held out from the guideline-development process, which undermines the benchmark conclusion as currently presented. The dataset release (on request) and the detailed baselines are strengths that could make this a solid contribution after the evaluation is made sound.

major comments (1)
  1. [Appendix B (Experimental Setups)] For the NER few-shot experiments, the manuscript says "we adopt the prompt template provided by EasyInstruct" but does not give the exact prompt for the few-shot setting beyond the zero-shot instruction. The prompt shown appears to be for the NER task with a single sentence, while the ICL experiments use various sample selection methods (BM25, KATE, etc.). Please include an example of the full few-shot prompt (with demonstrations) to make the experiments reproducible.
minor comments (1)
  1. [Table 4] Please clarify the caption of Table 4 by spelling out the relation-type order in the bracketed distributions.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is an empirical benchmark whose central claim rests on direct experimental comparison, not on a self-citation chain or on a fitted parameter renamed as a prediction.

full rationale

The paper is an empirical benchmark and dataset-construction study, not a derivation that claims to predict a quantity from a fitted model. The central conclusion—that supervised learning outperforms prompt-based LLMs for trust-in-AI information extraction—is supported by direct measurements reported in Tables 2 and 3, with models trained on the training split and evaluated on the test split described in Section 5. No parameter is fitted to the test set and then reported as a prediction; the supervised baselines and the LLM prompts are compared on the same held-out samples. The only self-citations are to Bölücü et al. (2023), used for the in-context-learning sample-selection method and as prior evidence that LLM-based IE is challenging; these are implementation details and external prior results, not load-bearing assumptions that force the outcome. The relation-extraction task assumes that a factor's relationship to the hidden concept of trust can be inferred from a sentence and its paragraph; that is an annotation design choice explicitly stated in Section 3 and Appendix A.1, not a circular definition of the experimental result. A reviewer-level concern that the test set may overlap with documents used to develop the annotation guidelines is a benchmark-validity issue, not a circular-reasoning reduction where an equation or fitted parameter collapses into its own input. Therefore, no circular step is established under the required standard.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No free parameters were introduced beyond standard model hyperparameters, which are tuned on the dev set and are not part of the paper's claim. The main axioms are domain assumptions about the annotation schema, in particular that relations to a hidden 'trust' concept can be inferred from local context, that the four entity categories are exhaustive, and that the five pilot articles can serve as a test set. No new entities are invented.

assumptions (4)
  • domain assumption A factor mention's relationship to the hidden concept 'trust' can be reliably inferred from the sentence and paragraph even when 'trust' is not explicitly stated.
    The RE task (Section 3) defines relations to the hidden concept trust; the dataset's validity depends on annotators making this inference consistently, and initial kappa of 0.323 for relations shows it is difficult.
  • domain assumption The four entity categories (application, human factor, technology factor, context factor) are exhaustive for trust antecedents in the selected literature.
    The annotation guidelines fix these categories (Section 4 and Appendix A.2); if real antecedents fall outside them, the dataset under-covers the space.
  • ad hoc to paper The five articles used in the pilot and guideline revision can serve as an adequate test set for benchmarking.
    The test set is drawn from the same publications as phases i-iii (Section 5); this is a methodological choice specific to this paper and not a general principle.
  • domain assumption Public English-language abstracts, results, and conclusions are sufficient to capture empirically tested trust antecedents.
    Appendix A.1 restricts annotation to these sections; factors may appear elsewhere in the papers.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Can AI Extract Antecedent Factors of Human Trust in AI? An Application of Information Extraction for Scientific Literature in Behavioural and Computer Sciences." pith.science (2026). https://pith.science/paper/ESWNYZEZ

@misc{pith2026241211344,
  author       = {Pith},
  title        = {Pith review of: Can AI Extract Antecedent Factors of Human Trust in AI? An Application of Information Extraction for Scientific Literature in Behavioural and Computer Sciences},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ESWNYZEZ}},
  note         = {Machine review of arXiv:2412.11344}
}
read the original abstract

Information extraction from the scientific literature is one of the main techniques to transform unstructured knowledge hidden in the text into structured data which can then be used for decision-making in down-stream tasks. One such area is Trust in AI, where factors contributing to human trust in artificial intelligence applications are studied. The relationships of these factors with human trust in such applications are complex. We hence explore this space from the lens of information extraction where, with the input of domain experts, we carefully design annotation guidelines, create the first annotated English dataset in this domain, investigate an LLM-guided annotation, and benchmark it with state-of-the-art methods using large language models in named entity and relation extraction. Our results indicate that this problem requires supervised learning which may not be currently feasible with prompt-based LLMs.

Figures

Figures reproduced from arXiv: 2412.11344 by the authors.

Figure 1
Figure 1. Interface for manual annotation of the NER task. [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. Distribution of factors and application of our [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 23 canonical work pages

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Kritika Agrawal, Aakash Mittal, and Vikram Pudi. 2019. https://doi.org/10.18653/v1/W19-2602 Scalable, semi-supervised extraction of structured information from scientific literature . In Proceedings of the Workshop on Extracting Structured Knowledge from Scientific Publications, pages 11--20

    work page doi:10.18653/v1/w19-2602 2019

  4. [4]

    Monica Agrawal, Stefan Hegselmann, Hunter Lang, Yoon Kim, and David Sontag. 2022. Large language models are few-shot clinical information extractors . In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pages 1998--2022

    work page 2022

  5. [5]

    Parikshit Bansal and Amit Sharma. 2023. Large language models as annotators: Enhancing generalization of nlp models at minimal cost . arXiv preprint arXiv:2306.15766

    arXiv 2023

  6. [6]

    o l \"u c \

    Necva B \"o l \"u c \"u , Maciej Rybinski, and Stephen Wan. 2023. impact of sample selection on in-context learning for entity extraction from scientific writing . In Findings of the Association for Computational Linguistics: EMNLP 2023 , pages 5090--5107

    work page 2023

  7. [7]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners . Advances in neural information processing systems , 33:1877--1901

    work page 2020

  8. [8]

    S \'e bastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al. 2023. Sparks of artificial general intelligence: Early experiments with gpt-4 . arXiv preprint arXiv:2303.12712

    arXiv 2023

Show all 43 references
  1. [9]

    Jacob Cohen. 1960. A coefficient of agreement for nominal scales . Educational and psychological measurement , 20(1):37--46

    1960

  2. [10]

    Annemarie Friedrich, Heike Adel, Federico Tomazic, Johannes Hingerl, Renou Benteau, Anika Maruscyk, and Lukas Lange. 2020. https://aclanthology.org/2020.acl-main.116.pdf The SOFC-Exp Corpus and Neural Approaches to Information Extraction in the Materials Science Domain . In Pr...

    2020

  3. [11]

    Ella Glikson and Anita Williams Woolley. 2020. Human trust in artificial intelligence: Review of empirical research . Academy of Management Annals , 14(2):627--660

    2020

  4. [12]

    Akshay Goel, Almog Gueta, Omry Gilon, Chang Liu, Sofia Erell, Lan Huong Nguyen, Xiaohong Hao, Bolous Jaber, Shashir Reddy, Rupesh Kartha, et al. 2023. LLMs Accelerate Annotation for Medical Information Extraction . In Machine Learning for Health (ML4H) , pages 82--100. PMLR

    2023

  5. [13]

    Yucan Guo, Zixuan Li, Xiaolong Jin, Yantao Liu, Yutao Zeng, Wenxuan Liu, Xiang Li, Pan Yang, Long Bai, Jiafeng Guo, et al. 2023. Retrieval-augmented code generation for universal information extraction . arXiv preprint arXiv:2311.02962

    2023 arXiv

  6. [14]

    Bernal Jim \'e nez Guti \'e rrez, Nikolas McNeal, Clay Washington, You Chen, Lang Li, Huan Sun, and Yu Su. 2022. Thinking about gpt-3 in-context learning for biomedical ie? think again . arXiv preprint arXiv:2203.08410

    2022 arXiv

  7. [15]

    Hancock, Deborah R

    Peter A. Hancock, Deborah R. Billings, Kristin E. Schaefer, Jessie Y.C. Chen, Ewart J. De Visser, and Raja Parasuraman. 2011. https://doi.org/10.1177/0018720811417254 A meta-analysis of factors affecting trust in human-robot interaction . Human Factors, 53(5):517--527

    2011 doi

  8. [16]

    Matthew Honnibal, Ines Montani, Sofie Van Landeghem, Adriane Boyd, et al. 2020. spaCy: Industrial-strength natural language processing in python

    2020

  9. [17]

    Alon Jacovi, Ana Marasovi \'c , Tim Miller, and Yoav Goldberg. 2021. Formalizing trust in artificial intelligence: Prerequisites, causes and goals of human trust in AI . In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency , pages 624--635

    2021

  10. [18]

    Sarthak Jain, Madeleine van Zuylen, Hannaneh Hajishirzi, and Iz Beltagy. 2020. https://doi.org/10.18653/v1/2020.acl-main.670 SciREX: A Challenge Dataset for Document-Level Information Extraction . In ACL, pages 7506--7516

    2020 doi

  11. [19]

    Kaplan, Theresa T

    Alexandra D. Kaplan, Theresa T. Kessler, J. Christopher Brill, and P. A. Hancock. 2021. https://doi.org/10.1177/00187208211013988 Trust in artificial intelligence: Meta -analytic findings . Human Factors, page 001872082110139

    2021 doi

  12. [20]

    Sarvnaz Karimi, Alejandro Metke-Jimenez, Madonna Kemp, and Chen Wang. 2015. https://www.sciencedirect.com/science/article/pii/S1532046415000532 CADEC: A corpus of adverse drug event annotations . JBI, 55

    2015

  13. [21]

    Kim, T Ohta, Y Tateisi, and J Tsujii

    J.-D. Kim, T Ohta, Y Tateisi, and J Tsujii. 2003. https://academic.oup.com/bioinformatics/article/19/suppl_1/i180/227927 GENIA corpus--a semantically annotated corpus for bio-textmining . Bioinformatics, 19

    2003

  14. [22]

    Peng Li, Tianxiang Sun, Qiong Tang, Hang Yan, Yuanbin Wu, Xuanjing Huang, and Xipeng Qiu. 2023. Codeie: Large code generation models are better few-shot information extractors. arXiv preprint arXiv:2305.05711

    2023 arXiv

  15. [23]

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Ro BERT a: A R obustly O ptimized BERT P retraining A pproach. arXiv preprint arXiv:1907.11692

    2019 arXiv

  16. [24]

    Steven Lockey, Nicole Gillespie, Daniel Holm, and Ida Asadi Someh. 2021. https://doi.org/10.24251/hicss.2021.664 A review of trust in artificial intelligence: Challenges , vulnerabilities and future directions . In Proceedings of the Annual Hawaii International Conference on S...

    2021 doi

  17. [25]

    Mary L McHugh. 2012. Interrater reliability: the kappa statistic . Biochemia medica , 22(3):276--282

    2012

  18. [26]

    Ines Montani and Matthew Honnibal. 2018. Prodigy: A new annotation tool for radically efficient machine teaching . Artificial Intelligence

    2018

  19. [27]

    Hiroki Nakayama. 2018. https://github.com/chakki-works/seqeval seqeval : A python framework for sequence labeling evaluation . Software available from https://github.com/chakki-works/seqeval

    2018

  20. [28]

    Yixin Ou, Ningyu Zhang, Honghao Gui, Ziwen Xu, Shuofei Qiao, Zhen Bi, and Huajun Chen. 2024. EasyInstruct: An Easy-to-use Instruction Processing Framework for Large Language Models . arXiv preprint arXiv:2402.03049

    2024 arXiv

  21. [29]

    Raja Parasuraman and Victor Riley. 1997. https://doi.org/https://doi.org/10.1518/001872097778543886 Humans and automation: Use , misuse, disuse, abuse . Human Factors, 39(2):230--253. Publication Title: HUMAN FACTORS Volume: 39 Issue: 2

    1997 doi

  22. [30]

    Ji Qi, Chuchun Zhang, Xiaozhi Wang, Kaisheng Zeng, Jifan Yu, Jinxin Liu, Jiuding Sun, Yuxiang Chen, Lei How, Juanzi Li, et al. 2023. Preserving knowledge invariance: Rethinking robustness evaluation of open information extraction. arXiv preprint arXiv:2305.13981

    2023 arXiv

  23. [31]

    Oscar Sainz, Iker Garc \' a-Ferrero, Rodrigo Agerri, Oier Lopez de Lacalle, German Rigau, and Eneko Agirre. 2023. Gollie: Annotation guidelines improve zero-shot information-extraction . arXiv preprint arXiv:2310.03668

    2023 arXiv

  24. [32]

    Till Sa mannshausen, Peter Burggr \"a f, Marc Hassenzahl, and Johannes Wagner. 2023. Human trust in otherware--a systematic literature review bringing all antecedents together . Ergonomics , 66(7):976--998

    2023

  25. [33]

    Schaefer, Jessie Y.C

    Kristin E. Schaefer, Jessie Y.C. Chen, James L. Szalma, and P. A. Hancock. 2016. https://doi.org/10.1177/0018720816634228 A meta-analysis of factors influencing the development of trust in automation: Implications for understanding autonomy in future systems . Human Factors, 5...

    2016 doi

  26. [34]

    Schaefer, Brandon S

    Kristin E. Schaefer, Brandon S. Perelman, Gregory M. Gremillion, Amar R. Marathe, and Jason S. Metcalfe. 2021. https://doi.org/10.1016/B978-0-12-819472-0.00012-5 A roadmap for developing team trust metrics for human-autonomy teams . In Trust in Human - Robot Interaction , page...

    2021 doi

  27. [35]

    Jana Strakov \'a , Milan Straka, and Jan Hajic. 2019. Neural Architectures for Nested NER through Linearization . In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , pages 5326--5331

    2019

  28. [36]

    Human-AI Teaming. 2022. https://doi.org/10.17226/26355 Human-AI State-of-the-Art and Research Needs . Washington, DC: National Academies Press . Pages: 26355

    2022 doi

  29. [37]

    Chenguang Wang, Xiao Liu, Zui Chen, Haoyun Hong, Jie Tang, and Dawn Song. 2022. DeepStruct: Pretraining of Language Models for Structure Prediction . In Findings of the Association for Computational Linguistics: ACL 2022 , pages 803--823

    2022

  30. [38]

    Juntao Yu, Bernd Bohnet, and Massimo Poesio. 2020. Named Entity Recognition as Dependency Parsing . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages 6470--6476

    2020

  31. [39]

    Kai Zhang, Bernal Jim \'e nez Guti \'e rrez, and Yu Su. 2023 a . Aligning instruction tasks unlocks large language models as zero-shot relation extractors . arXiv preprint arXiv:2305.11159

    2023 arXiv

  32. [40]

    Mozhi Zhang, Hang Yan, Yaqian Zhou, and Xipeng Qiu. 2023 b . Promptner: A prompting method for few-shot named entity recognition via k nearest neighbor search. arXiv preprint arXiv:2305.12217

    2023 arXiv

  33. [41]

    Ruoyu Zhang, Yanzeng Li, Yongliang Ma, Ming Zhou, and Lei Zou. 2023 c . Llmaaa: Making large language models as active annotators . arXiv preprint arXiv:2310.19596

    2023 arXiv

  34. [42]

    Zexuan Zhong and Danqi Chen. 2021. A Frustratingly Easy Approach for Entity and Relation Extraction . In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , pages 50--61

    2021

  35. [43]

    Wenxuan Zhou, Sheng Zhang, Yu Gu, Muhao Chen, and Hoifung Poon. 2023. Universalner: Targeted distillation from large language models for open named entity recognition. arXiv preprint arXiv:2308.03279

    2023 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.