pith. machine review for the scientific record. sign in

arxiv: 2605.10125 · v2 · submitted 2026-05-11 · 💻 cs.AI · cs.HC

Recognition: no theorem link

Useful for Exploration, Risky for Precision: Evaluating AI Tools in Academic Research

Anthea Dathe , Kiran Hoffmann , Aline Mangold
Authors on Pith no claims yet

Pith reviewed 2026-05-13 03:31 UTC · model grok-4.3

classification 💻 cs.AI cs.HC
keywords AI toolsacademic researchquestion answeringliterature reviewbenchmarkingexplainable AIhuman-centered evaluationreproducibility
0
0 comments X

The pith

AI tools give useful overviews for early research but prove unreliable for precise details and systematic literature work.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tests current AI question-answering and literature-review tools against both technical accuracy measures and human-centered criteria such as usability and workflow fit. It finds that Q&A systems produce generally accurate summaries and broad overviews yet frequently fail to link answers to correct source passages and cannot be trusted for exact data extraction. Literature tools help with initial, open-ended searches but deliver inconsistent results, hide their source selection process, and draw from uneven-quality databases, so they cannot support reproducible or systematic reviews. The authors conclude that these tools increase speed on shallow tasks but still place the full verification burden on the researcher.

Core claim

Q&A tools can offer valuable overviews and generally accurate summaries; however, they are not always reliable for precise information extraction, with particularly low explainable AI accuracy where highlighted passages often fail to support the generated answers. Literature review tools support exploratory searches but show low reproducibility, limited transparency on chosen sources and databases, and inconsistent source quality, rendering them unsuitable for systematic reviews.

What carries the argument

A benchmarking framework that combines human-centered metrics (usability, interpretability, workflow integration) with computer-centered metrics (accuracy, reproducibility) to evaluate AI research tools.

If this is right

  • AI tools can increase efficiency during the initial, exploratory stages of research.
  • All outputs from these tools still require careful human verification before use in formal work.
  • Explainability features would reduce the time researchers spend checking AI answers.
  • Systematic reviews and precision-critical tasks should continue to rely on traditional methods rather than current AI tools.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Training programs for researchers could include explicit modules on when and how to verify AI-generated literature summaries.
  • Tool developers might prioritize consistent source disclosure and reproducibility over speed alone.

Load-bearing premise

That the specific AI tools tested and the chosen human-centered metrics represent the wider range of available tools and typical academic research workflows.

What would settle it

Run the same literature-review query on the tested tools multiple times with different researchers and measure whether the returned sources, summaries, and selection criteria match across runs.

read the original abstract

Artificial intelligence (AI) tools are being incorporated into scientific research workflows with the potential to enhance efficiency in tasks such as document analysis, question answering (Q&A), and literature search. However, system outputs are often difficult to verify, lack transparency in their generation and remain prone to errors. Suitable benchmarks are needed to document and evaluate arising issues. Nevertheless, existing benchmarking approaches are not adequately capturing human-centered criteria such as usability, interpretability, and integration into research workflows. To address this gap, the present work proposes and applies a benchmarking framework combining human-centered and computer-centered metrics to evaluate AI-based Q&A and literature review tools for research use. The findings suggest that Q&A tools can offer valuable overviews and generally accurate summaries; however, they are not always reliable for precise information extraction. Explainable AI (xAI) accuracy was particularly low, meaning highlighted source passages frequently failed to correspond to generated answers. This shifted the burden of validation back onto the researcher. Literature review tools supported exploratory searches but showed low reproducibility, limited transparency regarding chosen sources and databases, and inconsistent source quality, making them unsuitable for systematic reviews. A comparison of these tool groups reveals a similar pattern: while AI tools can enhance efficiency in the early stages of the research workflow and shallow tasks, their outputs still require human verification. The findings underscore the importance of explainability features to enhance transparency, verification efficiency and careful integration of AI tools into researchers' workflows. Further, human-centered evaluation remains an important concern to ensure practical applicability.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The manuscript proposes a benchmarking framework that integrates human-centered metrics (usability, interpretability, verification burden) with computer-centered metrics to evaluate AI-based Q&A and literature review tools for academic research. Application of the framework to selected tools yields the claims that Q&A tools deliver useful overviews and generally accurate summaries but are unreliable for precise information extraction, with particularly low xAI accuracy where highlighted source passages fail to correspond to generated answers; literature review tools aid exploratory searches but exhibit low reproducibility, limited transparency on sources and databases, and inconsistent source quality, rendering them unsuitable for systematic reviews. The work concludes that AI tools enhance efficiency in early-stage and shallow tasks yet still require human verification, underscoring the need for improved explainability and careful workflow integration.

Significance. If the framework is shown to be robust and the empirical patterns hold beyond the tested instances, the paper would provide a timely, human-centered lens on AI tool adoption in research that existing technical benchmarks largely omit. It supplies concrete evidence on verification burden and reproducibility gaps that could guide both tool developers and researchers toward safer integration practices, particularly by highlighting the mismatch between exploratory utility and precision requirements.

major comments (2)
  1. [Evaluation Framework and Results] The central claims that Q&A tools are 'not always reliable for precise information extraction' and that literature tools are 'unsuitable for systematic reviews' rest on the tested tools and human-centered metrics being representative of the broader landscape. However, the manuscript provides no explicit selection criteria, sample-size justification, or cross-validation against other tools/models in the evaluation design, leaving the extrapolation from specific instances to general statements about 'AI tools' unsupported.
  2. [Methods and Empirical Application] The reported low xAI accuracy and low reproducibility findings are presented as key outcomes, yet the paper does not detail the number of test queries/tasks, the exact tool versions used, or inter-rater procedures for the human metrics; without these, it is impossible to assess whether post-hoc choices or limited cases affect the load-bearing conclusions about verification burden and transparency.
minor comments (1)
  1. [Introduction] The abstract and introduction could more clearly distinguish the proposed framework from prior benchmarking efforts in AI evaluation literature to strengthen the novelty claim.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback, which has helped us improve the transparency and rigor of our evaluation framework. We address each major comment below and have revised the manuscript to incorporate additional methodological details and justifications.

read point-by-point responses

  1. Referee: [Evaluation Framework and Results] The central claims that Q&A tools are 'not always reliable for precise information extraction' and that literature tools are 'unsuitable for systematic reviews' rest on the tested tools and human-centered metrics being representative of the broader landscape. However, the manuscript provides no explicit selection criteria, sample-size justification, or cross-validation against other tools/models in the evaluation design, leaving the extrapolation from specific instances to general statements about 'AI tools' unsupported.

    Authors: We agree that greater clarity on tool selection and scope is needed to support our claims. In the revised manuscript, we have added Section 3.1 detailing explicit selection criteria (popularity in academic workflows, public availability, and diversity of underlying models), the sample size (four Q&A tools and three literature review tools), and justification based on the exploratory goals of the study and practical constraints on human evaluation effort. We have also revised the language in the abstract and conclusion to emphasize that findings apply to the evaluated tools and to explicitly call for future cross-validation with additional models. While broader cross-validation was outside the scope of this initial work, the proposed framework is designed to facilitate such extensions. revision: yes

  2. Referee: [Methods and Empirical Application] The reported low xAI accuracy and low reproducibility findings are presented as key outcomes, yet the paper does not detail the number of test queries/tasks, the exact tool versions used, or inter-rater procedures for the human metrics; without these, it is impossible to assess whether post-hoc choices or limited cases affect the load-bearing conclusions about verification burden and transparency.

    Authors: We acknowledge these omissions limit the assessability of our results. The revised Methods section now specifies the number of test queries and tasks (25 Q&A queries and 15 literature review tasks), the exact tool versions and testing dates used, and the inter-rater procedures (two independent raters, with reported agreement of 82% on accuracy metrics and consensus-based resolution of disagreements). These additions directly address concerns about verification burden and transparency while preserving the original empirical patterns. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical evaluation of external tools

full rationale

The manuscript proposes a benchmarking framework and reports direct observations from its application to a finite set of commercial AI tools (Q&A and literature review). Claims about overview utility, extraction unreliability, low reproducibility, and the need for human verification are presented as outcomes of that testing rather than reductions of any equation, fitted parameter, or self-citation chain. No derivation steps, ansatzes, or uniqueness theorems appear; the work is self-contained as an empirical study whose generalizability rests on the tested sample rather than internal redefinition.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The central claims rest on the representativeness of the tested tools and the validity of the proposed human-centered metrics; these are domain assumptions rather than derived results.

axioms (2)
  • domain assumption The selected AI Q&A and literature review tools are representative of current widely used systems in academic research.
    The evaluation and its generalizations depend on this without explicit justification or sampling from the full tool population.
  • domain assumption The human-centered criteria (usability, interpretability, workflow integration) combined with technical metrics capture the practically relevant dimensions of tool performance.
    This underpins the benchmarking framework and the conclusion that tools are suitable only for early-stage work.

pith-pipeline@v0.9.0 · 5576 in / 1440 out tokens · 107481 ms · 2026-05-13T03:31:29.838221+00:00 · methodology

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

30 extracted references · 30 canonical work pages · 1 internal anchor

  1. [1]

    Aisha Alansari and Hamzah Luqman.Large Language Models Hallucination: A Comprehensive Survey. Mar. 2026.DOI:10.48550/arXiv.2510.06265. arXiv:2510.06265 [cs]. (Visited on 04/09/2026)

    work page doi:10.48550/arxiv.2510.06265 2026

  2. [2]

    May 2025.DOI:10.21203/rs.3.rs-6328602/v1

    Nisha Biju et al.Designing an AI-Driven SLR Workflow for Academic Research: A Rubric for Comparative Analysis of AI Tools. May 2025.DOI:10.21203/rs.3.rs-6328602/v1. (Visited on 01/26/2026)

    work page doi:10.21203/rs.3.rs-6328602/v1 2025

  3. [3]

    What Is Generative in Generative Artificial Intelligence? A Design-Based Perspective

    Antoine Bordas et al. “What Is Generative in Generative Artificial Intelligence? A Design-Based Perspective”. In:Research in Engineering Design35.4 (Oct. 2024), pp. 427–443.ISSN: 1435-6066.DOI: 10.1007/s00163- 024-00441-x. (Visited on 03/27/2026)

    work page doi:10.1007/s00163- 2024

  4. [4]

    Collage Is the New Writing: Exploring the Fragmentation of Text and User Interfaces in AI Tools

    Daniel Buschek. “Collage Is the New Writing: Exploring the Fragmentation of Text and User Interfaces in AI Tools”. In:Proceedings of the 2024 ACM Designing Interactive Systems Conference. DIS ’24. New York, NY , USA: Association for Computing Machinery, July 2024, pp. 2719–2737.ISBN: 979-8-4007-0583-0.DOI: 10.1145/3643834.3660681. (Visited on 04/23/2026)

    work page doi:10.1145/3643834.3660681 2024

  5. [5]

    June 2022.DOI: 10.48550/arXiv.2206.12390

    Andres Campero et al.A Test for Evaluating Performance in Human-Computer Systems. June 2022.DOI: 10.48550/arXiv.2206.12390. arXiv:2206.12390 [cs]. (Visited on 04/23/2026)

    work page doi:10.48550/arxiv.2206.12390 2022

  6. [6]

    Analysis of Article Screening and Data Extraction Performance by an AI Systematic Literature Review Platform

    Kelsie Cassell et al. “Analysis of Article Screening and Data Extraction Performance by an AI Systematic Literature Review Platform”. In:Frontiers in Artificial Intelligence8 (), p. 1662202.ISSN: 2624-8212.DOI: 10.3389/frai.2025.1662202. (Visited on 04/09/2026)

    work page doi:10.3389/frai.2025.1662202 2025

  7. [7]

    AI- and LLM-driven Search Tools: A Paradigm Shift in Informa- tion Access for Education and Research

    Gobinda Chowdhury and Sudatta Chowdhury. “AI- and LLM-driven Search Tools: A Paradigm Shift in Informa- tion Access for Education and Research”. In:Journal of Information Science(Oct. 2024), p. 01655515241284046. ISSN: 0165-5515.DOI:10.1177/01655515241284046. (Visited on 01/26/2026)

    work page doi:10.1177/01655515241284046 2024

  8. [8]

    Quality and Effectiveness of AI Tools for Students and Researchers for Scientific Literature Review and Analysis

    Martin Danler et al. “Quality and Effectiveness of AI Tools for Students and Researchers for Scientific Literature Review and Analysis”. In:dHealth 2024. IOS Press, 2024, pp. 203–208.DOI: 10.3233/SHTI240038. (Visited on 01/26/2026)

    work page doi:10.3233/shti240038 2024

  9. [9]

    Evaluating Explainability in Language Classification Models: A Unified Framework Incorporating Feature Attribution Methods and Key Factors Affecting Faithfulness

    Tahereh Dehdarirad. “Evaluating Explainability in Language Classification Models: A Unified Framework Incorporating Feature Attribution Methods and Key Factors Affecting Faithfulness”. In:Data and Information Management9.4 (Dec. 2025), p. 100101.ISSN: 2543-9251.DOI: 10.1016/j.dim.2025.100101. (Visited on 04/09/2026)

    work page doi:10.1016/j.dim.2025.100101 2025

  10. [10]

    Rise of Generative Artificial Intelligence in Science

    Liangping Ding, Cornelia Lawson, and Philip Shapira. “Rise of Generative Artificial Intelligence in Science”. In: Scientometrics130.9 (Sept. 2025), pp. 5093–5114.ISSN: 1588-2861.DOI: 10.1007/s11192-025-05413-z . (Visited on 04/23/2026)

    work page doi:10.1007/s11192-025-05413-z 2025

  11. [11]

    Opinion Paper: “So What If ChatGPT Wrote It?

    Yogesh K. Dwivedi et al. “Opinion Paper: “So What If ChatGPT Wrote It?” Multidisciplinary Perspectives on Opportunities, Challenges and Implications of Generative Conversational AI for Research, Practice and Policy”. In:International Journal of Information Management71 (Aug. 2023), p. 102642.ISSN: 0268-4012. DOI:10.1016/j.ijinfomgt.2023.102642. (Visited o...

    work page doi:10.1016/j.ijinfomgt.2023.102642 2023

  12. [12]

    Artificial Intelligence Search Tools for Evidence Synthesis: Comparative Analysis and Implementation Recommendations

    Robin Featherstone et al. “Artificial Intelligence Search Tools for Evidence Synthesis: Comparative Analysis and Implementation Recommendations”. In:Cochrane Evidence Synthesis and Methods3.5 (2025), e70045.ISSN: 2832-9023.DOI:10.1002/cesm.70045. (Visited on 01/26/2026)

    work page doi:10.1002/cesm.70045 2025

  13. [13]

    DARPA’s Explainable Artificial Intelligence Program

    David Gunning and David W. Aha. “DARPA’s Explainable Artificial Intelligence Program”. In:AI Magazine 40.2 (June 2019), pp. 44–58.ISSN: 0738-4602, 2371-9621.DOI: 10.1609/aimag.v40i2.2850 . (Visited on 08/04/2023)

    work page doi:10.1609/aimag.v40i2.2850 2019

  14. [14]

    The Strain on Scientific Publishing

    Mark A. Hanson et al. “The Strain on Scientific Publishing”. In:Quantitative Science Studies5.4 (Nov. 2024), pp. 823–843.ISSN: 2641-3337.DOI:10.1162/qss_a_00327. (Visited on 04/23/2026)

    work page doi:10.1162/qss_a_00327 2024

  15. [15]

    Examining the Associations between PTSD Symptoms and Aspects of Emotion Dysregulation through Network Analysis

    James Kyle Haws et al. “Examining the Associations between PTSD Symptoms and Aspects of Emotion Dysregulation through Network Analysis”. In:Journal of Anxiety Disorders86 (Mar. 2022), p. 102536.ISSN: 0887-6185.DOI:10.1016/j.janxdis.2022.102536. (Visited on 05/09/2025). Evaluating AI Tools in Academic Research 16

    work page doi:10.1016/j.janxdis.2022.102536 2022

  16. [16]

    Using Artificial Intelligence Tools as Second Reviewers for Data Extraction in Systematic Reviews: A Performance Comparison of Two AI Tools Against Human Reviewers

    T. Helms Andersen et al. “Using Artificial Intelligence Tools as Second Reviewers for Data Extraction in Systematic Reviews: A Performance Comparison of Two AI Tools Against Human Reviewers”. In:Cochrane Evidence Synthesis and Methods3.4 (2025), e70036.ISSN: 2832-9023.DOI: 10.1002/cesm.70036. (Visited on 01/26/2026)

    work page doi:10.1002/cesm.70036 2025

  17. [17]

    Emotion in Criminal Offenders With Psychopathy and Borderline Personality Disorder

    Sabine C. Herpertz et al. “Emotion in Criminal Offenders With Psychopathy and Borderline Personality Disorder”. In:Archives of General Psychiatry58.8 (Aug. 2001), pp. 737–745.ISSN: 0003-990X.DOI: 10.1001/archpsyc. 58.8.737. (Visited on 08/13/2023)

    work page doi:10.1001/archpsyc 2001

  18. [18]

    Frequent Itemset Discovery

    Uday Kamath and John Liu.Explainable Artificial Intelligence: An Introduction to Interpretable Machine Learning. Cham: Springer International Publishing, 2021.ISBN: 978-3-030-83355-8.DOI: 10.1007/978-3- 030-83356-5. (Visited on 08/04/2023)

    work page doi:10.1007/978-3- 2021

  19. [19]

    Laurent and Joseph D

    Jon M. Laurent et al.LAB-Bench: Measuring Capabilities of Language Models for Biology Research. July 2024. DOI:10.48550/arXiv.2407.10362. arXiv:2407.10362 [cs]. (Visited on 01/26/2026)

    work page doi:10.48550/arxiv.2407.10362 2024

  20. [20]

    Machine Learning in Concrete Science: Applications, Challenges, and Best Practices

    Zhanzhao Li et al. “Machine Learning in Concrete Science: Applications, Challenges, and Best Practices”. In: npj Computational Materials8.1 (June 2022), p. 127.ISSN: 2057-3960.DOI: 10.1038/s41524-022-00810-x . (Visited on 04/23/2026)

    work page doi:10.1038/s41524-022-00810-x 2022

  21. [21]

    Facial Reactions during Emotion Recognition in Borderline Personality Disorder: A Facial Electromyography Study

    Burkhard Matzke et al. “Facial Reactions during Emotion Recognition in Borderline Personality Disorder: A Facial Electromyography Study”. In:Psychopathology47.2 (Sept. 2013), pp. 101–110.ISSN: 0254-4962.DOI: 10.1159/000351122. (Visited on 07/27/2023)

    work page doi:10.1159/000351122 2013

  22. [22]

    Artificial Intelligence Capability: Conceptualization, Measurement Calibra- tion, and Empirical Study on Its Impact on Organizational Creativity and Firm Performance

    Patrick Mikalef and Manjul Gupta. “Artificial Intelligence Capability: Conceptualization, Measurement Calibra- tion, and Empirical Study on Its Impact on Organizational Creativity and Firm Performance”. In:Information & Management58.3 (Apr. 2021), p. 103434.ISSN: 0378-7206.DOI: 10.1016/j.im.2021.103434. (Visited on 04/23/2026)

    work page doi:10.1016/j.im.2021.103434 2021

  23. [23]

    AI Tools for Automating Systematic Literature Reviews

    Andrei Mikriukov et al. “AI Tools for Automating Systematic Literature Reviews”. In:Proceedings of the 2025 International Conference on Software Engineering and Computer Applications. SECA ’25. New York, NY , USA: Association for Computing Machinery, Aug. 2025, pp. 25–30.ISBN: 979-8-4007-1513-6.DOI: 10.1145/3747912.3747962. (Visited on 04/09/2026)

    work page doi:10.1145/3747912.3747962 2025

  24. [24]

    Benchmarking Generative AI: A Comparative Evaluation and Practical Guidelines for Responsible Integration into Academic Research

    Swapnil Morande. “Benchmarking Generative AI: A Comparative Evaluation and Practical Guidelines for Responsible Integration into Academic Research”. In:SSRN Electronic Journal(2023).ISSN: 1556-5068.DOI: 10.2139/ssrn.4571867. (Visited on 01/26/2026)

    work page doi:10.2139/ssrn.4571867 2023

  25. [25]

    Cognitive Reappraisal Impairs Negative Affect Regulation in the Context of Social Rejection for Youth With Early-Stage Borderline Personality Disorder

    Elizabeth Pizarro-Campagna et al. “Cognitive Reappraisal Impairs Negative Affect Regulation in the Context of Social Rejection for Youth With Early-Stage Borderline Personality Disorder”. In:Journal of Personality Disorders37.2 (Apr. 2023), pp. 156–176.ISSN: 0885-579X.DOI: 10.1521/pedi.2023.37.2.156. (Visited on 08/14/2023)

    work page doi:10.1521/pedi.2023.37.2.156 2023

  26. [26]

    Evaluation of Different AI Online Tools for General Scientific Research and Scientific Publi- cation

    Joseph Shenekji. “Evaluation of Different AI Online Tools for General Scientific Research and Scientific Publi- cation”. In:Journal of Advances in Machine Learning & Artificial Intelligence(2025). (Visited on 01/26/2026)

    work page 2025

  27. [27]

    Decreased Facial Reactivity and Mirroring in Women with Borderline Personality Disorder - A Facial Electromyography Study

    Anna Steinbrenner et al. “Decreased Facial Reactivity and Mirroring in Women with Borderline Personality Disorder - A Facial Electromyography Study”. In:Psychiatry Research Communications2.2 (June 2022), p. 100040.ISSN: 2772-5987.DOI:10.1016/j.psycom.2022.100040. (Visited on 07/27/2023)

    work page doi:10.1016/j.psycom.2022.100040 2022

  28. [28]

    Romal Thoppilan et al.LaMDA: Language Models for Dialog Applications. Feb. 2022.DOI: 10.48550/arXiv. 2201.08239. arXiv:2201.08239 [cs]. (Visited on 04/23/2026)

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv 2022

  29. [29]

    A Comparative Analysis of ChatGPT and AI- Powered Research Tools for Scientific Writing and Research

    Zineb Touati Hamad, Mohamed Laouar, and M.H Yaccoub. “A Comparative Analysis of ChatGPT and AI- Powered Research Tools for Scientific Writing and Research”. In: Dec. 2024, pp. 251–268.ISBN: 978-3-031- 71428-3.DOI:10.1007/978-3-031-71429-0_19

    work page doi:10.1007/978-3-031-71429-0_19 2024

  30. [30]

    AI and Science: What 1,600 Researchers Think

    Richard Van Noorden and Jeffrey M. Perkel. “AI and Science: What 1,600 Researchers Think”. In:Nature 621.7980 (Sept. 2023), pp. 672–675.DOI:10.1038/d41586-023-02980-0. (Visited on 06/13/2025). Evaluating AI Tools in Academic Research 17 A Q&A Measures Table 11: Benchmarking of Q&A tools: Investigated metrics Metric Description Evaluation Consistency exter...

    work page doi:10.1038/d41586-023-02980-0 2023