Pith. sign in

REVIEW 2 major objections 5 minor 1 cited by

From Assistance to Autonomy -- A Researcher Study on the Potential of AI Support for Qualitative Data Analysis

T0 review · 2 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read AI support for qualitative data analysis fits a three-level framework — minimal, moderate, high — grounded in how HCI researchers actually work.

desk verdict A solid, preregistered interview study with a usable framework of AI roles for QDA; the main risk is the handwritten-note method, which deserves a clear limitation statement but doesn't sink the paper. read the letter →

arxiv 2501.19275 v4 pith:UOB56D7M submitted 2025-01-31 cs.CY

Elisabeth Kirsten , Annalina Buckmann , Leona Lassak , Nele Borgert , Abraham Mhaidli , Steffen Becker This is my paper · ORCID
classification cs.CY
keywords qualitativedataanalysislargelanguagemodelshuman-AIcollaborationHCIresearchersCSCWinterviewstudyAIinvolvementframeworkcoding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that AI involvement in Qualitative Data Analysis can be organized into a three-level framework — minimal, moderate, and high — and that this framework matches the real-life workflows of HCI researchers. Based on semi-structured interviews with 15 experienced qualitative researchers, the authors identify concrete roles AI can play, from transcription and training tools through mediator, sparring partner, and validator, to autonomous coders with human approval or oversight. Researchers were broadly open to AI support, but conditioned it on data privacy, verifiability of outputs, and preserving human control over interpretation. The framework is offered as a practical guide for deciding when AI should assist, collaborate with, or lead qualitative coding, and as a shared vocabulary for community standards on responsible human-AI collaboration in QDA.

What carries the argument

The central object is the three-level framework of AI involvement in QDA, from minimal to moderate to high, with each level defined by specific roles AI can adopt (e.g., mediator, sparring partner, validator, autonomous coder with conditional autonomy). The framework carries the argument by linking the interview-derived workflow stages, pain points, and concerns to a continuum of AI agency, so that each role names a concrete intervention point that preserves human oversight while expanding AI's contribution.

What would settle it

Re-run the interview study with audio recording and independent transcription on a comparable sample; if the resulting workflow maps reveal stages, pain points, or AI-role preferences that do not fit within the three-level framework, the framework would be shown to be incomplete or misaligned.

Watch

Extended reading notes

Core claim

The central discovery is an empirically derived taxonomy: HCI researchers' QDA workflows converge on a three-stage structure — codebook creation, codebook refinement, codebook application — while varying strongly by project context, and researchers are willing to integrate AI at many of these stages as long as they retain agency. From interview data the authors derive three levels of AI involvement — minimal, moderate, high — populated with concrete roles such as AI as technical support, productivity tool, training tool, mediator, validator, sparring partner, and, at the highest levels, human-in-the-loop, human approval, conditional autonomy, and full delegation. The paper argues this framework is aligned with real-life workflows and addresses researchers' concerns, making it a usable map for designers.

Load-bearing premise

The framework's completeness depends on the handwritten protocols produced by two researchers who deliberately did not audio-record interviews; if those notes missed subtle interpretive statements, the derived roles and concerns could be incomplete or skewed.

Editorial extensions

If this is right

  • Designers can map the framework's roles onto specific QDA stages, such as pre-processing, researcher onboarding, codebook refinement, conflict mediation, and validation, to decide where AI features belong.
  • Researchers who worry about losing interpretive control can begin at minimal or moderate involvement and increase AI's role only when trust conditions are met.
  • The finding that willingness to use AI does not correlate with experience level implies that tools should be configurable for both novice guidance and expert customization.
  • The framework gives QDA software vendors concrete role models — such as AI computing inter-rater reliability or mediating coding disagreements — to implement in commercial tools.
  • By framing responsible AI integration as a spectrum rather than a binary, the paper provides a shared vocabulary for community discussions and standards.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the framework holds, the highest-value near-term AI tools are likely those that preserve what participants called 'messiness' — e.g., AI that flags discrepancies or suggests codes for review — rather than full automation that removes researchers from the data.
  • A testable extension is to turn the framework's roles into design probes and measure whether, say, an AI mediator reduces consensus-building time without eroding interpretive depth.
  • Because the sample was drawn entirely from HCI and usable-security researchers, the framework's transferability to other qualitative disciplines (sociology, education, health research) remains an open question; a cross-disciplinary replication would test whether the roles and levels are universal.
  • The paper's emphasis on configurable AI suggests an agentic design space in which the researcher sets the delegation boundary per project; future work could identify which project attributes (data type, team size, data sensitivity) predict the appropriate involvement level.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. This paper reports an interview study with 15 HCI researchers experienced in qualitative data analysis (QDA). It maps participants' real-world QDA workflows across codebook creation, refinement, and application, identifies pain points and rewarding aspects, and analyzes willingness, concerns, and conditions for AI support. Based on these findings, the authors propose a framework distinguishing minimal, moderate, and high AI involvement, with roles such as AI as a productivity tool, training tool, mediator, validator, analytics provider, sparring partner, human-in-the-loop, approval, conditional autonomy, and full delegation. The paper claims that this framework is aligned with real-life QDA workflows and can help guide responsible integration of AI into qualitative research.

Significance. The empirical mapping of HCI researchers' actual coding practices is a useful contribution, and the framework gives designers a concrete vocabulary for placing AI support on an autonomy spectrum. The preregistration, the detailed codebooks with frequencies and examples in Appendix B, the consensus-based coding, and the use of participant workflow sketches to construct Figure 1 are notable strengths. The claims are concrete and falsifiable. However, the evidentiary foundation rests entirely on handwritten interview protocols, and the analytical route from coded themes to the framework roles is not fully demonstrated; both points need attention before the paper can fully support its central claim.

major comments (2)
  1. [§3.2 and §6.3] The central empirical claim that the framework and workflow map are aligned with real-life QDA practice rests on the fidelity of the interview corpus, but that corpus consists solely of handwritten protocols. Section 3.2 states that interviews were deliberately not audio-recorded and that the second researcher 'noted down points that stood out without performing any immediate analysis or interpretation.' The frequency counts in Appendix B and the saturation statement in Section 3.2 can therefore reflect saturation in the notes rather than in the actual interviews. Section 6.3 acknowledges self-report, recall, and social desirability biases but does not address the note-taking filter as a distinct source of potential loss or selectivity. The authors should either provide evidence for the completeness of the notes (for example, a documented note-taking protocol, immediate member checking, or a pilot-based comparison of notes against a reference transcript) or temper the saturation and alignment claims and explicitly add this limitation.
  2. [§5 and Figure 2] The framework is presented as 'based on our results' (Section 5), but the derivation from the interview data is not transparent. No code in Appendix B corresponds directly to the three levels or to most named roles, and the text introduces roles with example quotes and related work rather than showing how they were abstracted from the codes. In particular, 'Full AI Delegation' in Section 5.3 is an extrapolated endpoint: no participant endorsed it, and Table 6 reports one participant as unwilling to use AI at all. The authors should provide an analytic trace, such as a table linking specific participant suggestions and pain-point codes to each framework role and level, or explicitly label the higher-autonomy roles as design extrapolations rather than empirical findings.
minor comments (5)
  1. [§1 and §7] There are small language errors: 'integaration' in the introduction should be 'integration', and 'benefit to the broader academic and societal landscape' should be 'benefit the broader academic and societal landscape'.
  2. [Table 2] The description for Google Sheets says 'real-time from any advice'; this should be 'real-time from any device'.
  3. [§4.2] The statement 'We found no correlation between participants’ level of experience in QDA and their willingness to adopt AI' is too strong for a sample of 15 with no described inferential test; a descriptive cross-tabulation or clearly qualified wording would avoid overclaiming.
  4. [§2.1] The sentence 'there has been no systematic documentation of how CSCW and HCI researchers actually conduct QDA in practice' is stronger than the cited literature supports; 'limited systematic documentation' would be more precise.
  5. [§6.2] The advice on 'determining the appropriate level of AI involvement' is somewhat generic; adding a worked example of how a specific project would choose between minimal, moderate, and high involvement would strengthen the framework's practical value.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the framework is induced from interview data, and the only self-citation is peripheral.

full rationale

The paper's central contribution is an empirically grounded framework for AI involvement in qualitative data analysis, derived from semi-structured interviews with 15 HCI researchers. The framework levels and roles (Sections 5.1-5.3) are presented as induced from recurring interview themes, participants' conditions and concerns, and workflow stages; the paper quotes participants directly for several roles (e.g., P7's 'sparring partner', P12's transcription condition, P13's random-subsample approval) and contextualizes them with external prior work. There is no equation whose output equals its input, no fitted parameter renamed as a prediction, and no uniqueness theorem imported from the authors' own prior work. The only self-citation is reference [47], used in Section 6.2 to support the general claim that coding performance depends on task complexity; this point is corroborated by an independent citation [59] and is not load-bearing for the framework's derivation. The non-audio-recording and handwritten-protocol procedure (Section 3.2) is a methodological limitation that could affect data fidelity and the saturation claim, but it is a validity concern, not circularity: the framework is still an induction from the collected (if filtered) evidence rather than a restatement of the paper's inputs. The paper also explicitly disclaims empirical evaluation of its proposed integrations (Section 6.3), which further confirms that no predictive result is being forced by construction. Overall, the derivation chain is self-contained with respect to circularity concerns.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

Central claims rest on qualitative research assumptions: fidelity of handwritten protocols, asserted saturation, sample representativeness, and consensus coding. These are stated or acknowledged in the paper, and none are formalized.

assumptions (4)
  • domain assumption Handwritten protocols (no audio) preserve sufficient fidelity for analysis.
    Section 3.2 explains that interviews were not audio-recorded to protect privacy; instead two researchers took notes that were merged and translated. The accuracy of subsequent coding and framework synthesis depends on the notes capturing participants' exact meanings.
  • domain assumption Data saturation was reached after 12 interviews.
    Section 3.2 states saturation was reached around interview 12 with three additional interviews to confirm, but no formal saturation assessment is described.
  • domain assumption The sample generalizes to HCI and CSCW researchers.
    Section 6.3 acknowledges all participants were HCI researchers, many with usable security or privacy specialization, which may skew findings; the framework's generalizability rests on this assumption.
  • domain assumption Consensus coding by two authors yields trustworthy themes.
    Section 3.4 describes two authors coding the full dataset and resolving disagreements through discussion; no inter-rater reliability metric is reported, and the synthesis is not externally auditable.

how reviews work

0 comments
Cite this review

Pith. "Pith review of From Assistance to Autonomy -- A Researcher Study on the Potential of AI Support for Qualitative Data Analysis." pith.science (2026). https://pith.science/paper/UOB56D7M

@misc{pith2026250119275,
  author       = {Pith},
  title        = {Pith review of: From Assistance to Autonomy -- A Researcher Study on the Potential of AI Support for Qualitative Data Analysis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UOB56D7M}},
  note         = {Machine review of arXiv:2501.19275}
}
read the original abstract

The advent of AI technologies, such as Large Language Models, has introduced new possibilities for Qualitative Data Analysis (QDA), offering both opportunities and challenges. To help navigate the responsible integration of AI into QDA, we conducted semi-structured interviews with 15 Human-Computer Interaction (HCI) researchers experienced in QDA. While our participants were open to AI support in their QDA workflows, they expressed concerns about data privacy, autonomy, and the quality of AI outputs. In response, we developed a framework that spans from minimal to high AI involvement, providing tangible scenarios for integrating AI into QDA practices while addressing researchers' needs and concerns. Aligned with real-life QDA workflows, we identify potential for AI tools in areas such as data pre-processing, researcher onboarding, or conflict mediation. Our framework aims to provoke further discussion on the development of AI-supported QDA and to help establish community standards for responsible Human-AI collaboration.

Figures

Figures reproduced from arXiv: 2501.19275 by the authors.

Figure 1
Figure 1. Detailed overview of participants’ real-world coding workflow, consisting of three stages for codebook creation, codebook [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. Our framework for integrating AI into QDA, depicting 3 stages from less to more AI involvement, visualizing possible roles for [PITH_FULL_IMAGE:figures/full_fig_p014_2.png] view at source ↗
Figure 3
Figure 3. AI provides step-by-step guidance within QDA software, suggesting functionalities based on user behavior and [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: AI generates summary reports of QDA projects, such as identifying the most or least frequent codes, or calculating [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]
Figure 5
Figure 5. Figure 5: AI supports new researchers in learning and practicing QDA through personalized, interactive instructions, real-time [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: AI helps researchers resolve coding conflicts and reach consensus efficiently by analyzing discrepancies and suggesting [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]
Figure 7
Figure 7. Figure 7: AI helps identify errors and inconsistencies while keeping researchers actively engaged, e.g., by suggesting potentially [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: AI provides support by offering concise code definitions and names, e.g., for the final codebook, and suiting quotes [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: AI identifies patterns in coding and provides warnings to support decision-making, e.g., when codes overlap or themes [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]
Figure 10
Figure 10. Figure 10: AI analyzes the data and recommends codes which can be reviewed by the researcher and refined by the AI, [PITH_FULL_IMAGE:figures/full_fig_p018_10.png]
Figure 11
Figure 11. Figure 11: Researchers and AI collaborate interactively throughout the [PITH_FULL_IMAGE:figures/full_fig_p019_11.png]
Figure 12
Figure 12. Figure 12: AI takes the lead in QDA, e. g., creating a codebook draft (left) or performing initial coding (right), with human oversight throughout. AI with Human Approval. AI autonomously performs QDA tasks, but human approval is required before finalizing outputs. This ensures …
Figure 13
Figure 13. Figure 13: AI creates a codebook draft (top) or assigns codes (bottom). A researcher approves before decisions are finalized and [PITH_FULL_IMAGE:figures/full_fig_p020_13.png]
Figure 14
Figure 14. Figure 14: AI autonomously handles routine coding but alerts researchers when it encounters data requiring nuanced under [PITH_FULL_IMAGE:figures/full_fig_p020_14.png]
Figure 15
Figure 15. Figure 15: Four examples of coding workflow drawings created by our participants during interviews. The drawings varied considerably [PITH_FULL_IMAGE:figures/full_fig_p039_15.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Co-DETECT: Collaborative Discovery of Edge Cases in Text Classification

    cs.CL 2025-07 conditional novelty 6.0 of 10

    A mixed-initiative system where LLMs flag low-confidence annotations, cluster them, and propose codebook rules that human experts review and iterate on.

Reference graph

Works this paper leans on

92 extracted references · 49 canonical work pages · cited by 1 Pith paper

  1. [1]

    Icek Ajzen, Martin Fishbein, Sophie Lohmann, and Dolores Albarracín. 2019. The Influence of Attitudes on Behavior. InThe handbook of attitudes, Dolores Albarracín and Blair T. Johnson (Eds.). Routledge, New York, NY, USA

    2019

  2. [2]

    Shafiullah Anis and Juliana A French. 2023. Efficient, explicatory, and equitable: Why qualitative researchers should embrace AI, but cautiously. Business & Society62, 6 (2023), 1139–1144. https://doi.org/10.1177/00076503231163286

    work page doi:10.1177/00076503231163286 2023

  3. [3]

    Argyle, Ethan C

    Lisa P. Argyle, Ethan C. Busby, Nancy Fulda, Joshua Gubler, Christopher Rytting, and David Wingate. 2023. Out of one, many: Using language models to simulate human samples.Political Analysis31, 3 (2023), 337–351. https://doi.org/10.48550/ARXIV.2209.06899 arXiv:2209.06899

    arXiv doi:10.48550/arxiv.2209.06899 2023

  4. [4]

    Julian Ashwin, Aditya Chhabra, and Vijayendra Rao. 2023. Using large language models for qualitative analysis can introduce serious bias.arXiv preprint arXiv:2309.17147(2023)

    arXiv 2023

  5. [5]

    Christopher A Bail. 2024. Can Generative AI improve social science?Proceedings of the National Academy of Sciences121, 21 (2024). https: //doi.org/10.1073/pnas.2314021121

    doi:10.1073/pnas.2314021121 2024

  6. [6]

    Albert Bandura. 2012. On the Functional Properties of Perceived Self-Efficacy Revisited.Journal of Management38, 1 (2012), 9–44. https: //doi.org/10.1177/0149206311410606

    work page doi:10.1177/0149206311410606 2012

  7. [7]

    Muneera Bano, Didar Zowghi, and Jon Whittle. 2023. AI and Human Reasoning: Qualitative Research in the Age of Large Language Models.The AI Ethics Journal3, 1 (2023)

    2023

  8. [8]

    Muneera Bano, Didar Zowghi, and Jon Whittle. 2024. AI and Human Reasoning: Qualitative Research in the Age of Large Language Models.AI Ethics Journal4 (2024), 1–15. https://doi.org/10.47289/AIEJ20240122

    work page doi:10.47289/aiej20240122 2024

Show all 92 references
  1. [9]

    Amanda Barany, Nidhi Nasiar, Chelsea Porter, Andres Zambrano, Alexandra Andres, Dara Bright, Mamta Shah, Xiner Liu, Sabrina Gao, Jiayi Zhang, Shruti Mehta, Jaeyoon Choi, Camille Giordano, and Ryan Baker. 2024. ChatGPT for Education Research: Exploring the Potential of Large La...

    2024

  2. [10]

    Elaine Barnett-Page and James Thomas. 2009. Methods for the synthesis of qualitative research: a critical review.BMC medical research methodology 9 (2009), 1–11. https://doi.org/10.1186/1471-2288-9-59

    2009 doi

  3. [11]

    Eric PS Baumer, David Mimno, Shion Guha, Emily Quan, and Geri K Gay. 2017. Comparing grounded theory and topic modeling: Extreme divergence or unlikely convergence?Journal of the Association for Information Science and Technology68, 6 (2017), 1397–1410

    2017

  4. [12]

    Umang Bhatt and Holli Sargeant. 2024. When Should Algorithms Resign?CoRRabs/2209.06899 (2024), 1–5. https://doi.org/10.48550/arXiv.2402.18326 arXiv:2402.18326

    2024 doi arXiv

  5. [13]

    2012.Systematic Approaches to a Successful Literature Review

    Andrew Booth, Anthea Sutton, and Diana Papaioannou. 2012.Systematic Approaches to a Successful Literature Review. Sage Publications Limited, London, UK

    2012

  6. [14]

    Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology.Qualitative research in psychology3, 2 (2006), 77–101. https: //doi.org/10.1191/1478088706qp063oa

    2006 doi

  7. [15]

    Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, Harsha Nori, Hamid Palangi, Marco Tulio Ribeiro, and Yi Zhang. 2023. Sparks of Artificial General Intelligence: Early Experiments ...

    2023 arXiv

  8. [16]

    Zana Buçinca, Maja Barbara Malaya, and Krzysztof Z. Gajos. 2021. To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making.Proc. ACM Hum.-Comput. Interact.5, CSCW1, Article 188 (apr 2021), 21 pages. https://doi.org/10.1145/3449287

    2021 doi

  9. [17]

    Robert Chew, John Bollenbacher, Michael Wenger, Jessica Speer, and Annice Kim. 2023. LLM-assisted content analysis: Using large language models to support deductive coding.arXiv preprint arXiv:2306.14924abs/2402.18326 (2023), 1–27. https://doi.org/10.48550/ARXIV.2306.14924

    2023 doi arXiv

  10. [18]

    Clayton Cohn, Caitlin Snyder, Justin Montenegro, and Gautam Biswas. 2024. Towards A Human-in-the-Loop LLM Approach to Collaborative Discourse Analysis. InArtificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation T...

    2024 arXiv

  11. [19]

    Alejandro Cuevas, Jennifer V Scurrell, Eva M Brown, Jason Entenmann, and Madeleine IG Daepp. 2025. Collecting Qualitative Data at Scale with Large Language Models: A Case Study.Proceedings of the ACM on Human-Computer Interaction9, 2 (2025), 1–27. 26 Kirsten et al

    2025

  12. [20]

    Shih-Chieh Dai, Aiping Xiong, and Lun-Wei Ku. 2023. LLM-in-the-loop: Leveraging Large Language Model for Thematic Analysis. InFindings of the Association for Computational Linguistics, Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). Association for Computational Linguistics,...

    2023 doi

  13. [21]

    Stefano De Paoli. 2024. Performing an Inductive Thematic Analysis of Semi-Structured Interviews With a Large Language Model: An Exploration and Provocation on the Limits of the Approach.Social Science Computer Review42, 4 (Aug. 2024), 997–1019. https://doi.org/10.1177/08944393...

    2024 doi

  14. [22]

    Jakub Drápal, Hannes Westermann, and Jaromir Savelka. 2023. Using Large Language Models to Support Thematic Analysis in Empirical Legal Studies. arXiv:2310.18729 [cs]

    2023 arXiv

  15. [23]

    Margaret Drouhard, Nan-Chen Chen, Jina Suh, Rafal Kocielnik, Vanessa Peña-Araya, Keting Cen, Xiangyi Zheng, and Cecilia R. Aragon. 2017. Aeonium: Visual Analytics to Support Collaborative Qualitative Coding. InIEEE Pacific Visualization Symposium (PacificVis). IEEE Computer So...

    2017

  16. [24]

    Ferguson and John A

    Melissa J. Ferguson and John A. Bargh. 2024. How social perception can automatically influence behavior.TRENDS in Cognitive Sciences8, 1 (2024), 33–39

    2024

  17. [25]

    Feuston and Jed R

    Jessica L. Feuston and Jed R. Brubaker. 2021. Putting Tools in Their Place: The Role of Time and Perspective in Human-AI Collaboration for Qualitative Analysis.Proc. ACM Hum.-Comput. Interact.5, CSCW2 (Oct. 2021), 469:1–469:25. https://doi.org/10.1145/3479856

    2021 doi

  18. [26]

    Casey Fiesler, Jed R Brubaker, Andrea Forte, Shion Guha, Nora McDonald, and Michael Muller. 2019. Qualitative methods for CSCW: Challenges and opportunities. InCompanion Publication of the 2019 Conference on Computer Supported Cooperative Work and Social Computing. 455–460

    2019

  19. [27]

    McDonald

    Abbas Ganji, Mania Orand, and David W. McDonald. 2018. Ease on Down the Code: Complex Collaborative Qualitative Coding Simplified with ’Code Wizard’.Proc. ACM Hum.-Comput. Interact.2, CSCW (Nov. 2018), 132:1–132:24. https://doi.org/10.1145/3274401

    2018 doi

  20. [28]

    Jie Gao, Kenny Tsu Wei Choo, Junming Cao, Roy Ka-Wei Lee, and Simon Perrault. 2023. CoAIcoder: Examining the Effectiveness of AI-assisted Human-to-Human Collaboration in Qualitative Analysis.ACM Trans. Comput.-Hum. Interact.31, 1 (2023), 6:1–6:38. https://doi.org/10.1145/3617362

    2023 doi

  21. [29]

    Jie Gao, Yuchen Guo, Gionnieve Lim, Tianqin Zhang, Zheng Zhang, Toby Jia-Jun Li, and Simon Tangi Perrault. 2024. CollabCoder: a lower-barrier, rigorous workflow for inductive collaborative qualitative analysis with large language models. InProceedings of the CHI Conference on ...

    2024

  22. [30]

    Robert P Gauthier and James R Wallace. 2022. The computational thematic analysis toolkit.Proceedings of the ACM on Human-Computer Interaction 6, GROUP (2022), 1–15

    2022

  23. [31]

    2012.Cognitive consistency: A fundamental principle in social cognition

    Bertram Gawronski and Fritz Strack (Eds.). 2012.Cognitive consistency: A fundamental principle in social cognition. Guilford Press, New York, NY. http://site.ebrary.com/lib/alltitles/docDetail.action?docID=10527228

    2012

  24. [32]

    Fabrizio Gilardi, Meysam Alizadeh, and Maël Kubli. 2023. ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks.Proceedings of the National Academy of Sciences120, 30 (2023), e2305016120. https://doi.org/10.1073/pnas.2305016120 arXiv:2303.15056 [cs]

    2023 arXiv

  25. [33]

    Barney Glaser and Anselm Strauss. 1967. Grounded theory: The discovery of grounded theory.Sociology the journal of the British sociological association12, 1 (1967), 27–49

    1967

  26. [34]

    Perttu Hämäläinen, Mikke Tavast, and Anton Kunnari. 2023. Evaluating large language models in generating synthetic hci research data: a case study. InProceedings of the 2023 CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1–19

    2023

  27. [35]

    Leah Hamilton, Desha Elliott, Aaron Quick, Simone Smith, and Victoria Choplin. 2023. Exploring the Use of AI in Qualitative Analysis: A Comparative Study of Guaranteed Income Data.International Journal of Qualitative Methods22 (2023), 16094069231201504. https://doi.org/10.1177...

    2023

  28. [36]

    David Hammer. 1996. More than misconceptions: Multiple perspectives on student knowledge and reasoning, and an appropriate role for education research.American Journal of Physics64, 10 (1996), 1316–1325. https://doi.org/10.1119/1.18376

    1996 doi

  29. [37]

    Ayako A Hasegawa, Daisuke Inoue, and Mitsuaki Akiyama. 2024. How{WEIRD} is Usable Privacy and Security Research?. In33rd USENIX Security Symposium (USENIX Security 24). 3241–3258

    2024

  30. [38]

    2012.Kreatives Prozessdesign: Konzepte und Methoden zur Integration von Prozessorganisation, Technik und Arbeitsgestaltung

    Thomas Herrmann. 2012.Kreatives Prozessdesign: Konzepte und Methoden zur Integration von Prozessorganisation, Technik und Arbeitsgestaltung. Springer, Heidelberg. https://doi.org/10.1007/978-3-642-24370-7

    2012 doi

  31. [39]

    Michael Heseltine and Bernhard Clemm von Hohenberg. 2024. Large language models as a substitute for human experts in annotating political text. Research & Politics11, 1 (2024)

    2024

  32. [40]

    Danielle Hitch. 2024. Artificial Intelligence Augmented Qualitative Analysis: The Way of the Future?Qualitative Health Research34, 7 (2024), 595–606

    2024

  33. [41]

    Heesoo Jang, Narayanamoorthy Nanditha, Laura Schelenz, Lou Therese Brandner, Anne Burkhardt, Simon David Hirsbrunner, and Scott Timcke

  34. [42]

    Brubaker

    Jialun Aaron Jiang, Kandrea Wade, Casey Fiesler, and Jed R. Brubaker. 2021. Supporting Serendipity: Opportunities and Challenges for Human-AI Collaboration in Qualitative Analysis.Proc. ACM Hum.-Comput. Interact.5, CSCW1 (2021), 94:1–94:23. https://doi.org/10.1145/3449168

    2021 doi

  35. [44]

    I’m categorizing LLM as a productivity tool

    Shivani Kapania, Ruiyi Wang, Toby Jia-Jun Li, Tianshi Li, and Hong Shen. 2024. "I’m categorizing LLM as a productivity tool": Examining ethics of LLM use in HCI research practices.arXiv preprint arXiv:2403.19876(2024). https://arxiv.org/abs/2403.19876 A Researcher Study on the...

    2024 arXiv

  36. [45]

    Erin Kenneally and David Dittrich. 2012. The menlo report: Ethical principles guiding information and communication technology research. A vailable at SSRN 244510210, 2 (2012), 71–75. https://catalog.caida.org/paper/2012_menlo_report_actual_formatt

    2012

  37. [46]

    Mahdi Khalili. 2023. Against the opacity, and for a qualitative understanding, of artificially intelligent technologies.AI and Ethics(2023), 1–9

    2023

  38. [47]

    Elisabeth Kirsten, Annalina Buckmann, Abraham Mhaidli, and Steffen Becker. 2024. Decoding Complexity: Exploring Human-AI Concordance in Qualitative Coding. , 6 pages. arXiv:2403.06607 https://arxiv.org/abs/2403.06607

    2024 arXiv

  39. [48]

    Neha Kumar and Naveena Karusala. 2021. Braving citational justice in human-computer interaction. InExtended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. 1–9

    2021

  40. [49]

    Michelle S Lam, Janice Teoh, James A Landay, Jeffrey Heer, and Michael S Bernstein. 2024. Concept induction: Analyzing unstructured text with high-level concepts using lloom. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–28

    2024

  41. [50]

    Robert P Lennon, Robbie Fraleigh, Lauren J Van Scoy, Aparna Keshaviah, Xindi C Hu, Bethany L Snyder, Erin L Miller, William A Calo, Aleksandra E Zgierska, and Christopher Griffin. 2021. Developing and Testing an Automated Qualitative Assistant (AQUA) to Support Qualitative Ana...

    2021 doi

  42. [51]

    Sebastian Linxen, Christian Sturm, Florian Brühlmann, Vincent Cassau, Klaus Opwis, and Katharina Reinecke. 2021. How weird is CHI?. In Proceedings of the 2021 chi conference on human factors in computing systems. 1–14

    2021

  43. [52]

    Brian Lubars and Chenhao Tan. 2019. Ask not what AI can do, but what AI should do: Towards a framework of task delegability.Advances in neural information processing systems32 (2019), 57–67. https://proceedings.neurips.cc/paper/2019/hash/d67d8ab4f4c10bf22aa353e27879133c-Abstract.html

    2019

  44. [53]

    Megh Marathe and Kentaro Toyama. 2018. Semi-Automated Coding for Qualitative Research: A User-Centered Inquiry and Initial Prototypes. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, Montreal QC Canada, 1–12. https://doi.org/10.1145/317357...

    2018 doi

  45. [54]

    Nora McDonald, Sarita Schoenebeck, and Andrea Forte. 2019. Reliability and inter-rater reliability in qualitative research: Norms and guidelines for CSCW and HCI practice.Proceedings of the ACM on human-computer interaction3, CSCW (2019), 1–23

    2019

  46. [55]

    David L. Morgan. 2023. Exploring the Use of Artificial Intelligence for Qualitative Data Analysis: The Case of ChatGPT.International Journal of Qualitative Methods22 (2023), 16094069231211248. https://doi.org/10.1177/16094069231211248

    2023 doi

  47. [56]

    Michael Muller, Shion Guha, Eric PS Baumer, David Mimno, and N Sadat Shami. 2016. Machine learning and grounded theory method: convergence, divergence, and combination. InProceedings of the 2016 ACM International Conference on Supporting Group Work. 3–8

    2016

  48. [57]

    Chinasa T Okolo, Nicola Dell, and Aditya Vashistha. 2022. Making AI explainable in the Global South: A systematic review. InProceedings of the 5th ACM SIGCAS/SIGCHI Conference on Computing and Sustainable Societies. 439–452

    2022

  49. [58]

    Jonas Oppenlaender and Joonas Hämäläinen. 2023. Mapping the challenges of HCI: An application and evaluation of ChatGPT and GPT-4 for cost-efficient question answering.arXiv preprint arXiv:2306.05036(2023)

    2023

  50. [59]

    Anna-Marie Ortloff, Matthias Fassl, Alexander Ponticello, Florin Martius, Anne Mertens, Katharina Krombholz, and Matthew Smith. 2023. Different Researchers, Different Results? Analyzing the Influence of Researcher Experience and Data Type During Qualitative Analysis of an Inte...

    2023

  51. [60]

    Kwame Owoahene Acheampong and Matthew Nyaaba. 2024. Review of Qualitative Research in the Era of Generative Artificial Intelligence.Review of Qualitative Research in the Era of Generative Artificial Intelligence (January 7, 2024)(2024)

    2024

  52. [61]

    Dustin Palea, Giridhar Vadhul, and David T Lee. 2024. Annota: Peer-based AI Hints Towards Learning Qualitative Coding at Scale. InProceedings of the 29th International Conference on Intelligent User Interfaces. ACM, New York, NY, USA, 455–470. https://doi.org/10.1145/3640543.3645168

    2024

  53. [62]

    Palinkas, Sarah McCue Horwitz, Carla A Green, Jennifer P

    Lawrence A. Palinkas, Sarah McCue Horwitz, Carla A Green, Jennifer P. Wisdom, Naihua Duan, and Kimberly Eaton Hoagwood. 2015. Purposeful Sampling for Qualitative Data Collection and Analysis in Mixed Method Implementation Research.Administration and Policy in Mental Health and...

    2015 doi

  54. [63]

    In Minutes Instead of Weeks

    Trena M Paulus and Vittorio Marone. 2024. “In Minutes Instead of Weeks”: Discursive Constructions of Generative AI and Qualitative Data Analysis. Qualitative Inquiry(2024)

    2024

  55. [64]

    Zeeshan Rasheed, Muhammad Waseem, Aakash Ahmad, Kai-Kristian Kemell, Wang Xiaofeng, Anh Nguyen Duc, and Pekka Abrahamsson. 2024. Can Large Language Models Serve as Data Analysts? A Multi-Agent Assisted Approach for Qualitative Data Analysis. arXiv:2402.01386 [cs]

    2024

  56. [65]

    Riessman

    C.K. Riessman. 2008.Narrative methods for the human sciences. Sage Publications Limited, London, UK

    2008

  57. [66]

    Tim Rietz and Alexander Maedche. 2021. Cody: An AI-Based System to Semi-Automate Coding for Qualitative Research. InProceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI ’21). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi...

    2021

  58. [67]

    Mohammad Rashidujjaman Rifat, Ayesha Bhimdiwala, Ananya Bhattacharjee, Amna Batool, Dipto Das, Nusrat Jahan Mim, Abdullah Hasan Safir, Sharifa Sultana, Taslima Akter, C Estelle Smith, et al. 2023. Many Worlds of Ethics: Ethical Pluralism in CSCW. InCompanion Publication of the...

    2023

  59. [68]

    Nadine B Sarter and David D Woods. 1995. How in the world did we ever get into that mode? Mode error and awareness in supervisory control. Human factors37, 1 (1995), 5–19

    1995

  60. [69]

    Albrecht Schmidt, Passant Elagroudy, Fiona Draxler, Frauke Kreuter, and Robin Welsch. 2024. Simulating the human in HCD with ChatGPT: Redesigning interaction design with AI.Interactions31, 1 (2024), 24–31. 28 Kirsten et al

    2024

  61. [70]

    Orit Shaer, Angelora Cooper, Osnat Mokryn, Andrew L Kun, and Hagit Ben Shoshan. 2024. AI-Augmented Brainwriting: Investigating the use of LLMs in group ideation. InProceedings of the CHI Conference on Human Factors in Computing Systems. 1–17

    2024

  62. [71]

    Yonadav Shavit, Sandhini Agarwal, Miles Brundage, Steven Adler, Cullen O’Keefe, Rosie Campbell, Teddy Lee, Pamela Mishkin, Tyna Eloundou, Alan Hickey, et al. 2023. Practices for governing agentic AI systems.Research Paper, OpenAI, December(2023)

    2023

  63. [72]

    Ben Shneiderman. 2020. Human-centered artificial intelligence: Reliable, safe & trustworthy.International Journal of Human–Computer Interaction 36, 6 (2020), 495–504

    2020

  64. [73]

    Stone, Robert F

    Philip J. Stone, Robert F. Bales, J. Zvi Namenwirth, and Daniel M. Ogilvie. 1962. The General Inquirer: A Computer System for Content Analysis and Retrieval Based on the Sentence as a Unit of Information.Behavioral Science7, 4 (1962), 484–498

    1962

  65. [74]

    Christian Sturm, Alice Oh, Sebastian Linxen, Jose Abdelnour Nocera, Susan Dray, and Katharina Reinecke. 2015. How WEIRD is HCI? Extending HCI principles to other countries and cultures. InProceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Comp...

    2015

  66. [75]

    Viktor Suter and Miriam Meckel. 2024. Using GPT-4 for Text Analysis: Insights from English and German Language News Classification Tasks. Proceedings der International Conference on Web and Social Media (ICWSM) 2024(2024)

    2024

  67. [76]

    Tai, Lillian R

    Robert H. Tai, Lillian R. Bentley, Xin Xia, Jason M. Sitt, Sarah C. Fankhauser, Ana M. Chicas-Mosier, and Barnas G. Monteith. 2024. An Examination of the Use of Large Language Models to Aid Analysis of Textual Data. , 2023.07.17.549361 pages. https://doi.org/10.1101/2023.07.17.549361

    2024 doi

  68. [77]

    Maya Grace Torii, Takahito Murakami, and Yoichi Ochiai. 2024. Expanding Horizons in HCI Research Through LLM-Driven Qualitative Analysis. CoRRabs/2401.04138 (2024), 1–9. https://doi.org/10.48550/ARXIV.2401.04138 arXiv:2401.04138

    2024 doi arXiv

  69. [78]

    Petter Törnberg. 2023. ChatGPT-4 Outperforms Experts and Crowd Workers in Annotating Political Twitter Messages with Zero-Shot Learning. https://arxiv.org/abs/2304.06588v1

    2023 arXiv

  70. [79]

    Petter Törnberg. 2023. How to use llms for text analysis.CoRR(2023). https://doi.org/10.48550/ARXIV.2307.13106

    2023 doi arXiv

  71. [80]

    Usman Ahmad Usmani, Ari Happonen, and Junzo Watada. 2023. Human-centered artificial intelligence: Designing for user empowerment and ethical considerations. In2023 5th international congress on human-computer interaction, optimization and robotic applications (HORA). IEEE, 1–7

    2023

  72. [81]

    Chat Wacharamanotham, Lukas Eisenring, Steve Haroz, and Florian Echtler. 2020. Transparency of CHI research artifacts: Results of a self-reported survey. InProceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–14

    2020

  73. [82]

    James R Wallace, Saba Oji, and Craig Anslow. 2017. Technologies, methods, and values: changes in empirical research at CSCW 1990-2015. Proceedings of the ACM on Human-Computer Interaction1, CSCW (2017), 1–18

    2017

  74. [84]

    Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus

    Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. 2022. Emergent Abilities of Large Language Mod...

    2022 arXiv

  75. [85]

    Sue Wilkinson. 2000. Women with breast cancer talking causes: Comparing content, biographical and discursive analyses.Feminism & Psychology 10, 4 (2000), 431–460. https://doi.org/10.1177/0959353500010004003

    2000 doi

  76. [86]

    Yijun Xiao and William Yang Wang. 2021. On Hallucination and Predictive Uncertainty in Conditional Language Generation.CoRRabs/2103.15025 (2021), 2734–2744. arXiv:2103.15025 https://arxiv.org/abs/2103.15025

    2021 arXiv

  77. [87]

    Vera Liao, Rania Abdelghani, and Pierre-Yves Oudeyer

    Ziang Xiao, Xingdi Yuan, Q. Vera Liao, Rania Abdelghani, and Pierre-Yves Oudeyer. 2023. Supporting Qualitative Analysis with Large Language Models: Combining Codebook with GPT-3 for Deductive Coding. In28th International Conference on Intelligent User Interfaces. ACM, New York...

    2023

  78. [88]

    Wei Xu. 2007. Identifying problems and generating recommendations for enhancing complex systems: Applying the abstraction hierarchy framework as an analytical tool.Human factors49, 6 (2007), 975–994

    2007

  79. [89]

    Wei Xu, Marvin J Dainoff, Liezhong Ge, and Zaifeng Gao. 2023. Transitioning to human interaction with AI systems: New challenges and opportunities for HCI professionals to enable human-centered AI.International Journal of Human–Computer Interaction39, 3 (2023), 494–518

    2023

  80. [90]

    He Zhang, Chuhao Wu, Jingyi Xie, ChanMin Kim, and John M. Carroll. 2023. QualiGPT: GPT as an Easy-to-Use Tool for Qualitative Coding. https://doi.org/10.48550/arXiv.2310.07061 arXiv:2310.07061 [cs]

    2023 doi arXiv

  81. [91]

    He Zhang, Chuhao Wu, Jingyi Xie, Yao Lyu, Jie Cai, and John M. Carroll. 2024. Redefining Qualitative Analysis in the AI Era: Utilizing ChatGPT for Efficient Thematic Analysis. arXiv:2309.10771 [cs]

    2024 arXiv

  82. [92]

    Caleb Ziems, William Held, Omar Shaikh, Jiaao Chen, Zhehao Zhang, and Diyi Yang. 2024. Can Large Language Models Transform Computational Social Science? https://doi.org/10.48550/arXiv.2305.03514 arXiv:2305.03514 [cs]

    2024 doi arXiv

  83. [93]

    Google Docs (nice collaboration features)

    Caleb Ziems, William Held, Omar Shaikh, Jiaao Chen, Zhehao Zhang, and Diyi Yang. 2024. Can large language models transform computational social science?Computational Linguistics50, 1 (2024), 237–291. https://doi.org/10.1162/COLI_A_00502 A Researcher Study on the Potential of A...

    2024 doi

  84. [2023]

    InCompanion Publication of the 2023 Conference on Computer Supported Cooperative Work and Social Computing

    Platform (In) Justice: A Call for a Global Research Agenda. InCompanion Publication of the 2023 Conference on Computer Supported Cooperative Work and Social Computing. 411–414

    2023

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.