pith. machine review for the scientific record. sign in

arxiv: 2605.00922 · v1 · submitted 2026-04-30 · 💻 cs.SE · cs.AI

Recognition: unknown

To Vibe Research or Not to Vibe Research? Generative AI in Qualitative Research

Authors on Pith no claims yet

Pith reviewed 2026-05-09 20:28 UTC · model grok-4.3

classification 💻 cs.SE cs.AI
keywords generative AIqualitative researchsoftware engineeringresearch philosophysmall-qBig Qethics in research
0
0 comments X

The pith

Generative AI suits qualitative research mainly when the approach is positivist small-q rather than non-positivist Big Q.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper reviews the debate on whether generative AI belongs in qualitative research and spells out what this means for software engineering researchers. A sympathetic reader would see that the small-q positivist approach versus the Big Q non-positivist approach forms the main dividing line for AI acceptance. Skills in using the tools, ethical issues around data and bias, and personal researcher preferences add further layers to the choice. Understanding these factors helps researchers maintain rigor when AI offers speed in coding and analysis but risks undermining interpretive work.

Core claim

The qualitative research approach, small-q (positivist or post-positivist) or Big Q (non-positivist), is among the major criteria for determining whether generative AI can be used in qualitative research. In addition to research philosophy and research approach, skills, ethics, and personal preferences also play a role in researchers' decisions about whether to use AI in qualitative research.

What carries the argument

The small-q versus Big Q classification of qualitative research approaches, which ties AI compatibility to the underlying research philosophy.

If this is right

  • Software engineering researchers using small-q approaches can more readily incorporate generative AI for tasks like data coding or summarization.
  • Big Q researchers should exercise greater caution or avoid AI to maintain interpretive depth and researcher involvement.
  • Ethical considerations around data privacy and AI bias must be addressed regardless of approach.
  • Developing AI literacy becomes essential for researchers deciding on tool use.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • This framework might encourage software engineering teams to declare their qualitative stance upfront when publishing AI-assisted studies.
  • Neighbouring fields like information systems research could adopt similar distinctions to guide AI tool policies.
  • A testable extension would be to compare AI-assisted and fully manual qualitative analyses in SE papers for differences in findings depth.

Load-bearing premise

Existing literature provides a neutral and complete summary of views on generative AI in qualitative research, and that philosophy, skills, ethics, and preferences are the primary factors researchers weigh.

What would settle it

A survey of software engineering researchers finding no consistent difference in generative AI adoption between small-q and Big Q users would challenge the central criterion.

Figures

Figures reproduced from arXiv: 2605.00922 by Jussi Kasurinen, Kari Smolander, Katja Karhu.

Figure 1
Figure 1. Figure 1: AI as a parallel researcher computer science or formal methods background, qualitative research is often viewed through a small-q lens. This is because of the strong quantitative tra￾dition in software engineering [17, 30]. Therefore, although the small-q and Big-Q division of qualitative research did not originate in the software engineer￾ing context, it describes the ”opposing forces” at play in qualitat… view at source ↗
Figure 2
Figure 2. Figure 2: AI as a layer between researcher and subject [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
read the original abstract

There has been intense debate among qualitative researchers about whether generative AI is suitable for qualitative research. In this paper, we summarize the broader ongoing discussion of generative AI in qualitative research and its implications for software engineering researchers. The qualitative research approach, small-q (positivist or post-positivist) or Big Q (non-positivist), is among the major criteria for determining whether generative AI can be used in qualitative research. In addition to research philosophy and research approach, skills, ethics, and personal preferences also play a role in researchers' decisions about whether to use AI in qualitative research.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

0 major / 2 minor

Summary. The paper summarizes the ongoing debate among qualitative researchers on the suitability of generative AI for qualitative research and discusses its implications specifically for software engineering researchers. It identifies the distinction between small-q (positivist or post-positivist) and Big Q (non-positivist) qualitative approaches as a primary criterion for deciding whether generative AI can be appropriately used. The paper further states that researchers' skills, ethical considerations, and personal preferences also factor into such decisions.

Significance. If the literature synthesis is accurate and balanced, the paper could provide a helpful entry point for software engineering researchers navigating the integration of generative AI into qualitative work by distilling key philosophical and practical considerations from the broader methods literature. Its value lies in contextualizing an active debate for a specific disciplinary audience rather than in presenting new empirical findings or formal derivations.

minor comments (2)
  1. The abstract and title use informal phrasing (e.g., 'To Vibe Research or Not to Vibe Research?') that may benefit from a more conventional academic tone or explicit definition of 'vibe research' to ensure accessibility for readers unfamiliar with the term.
  2. Consider adding a brief table or structured list in the main text that maps the small-q/Big Q distinction to specific generative AI use cases (e.g., coding, theme generation) to make the central criterion more concrete and actionable for software engineering practitioners.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for their review and for recommending minor revision. The referee's summary accurately captures the manuscript's contribution as a synthesis of the ongoing debate on generative AI in qualitative research, with specific implications for software engineering researchers. We appreciate the recognition of its potential value as an entry point for the disciplinary audience.

Circularity Check

0 steps flagged

No significant circularity; literature summary is self-contained

full rationale

The paper presents a summary of ongoing external debate on generative AI in qualitative research and its implications for software engineering. It advances no equations, derivations, fitted parameters, hypotheses, or novel empirical results. The central claim—that small-q vs. Big Q approaches plus skills/ethics/preferences influence AI use—is explicitly framed as a distillation of existing literature rather than a constructed output. No self-citation chains, ansatzes, or renamings appear in the provided text, and the work does not reduce any result to its own inputs by definition. This is the expected finding for a non-derivational discussion paper.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the standard distinction between small-q and Big Q qualitative research paradigms drawn from prior literature in the field; no new free parameters, axioms invented for this paper, or postulated entities are introduced.

axioms (1)
  • domain assumption The distinction between small-q (positivist or post-positivist) and Big Q (non-positivist) qualitative research approaches is a valid and major criterion for evaluating AI use.
    Invoked directly in the abstract as one of the primary determinants.

pith-pipeline@v0.9.0 · 5396 in / 1283 out tokens · 39208 ms · 2026-05-09T20:28:05.672113+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

295 extracted references · 248 canonical work pages

  1. [1]

    Lee, A. S. and Liebenau, J. , doi =. Information. doi:10.10 07/978-0-387-35309-8_1 , abstract =

  2. [2]

    , month = mar, year =

    Friese, Susanne and Nguyen-Trung, Kien and Powell, Steve and Morgan, David L. , month = mar, year =. Beyond. Qualitative Inquiry , publisher =. doi:10.1177/10778004261429393 , abstract =

  3. [3]

    Organizational Research Methods , author =

    Generative. Organizational Research Methods , author =. 2026 , note =. doi:10.1177/10944281251377154 , abstract =

  4. [4]

    Glaser, BG and Strauss, AL , year =. The

  5. [5]

    Engaged and

    Nguyen, Duc Cuong and Welch, Catherine , year =. Engaged and. doi:10.1177/00076503251386539 , journal =

  6. [6]

    and Kasperiuniene, Judita , month = apr, year =

    Costa, António Pedro and Bryda, Grzegorz and Christou, Prokopis A. and Kasperiuniene, Judita , month = apr, year =. International Journal of Qualitative Methods , publisher =. doi:10.1177/16094069251383739 , abstract =

  7. [7]

    AI Is Turning Research into a Scientific Monoculture

    Traberg, Cecilie Steenbuch and Roozenbeek, Jon and van der Linden, Sander , month = feb, year =. Communications Psychology , publisher =. doi:10.1038/s44271-026-00428-5 , abstract =

  8. [8]

    The Qualitative Report , author =

    How to. The Qualitative Report , author =. 2016 , pages =. doi:10.46743/2160-3715/2016.2361 , number =

  9. [9]

    In Proceedings of the 34th International Conference on Neural Information Processing Systems (NIPS '20)

    Vikan, Magnhild and Aryan, Ramtin and Kannelønning, Mari Serine and Riegler, Michael Alexander and Danielsen, Stein Ove , month = sep, year =. Reflecting on. Qualitative Health Research , publisher =. doi:10.1177/10497323251365211 , abstract =

  10. [10]

    and Lincoln, Yvonna S

    Guba, Egon G. and Lincoln, Yvonna S. , year =. Competing paradigms in qualitative research , isbn =. Handbook of qualitative research , publisher =

  11. [11]

    and Baskerville, Richard L

    Lee, Allen S. and Baskerville, Richard L. , month = sep, year =. Generalizing. Information Systems Research , publisher =. doi:10.1287/isre.14.3.221.16560 , abstract =

  12. [12]

    Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society , year =

    Methodological. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society , author =. 2025 , pages =. doi:10.1609/aies.v8i2.36667 , abstract =

  13. [13]

    Advances in Methods and Practices in Psychological Science , author =

    A. Advances in Methods and Practices in Psychological Science , author =

  14. [14]

    It might be able to produce a text, but it’s not able to produce knowledge: qualitative researchers’ perspectives on the use of

    Wheeler, Kathryn , month = jan, year =. Technological reflexivity in practice: how. Qualitative Research in Psychology , publisher =. doi:10.1080/14780887.2025.2602820 , abstract =

  15. [15]

    Response to: "

    Friese, Susanne , keywords =. Response to: ". 2025 , publisher =

  16. [16]

    Enhancing qualitative research in psychology with large language models: a methodological exploration and examples of simulations , volume =

    Braun, Virginia and Clarke, Victoria , month = apr, year =. Reporting guidelines for qualitative research: a values-based approach , volume =. Qualitative Research in Psychology , publisher =. doi:10.1080/14780887.2024.2382244 , abstract =

  17. [17]

    Supporting best practice in reflexive thematic analysis reporting in

    Braun, Virginia and Clarke, Victoria , month = jun, year =. Supporting best practice in reflexive thematic analysis reporting in. Palliative Medicine , publisher =. doi:10.1177/02692163241234800 , abstract =

  18. [18]

    American Journal of Pharmaceutical Education , author =

    A. American Journal of Pharmaceutical Education , author =. 2020 , keywords =. doi:10.5688/ajpe7120 , abstract =

  19. [19]

    The American Journal of Occupational Therapy , author =

    Rigor in. The American Journal of Occupational Therapy , author =. 1991 , pages =. doi:10.5014/ajot.45.3.214 , abstract =

  20. [20]

    Advances in Nursing Science , author =

    The problem of rigor in qualitative research , volume =. Advances in Nursing Science , author =. 1986 , pages =

  21. [21]

    and Gaskell, George and Allum, Nicholas C

    W.Bauer, Martin and Gaskell, George and Bauer, Martin W. and Gaskell, George and Allum, Nicholas C. , year =. Quality,. Qualitative. doi:10.4135/9781849209731 , pages =

  22. [22]

    Intercoder Reliability in Qualitative Research: Debates and Practical Guidelines

    O’Connor, Cliodhna and Joffe, Helene , month = jan, year =. Intercoder. International Journal of Qualitative Methods , publisher =. doi:10.1177/1609406919899220 , abstract =

  23. [23]

    Nemukula, Lufuno and Jegede, Oluseye , editor =. Factors. Digital. 2025 , keywords =. doi:10.1007/978-3-032-00444-4_28 , abstract =

  24. [24]

    Documenting debates on

    Nguyen-Trung, Kien , month = nov, year =. Documenting debates on. doi:10.2139/ssrn.5750283 , abstract =

  25. [25]

    2025 , keywords =

    Quality & Quantity , author =. 2025 , keywords =. doi:10.1007/s11135-025-02165-z , abstract =

  26. [26]

    Positioning

    Yang, Nan and Wang, Xiaofeng and Hyrynsalmi, Sami , month = feb, year =. Positioning. doi:10.5281/zenodo.18454000 , abstract =

  27. [27]

    , month = oct, year =

    Morgan, David L. , month = oct, year =. Exploring the. International Journal of Qualitative Methods , publisher =. doi:10.1177/16094069231211248 , abstract =

  28. [28]

    and Dowling, Nicki A

    Bijker, Rimke and Merkouris, Stephanie S. and Dowling, Nicki A. and Rodda, Simone N. , month = jul, year =. Journal of Medical Internet Research , publisher =. doi:10.2196/59050 , abstract =

  29. [29]

    , month = apr, year =

    Hayes, Adam S. , month = apr, year =. “. International Journal of Qualitative Methods , publisher =. doi:10.1177/16094069251322346 , abstract =

  30. [30]

    Kim, Chaewon and Ke, Fengfeng and Barrett, Alex and Zhang, Nuodi , editor =. From. Artificial. 2025 , keywords =. doi:10.1007/978-3-031-99264-3_4 , abstract =

  31. [31]

    Sinha, Ravi and Solola, Idris and Nguyen, Ha and Swanson, Hillary and Lawrence, LuEttaMae , month = jun, year =. The. Proceedings of the 2024. doi:10.1145/3663433.3663456 , abstract =

  32. [32]

    Friese, Susanne , month = jan, year =. From. Qualitative Inquiry , publisher =. doi:10.1177/10778004251412871 , abstract =

  33. [33]

    , year =

    Levitt, Heidi M. , year =. A consideration of the ethics and methodological integrity of generative artificial intelligence in qualitative research:. Qualitative Psychology , publisher =. doi:10.1037/qup0000353 , abstract =

  34. [34]

    On methodological incongruence in applying

    Nguyen-Trung, Kien and Friese, Susanne , month = dec, year =. On methodological incongruence in applying. doi:10.2139/ssrn.5874482 , abstract =

  35. [35]

    It might be able to produce a text, but it’s not able to produce knowledge: qualitative researchers’ perspectives on the use of

    Foley, Sarah and Dempsey, Maria and O’Sullivan, Catherine and Frost, Nollaig , year =. It might be able to produce a text, but it’s not able to produce knowledge: qualitative researchers’ perspectives on the use of. Qualitative Research in Psychology , publisher =. doi:10.1080/14780887.2025.2602816 , number =

  36. [36]

    Clarke, Victoria and Braun, Virginia and Adams, Jeffery and Callaghan, Jane E. M. and LaMarre, Andrea and Semlyen, Joanna , year =. “. Qualitative Psychology , publisher =. doi:10.1037/qup0000322 , abstract =

  37. [37]

    Review of

    Owoahene Acheampong, Kwame and Nyaaba, Matthew , month = jan, year =. Review of. doi:10.2139/ssrn.4686920 , abstract =

  38. [38]

    Jowsey, Tanisha and Braun, Virginia and Clarke, Victoria and Lupton, Deborah and Fine, Michelle , month = dec, year =. We. Qualitative Inquiry , publisher =. doi:10.1177/10778004251401851 , abstract =

  39. [39]

    Frankenstein, thematic analysis and generative artificial intelligence:

    Jowsey, Tanisha and Stapleton, Peta and Campbell, Shawna and Davidson, Alexandra and McGillivray, Cher and Maugeri, Isabella and Lee, Megan and Keogh, Justin , month = sep, year =. Frankenstein, thematic analysis and generative artificial intelligence:. PLOS ONE , publisher =. doi:10.1371/journal.pone.0330217 , abstract =

  40. [40]

    Journal for General Philosophy of Science , author =

    Towards a. Journal for General Philosophy of Science , author =. 2008 , keywords =. doi:10.1007/s10838-008-9068-7 , abstract =

  41. [41]

    Exploring the industry's challenges in software testing:

  42. [42]

    Studies on the

    Capretz, Luiz Fernando and Waychal, Pradeep and Jia, Jingdong and Varona, Daniel and Lizama, Yadira , month = may, year =. Studies on the. 2019. doi:10.1109/ICSE-Companion.2019.00105 , abstract =

  43. [43]

    IEEE Transactions on Engineering Management , author =

    An. IEEE Transactions on Engineering Management , author =. 2011 , keywords =. doi:10.1109/TEM.2010.2090730 , abstract =

  44. [44]

    Information and Software Technology , author =

    Challenges and strategies for motivating software testing personnel , volume =. Information and Software Technology , author =. 2016 , keywords =. doi:10.1016/j.infsof.2016.01.002 , abstract =

  45. [45]

    Ramler, Rudolf and Biffl, Stefan and Grünbacher, Paul , editor =. Value-. Value-. 2006 , keywords =. doi:10.1007/3-540-29263-2_11 , abstract =

  46. [46]

    Andriadi, Kus and Soeparno, Haryono and Gaol, Ford Lumban and Arifin, Yulyani , month = aug, year =. The. 2023. doi:10.1109/ICIMTech59029.2023.10277919 , abstract =

  47. [47]

    Medium , author =

    Shifting. Medium , author =

  48. [48]

    Research on software testing techniques and software automation testing tools , url =

    Sneha, Karuturi and Malle, Gowda M , month = aug, year =. Research on software testing techniques and software automation testing tools , url =. 2017. doi:10.1109/ICECDS.2017.8389562 , abstract =

  49. [49]

    Socio-technical management of software testing in information systems projects/

    Tosetto, Mauro and Bellini, Carlo Gabriel Porto , month = apr, year =. Socio-technical management of software testing in information systems projects/. Journal of Information Systems & Technology Management , publisher =

  50. [50]

    An instrument to measure factors that constitute the socio-technical context of testing experience , url =

    Swillus, Mark and Brandt, Carolin and Zaidman, Andy , month = may, year =. An instrument to measure factors that constitute the socio-technical context of testing experience , url =. doi:10.48550/arXiv.2505.01171 , abstract =

  51. [51]

    Who cares about testing?:

    Swillus, Mark and Hoda, Rashina and Zaidman, Andy , month = oct, year =. Who cares about testing?:. doi:10.48550/arXiv.2504.07208 , abstract =

  52. [52]

    Flyvbjerg, Bent , month = jun, year =. Case

  53. [53]

    Qualitative and quantitative methods:

    Kidder, Louise H and Fine, Michelle , year =. Qualitative and quantitative methods:. New directions for program evaluation , publisher =

  54. [54]

    Miller, Suzanne and Firesmith, Don , month = jan, year =. Four

  55. [55]

    Maibaum, T. S. E. , editor =. The. Testing of. 2005 , keywords =. doi:10.1007/11430230_1 , abstract =

  56. [56]

    Exploratory

    Neri, Giulia and Marchand, Rob and Walkinshaw, Neil , editor =. Exploratory. Agile. 2025 , keywords =. doi:10.1007/978-3-031-94544-1_11 , abstract =

  57. [57]

    Software Quality Journal , author =

    Exploring human factors of the agile software tester , volume =. Software Quality Journal , author =. 2022 , keywords =. doi:10.1007/s11219-021-09561-2 , abstract =

  58. [58]

    and Münch, Jürgen , month = sep, year =

    Pfahl, Dietmar and Yin, Huishi and Mäntylä, Mika V. and Münch, Jürgen , month = sep, year =. How is exploratory testing used?. Proceedings of the 8th. doi:10.1145/2652524.2652531 , abstract =

  59. [59]

    IEEE Transactions on Software Engineering , author =

    The. IEEE Transactions on Software Engineering , author =. 2013 , keywords =. doi:10.1109/TSE.2012.55 , abstract =

  60. [60]

    and Zuzul, Tiona , year =

    Edmondson, Amy C. and Zuzul, Tiona , year =. Quantitative and. The. doi:10.1057/978-1-137-00772-8_562 , abstract =

  61. [61]

    Investigacion y Educacion en Enfermeria , author =

    Principles,. Investigacion y Educacion en Enfermeria , author =. 2022 , pages =. doi:10.17533/udea.iee.v40n2e03 , abstract =

  62. [62]

    Provisional,

    Singh, Shivansh and Blomqvist, Kirsimarja , month = jan, year =. Provisional,

  63. [63]

    Software: Practice and Experience , author =

    Generative. Software: Practice and Experience , author =. 2025 , note =. doi:https://doi.org/10.1002/spe.70005 , abstract =

  64. [64]

    Perspectives on

    Kempe, Evelyn and Massey, Aaron , month = sep, year =. Perspectives on. 2021. doi:10.1109/RE51729.2021.00012 , abstract =

  65. [65]

    The ethical principles of research with human participants and ethical review in the human sciences in

    TENK, Finnish National Board on Research Integrity , year =. The ethical principles of research with human participants and ethical review in the human sciences in

  66. [66]

    Science of Computer Programming , author =

    Theorizing about software development practices , volume =. Science of Computer Programming , author =. 2015 , keywords =. doi:10.1016/j.scico.2014.11.012 , abstract =

  67. [67]

    Journal of Information Technology Theory and Application (JITTA) , author =

    A model for analyzing changes in systems development practices , volume =. Journal of Information Technology Theory and Application (JITTA) , author =

  68. [68]

    Flyvbjerg, Bent , editor =. Case. The. 2011 , pages =

  69. [69]

    Paré, Guy and Elam, J. J. , editor =. Using. Information. 1997 , pages =

  70. [70]

    Empirical Software Engineer- ing 14, 131–164

    Guidelines for conducting and reporting case study research in software engineering , volume =. Empirical Software Engineering , author =. 2009 , keywords =. doi:10.1007/s10664-008-9102-8 , abstract =

  71. [71]

    Factors affecting software testing time schedule , doi =

    Taipale, O and Smolander, K and Kalviainen, H , year =. Factors affecting software testing time schedule , doi =. Australian

  72. [72]

    , year =

    Webster, Jane and Watson, Richard T. , year =. Analyzing the. MIS Quarterly , publisher =

  73. [73]

    doi:10.1002/9781118715598.ch21 , abstract =

    Booth, Andrew and Hannes, Karin and Harden, Angela and Noyes, Jane and Harris, Janet and Jaure, Allison , month = aug, year =. doi:10.1002/9781118715598.ch21 , abstract =

  74. [74]

    doi:10.48550/arXiv.2409.03838 , abstract =

    Pereira, André and Lima, Bruno and Faria, João Pascoal , month = sep, year =. doi:10.48550/arXiv.2409.03838 , abstract =

  75. [75]

    Combining tsl and llm to automate rest api testing: A comparative study.arXiv preprint arXiv:2509.05540, 2025

    Barradas, Thiago and Paes, Aline and Neves, Vânia de Oliveira , month = sep, year =. Combining. doi:10.48550/arXiv.2509.05540 , abstract =

  76. [76]

    doi:10.48550/arXiv.2511.18038 , abstract =

    Han, Xiaoke and Zhu, Hong , month = nov, year =. doi:10.48550/arXiv.2511.18038 , abstract =

  77. [77]

    IEEE Access , author =

    Verifying. IEEE Access , author =. 2023 , keywords =. doi:10.1109/ACCESS.2023.3320576 , abstract =

  78. [78]

    Sanei, J

    Melegati, Jorge and Wang, Xiaofeng , month = may, year =. Surfacing. 2021. doi:10.1109/CHASE52884.2021.00013 , abstract =

  79. [79]

    Applications and

    De Morais Leça, Matheus and Valença, Lucas and Santos, Reydne and De Souza Santos, Ronnie , month = may, year =. Applications and. 2025. doi:10.1109/WSESE66602.2025.00013 , abstract =

  80. [80]

    Numbers and interpretations:

    Marecek, Jeanne , month = apr, year =. Numbers and interpretations:. Theory & Psychology , publisher =. doi:10.1177/0959354310391353 , abstract =

Showing first 80 references.