Recognition: unknown
To Vibe Research or Not to Vibe Research? Generative AI in Qualitative Research
Pith reviewed 2026-05-09 20:28 UTC · model grok-4.3
The pith
Generative AI suits qualitative research mainly when the approach is positivist small-q rather than non-positivist Big Q.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The qualitative research approach, small-q (positivist or post-positivist) or Big Q (non-positivist), is among the major criteria for determining whether generative AI can be used in qualitative research. In addition to research philosophy and research approach, skills, ethics, and personal preferences also play a role in researchers' decisions about whether to use AI in qualitative research.
What carries the argument
The small-q versus Big Q classification of qualitative research approaches, which ties AI compatibility to the underlying research philosophy.
If this is right
- Software engineering researchers using small-q approaches can more readily incorporate generative AI for tasks like data coding or summarization.
- Big Q researchers should exercise greater caution or avoid AI to maintain interpretive depth and researcher involvement.
- Ethical considerations around data privacy and AI bias must be addressed regardless of approach.
- Developing AI literacy becomes essential for researchers deciding on tool use.
Where Pith is reading between the lines
- This framework might encourage software engineering teams to declare their qualitative stance upfront when publishing AI-assisted studies.
- Neighbouring fields like information systems research could adopt similar distinctions to guide AI tool policies.
- A testable extension would be to compare AI-assisted and fully manual qualitative analyses in SE papers for differences in findings depth.
Load-bearing premise
Existing literature provides a neutral and complete summary of views on generative AI in qualitative research, and that philosophy, skills, ethics, and preferences are the primary factors researchers weigh.
What would settle it
A survey of software engineering researchers finding no consistent difference in generative AI adoption between small-q and Big Q users would challenge the central criterion.
Figures
read the original abstract
There has been intense debate among qualitative researchers about whether generative AI is suitable for qualitative research. In this paper, we summarize the broader ongoing discussion of generative AI in qualitative research and its implications for software engineering researchers. The qualitative research approach, small-q (positivist or post-positivist) or Big Q (non-positivist), is among the major criteria for determining whether generative AI can be used in qualitative research. In addition to research philosophy and research approach, skills, ethics, and personal preferences also play a role in researchers' decisions about whether to use AI in qualitative research.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper summarizes the ongoing debate among qualitative researchers on the suitability of generative AI for qualitative research and discusses its implications specifically for software engineering researchers. It identifies the distinction between small-q (positivist or post-positivist) and Big Q (non-positivist) qualitative approaches as a primary criterion for deciding whether generative AI can be appropriately used. The paper further states that researchers' skills, ethical considerations, and personal preferences also factor into such decisions.
Significance. If the literature synthesis is accurate and balanced, the paper could provide a helpful entry point for software engineering researchers navigating the integration of generative AI into qualitative work by distilling key philosophical and practical considerations from the broader methods literature. Its value lies in contextualizing an active debate for a specific disciplinary audience rather than in presenting new empirical findings or formal derivations.
minor comments (2)
- The abstract and title use informal phrasing (e.g., 'To Vibe Research or Not to Vibe Research?') that may benefit from a more conventional academic tone or explicit definition of 'vibe research' to ensure accessibility for readers unfamiliar with the term.
- Consider adding a brief table or structured list in the main text that maps the small-q/Big Q distinction to specific generative AI use cases (e.g., coding, theme generation) to make the central criterion more concrete and actionable for software engineering practitioners.
Simulated Author's Rebuttal
We thank the referee for their review and for recommending minor revision. The referee's summary accurately captures the manuscript's contribution as a synthesis of the ongoing debate on generative AI in qualitative research, with specific implications for software engineering researchers. We appreciate the recognition of its potential value as an entry point for the disciplinary audience.
Circularity Check
No significant circularity; literature summary is self-contained
full rationale
The paper presents a summary of ongoing external debate on generative AI in qualitative research and its implications for software engineering. It advances no equations, derivations, fitted parameters, hypotheses, or novel empirical results. The central claim—that small-q vs. Big Q approaches plus skills/ethics/preferences influence AI use—is explicitly framed as a distillation of existing literature rather than a constructed output. No self-citation chains, ansatzes, or renamings appear in the provided text, and the work does not reduce any result to its own inputs by definition. This is the expected finding for a non-derivational discussion paper.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption The distinction between small-q (positivist or post-positivist) and Big Q (non-positivist) qualitative research approaches is a valid and major criterion for evaluating AI use.
Reference graph
Works this paper leans on
-
[1]
Lee, A. S. and Liebenau, J. , doi =. Information. doi:10.10 07/978-0-387-35309-8_1 , abstract =
-
[2]
Friese, Susanne and Nguyen-Trung, Kien and Powell, Steve and Morgan, David L. , month = mar, year =. Beyond. Qualitative Inquiry , publisher =. doi:10.1177/10778004261429393 , abstract =
-
[3]
Organizational Research Methods , author =
Generative. Organizational Research Methods , author =. 2026 , note =. doi:10.1177/10944281251377154 , abstract =
-
[4]
Glaser, BG and Strauss, AL , year =. The
-
[5]
Nguyen, Duc Cuong and Welch, Catherine , year =. Engaged and. doi:10.1177/00076503251386539 , journal =
-
[6]
and Kasperiuniene, Judita , month = apr, year =
Costa, António Pedro and Bryda, Grzegorz and Christou, Prokopis A. and Kasperiuniene, Judita , month = apr, year =. International Journal of Qualitative Methods , publisher =. doi:10.1177/16094069251383739 , abstract =
-
[7]
AI Is Turning Research into a Scientific Monoculture
Traberg, Cecilie Steenbuch and Roozenbeek, Jon and van der Linden, Sander , month = feb, year =. Communications Psychology , publisher =. doi:10.1038/s44271-026-00428-5 , abstract =
-
[8]
The Qualitative Report , author =
How to. The Qualitative Report , author =. 2016 , pages =. doi:10.46743/2160-3715/2016.2361 , number =
-
[9]
Vikan, Magnhild and Aryan, Ramtin and Kannelønning, Mari Serine and Riegler, Michael Alexander and Danielsen, Stein Ove , month = sep, year =. Reflecting on. Qualitative Health Research , publisher =. doi:10.1177/10497323251365211 , abstract =
-
[10]
and Lincoln, Yvonna S
Guba, Egon G. and Lincoln, Yvonna S. , year =. Competing paradigms in qualitative research , isbn =. Handbook of qualitative research , publisher =
-
[11]
Lee, Allen S. and Baskerville, Richard L. , month = sep, year =. Generalizing. Information Systems Research , publisher =. doi:10.1287/isre.14.3.221.16560 , abstract =
-
[12]
Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society , year =
Methodological. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society , author =. 2025 , pages =. doi:10.1609/aies.v8i2.36667 , abstract =
-
[13]
Advances in Methods and Practices in Psychological Science , author =
A. Advances in Methods and Practices in Psychological Science , author =
-
[14]
Wheeler, Kathryn , month = jan, year =. Technological reflexivity in practice: how. Qualitative Research in Psychology , publisher =. doi:10.1080/14780887.2025.2602820 , abstract =
-
[15]
Response to: "
Friese, Susanne , keywords =. Response to: ". 2025 , publisher =
2025
-
[16]
Braun, Virginia and Clarke, Victoria , month = apr, year =. Reporting guidelines for qualitative research: a values-based approach , volume =. Qualitative Research in Psychology , publisher =. doi:10.1080/14780887.2024.2382244 , abstract =
-
[17]
Supporting best practice in reflexive thematic analysis reporting in
Braun, Virginia and Clarke, Victoria , month = jun, year =. Supporting best practice in reflexive thematic analysis reporting in. Palliative Medicine , publisher =. doi:10.1177/02692163241234800 , abstract =
-
[18]
American Journal of Pharmaceutical Education , author =
A. American Journal of Pharmaceutical Education , author =. 2020 , keywords =. doi:10.5688/ajpe7120 , abstract =
-
[19]
The American Journal of Occupational Therapy , author =
Rigor in. The American Journal of Occupational Therapy , author =. 1991 , pages =. doi:10.5014/ajot.45.3.214 , abstract =
-
[20]
Advances in Nursing Science , author =
The problem of rigor in qualitative research , volume =. Advances in Nursing Science , author =. 1986 , pages =
1986
-
[21]
and Gaskell, George and Allum, Nicholas C
W.Bauer, Martin and Gaskell, George and Bauer, Martin W. and Gaskell, George and Allum, Nicholas C. , year =. Quality,. Qualitative. doi:10.4135/9781849209731 , pages =
-
[22]
Intercoder Reliability in Qualitative Research: Debates and Practical Guidelines
O’Connor, Cliodhna and Joffe, Helene , month = jan, year =. Intercoder. International Journal of Qualitative Methods , publisher =. doi:10.1177/1609406919899220 , abstract =
-
[23]
Nemukula, Lufuno and Jegede, Oluseye , editor =. Factors. Digital. 2025 , keywords =. doi:10.1007/978-3-032-00444-4_28 , abstract =
-
[24]
Nguyen-Trung, Kien , month = nov, year =. Documenting debates on. doi:10.2139/ssrn.5750283 , abstract =
-
[25]
Quality & Quantity , author =. 2025 , keywords =. doi:10.1007/s11135-025-02165-z , abstract =
-
[26]
Yang, Nan and Wang, Xiaofeng and Hyrynsalmi, Sami , month = feb, year =. Positioning. doi:10.5281/zenodo.18454000 , abstract =
-
[27]
Morgan, David L. , month = oct, year =. Exploring the. International Journal of Qualitative Methods , publisher =. doi:10.1177/16094069231211248 , abstract =
-
[28]
Bijker, Rimke and Merkouris, Stephanie S. and Dowling, Nicki A. and Rodda, Simone N. , month = jul, year =. Journal of Medical Internet Research , publisher =. doi:10.2196/59050 , abstract =
-
[29]
Hayes, Adam S. , month = apr, year =. “. International Journal of Qualitative Methods , publisher =. doi:10.1177/16094069251322346 , abstract =
-
[30]
Kim, Chaewon and Ke, Fengfeng and Barrett, Alex and Zhang, Nuodi , editor =. From. Artificial. 2025 , keywords =. doi:10.1007/978-3-031-99264-3_4 , abstract =
-
[31]
Sinha, Ravi and Solola, Idris and Nguyen, Ha and Swanson, Hillary and Lawrence, LuEttaMae , month = jun, year =. The. Proceedings of the 2024. doi:10.1145/3663433.3663456 , abstract =
-
[32]
Friese, Susanne , month = jan, year =. From. Qualitative Inquiry , publisher =. doi:10.1177/10778004251412871 , abstract =
-
[33]
Levitt, Heidi M. , year =. A consideration of the ethics and methodological integrity of generative artificial intelligence in qualitative research:. Qualitative Psychology , publisher =. doi:10.1037/qup0000353 , abstract =
-
[34]
On methodological incongruence in applying
Nguyen-Trung, Kien and Friese, Susanne , month = dec, year =. On methodological incongruence in applying. doi:10.2139/ssrn.5874482 , abstract =
-
[35]
Foley, Sarah and Dempsey, Maria and O’Sullivan, Catherine and Frost, Nollaig , year =. It might be able to produce a text, but it’s not able to produce knowledge: qualitative researchers’ perspectives on the use of. Qualitative Research in Psychology , publisher =. doi:10.1080/14780887.2025.2602816 , number =
-
[36]
Clarke, Victoria and Braun, Virginia and Adams, Jeffery and Callaghan, Jane E. M. and LaMarre, Andrea and Semlyen, Joanna , year =. “. Qualitative Psychology , publisher =. doi:10.1037/qup0000322 , abstract =
-
[37]
Owoahene Acheampong, Kwame and Nyaaba, Matthew , month = jan, year =. Review of. doi:10.2139/ssrn.4686920 , abstract =
-
[38]
Jowsey, Tanisha and Braun, Virginia and Clarke, Victoria and Lupton, Deborah and Fine, Michelle , month = dec, year =. We. Qualitative Inquiry , publisher =. doi:10.1177/10778004251401851 , abstract =
-
[39]
Frankenstein, thematic analysis and generative artificial intelligence:
Jowsey, Tanisha and Stapleton, Peta and Campbell, Shawna and Davidson, Alexandra and McGillivray, Cher and Maugeri, Isabella and Lee, Megan and Keogh, Justin , month = sep, year =. Frankenstein, thematic analysis and generative artificial intelligence:. PLOS ONE , publisher =. doi:10.1371/journal.pone.0330217 , abstract =
-
[40]
Journal for General Philosophy of Science , author =
Towards a. Journal for General Philosophy of Science , author =. 2008 , keywords =. doi:10.1007/s10838-008-9068-7 , abstract =
-
[41]
Exploring the industry's challenges in software testing:
-
[42]
Capretz, Luiz Fernando and Waychal, Pradeep and Jia, Jingdong and Varona, Daniel and Lizama, Yadira , month = may, year =. Studies on the. 2019. doi:10.1109/ICSE-Companion.2019.00105 , abstract =
-
[43]
IEEE Transactions on Engineering Management , author =
An. IEEE Transactions on Engineering Management , author =. 2011 , keywords =. doi:10.1109/TEM.2010.2090730 , abstract =
-
[44]
Information and Software Technology , author =
Challenges and strategies for motivating software testing personnel , volume =. Information and Software Technology , author =. 2016 , keywords =. doi:10.1016/j.infsof.2016.01.002 , abstract =
-
[45]
Ramler, Rudolf and Biffl, Stefan and Grünbacher, Paul , editor =. Value-. Value-. 2006 , keywords =. doi:10.1007/3-540-29263-2_11 , abstract =
-
[46]
Andriadi, Kus and Soeparno, Haryono and Gaol, Ford Lumban and Arifin, Yulyani , month = aug, year =. The. 2023. doi:10.1109/ICIMTech59029.2023.10277919 , abstract =
-
[47]
Medium , author =
Shifting. Medium , author =
-
[48]
Research on software testing techniques and software automation testing tools , url =
Sneha, Karuturi and Malle, Gowda M , month = aug, year =. Research on software testing techniques and software automation testing tools , url =. 2017. doi:10.1109/ICECDS.2017.8389562 , abstract =
-
[49]
Socio-technical management of software testing in information systems projects/
Tosetto, Mauro and Bellini, Carlo Gabriel Porto , month = apr, year =. Socio-technical management of software testing in information systems projects/. Journal of Information Systems & Technology Management , publisher =
-
[50]
Swillus, Mark and Brandt, Carolin and Zaidman, Andy , month = may, year =. An instrument to measure factors that constitute the socio-technical context of testing experience , url =. doi:10.48550/arXiv.2505.01171 , abstract =
-
[51]
Swillus, Mark and Hoda, Rashina and Zaidman, Andy , month = oct, year =. Who cares about testing?:. doi:10.48550/arXiv.2504.07208 , abstract =
-
[52]
Flyvbjerg, Bent , month = jun, year =. Case
-
[53]
Qualitative and quantitative methods:
Kidder, Louise H and Fine, Michelle , year =. Qualitative and quantitative methods:. New directions for program evaluation , publisher =
-
[54]
Miller, Suzanne and Firesmith, Don , month = jan, year =. Four
-
[55]
Maibaum, T. S. E. , editor =. The. Testing of. 2005 , keywords =. doi:10.1007/11430230_1 , abstract =
-
[56]
Neri, Giulia and Marchand, Rob and Walkinshaw, Neil , editor =. Exploratory. Agile. 2025 , keywords =. doi:10.1007/978-3-031-94544-1_11 , abstract =
-
[57]
Software Quality Journal , author =
Exploring human factors of the agile software tester , volume =. Software Quality Journal , author =. 2022 , keywords =. doi:10.1007/s11219-021-09561-2 , abstract =
-
[58]
and Münch, Jürgen , month = sep, year =
Pfahl, Dietmar and Yin, Huishi and Mäntylä, Mika V. and Münch, Jürgen , month = sep, year =. How is exploratory testing used?. Proceedings of the 8th. doi:10.1145/2652524.2652531 , abstract =
-
[59]
IEEE Transactions on Software Engineering , author =
The. IEEE Transactions on Software Engineering , author =. 2013 , keywords =. doi:10.1109/TSE.2012.55 , abstract =
-
[60]
Edmondson, Amy C. and Zuzul, Tiona , year =. Quantitative and. The. doi:10.1057/978-1-137-00772-8_562 , abstract =
-
[61]
Investigacion y Educacion en Enfermeria , author =
Principles,. Investigacion y Educacion en Enfermeria , author =. 2022 , pages =. doi:10.17533/udea.iee.v40n2e03 , abstract =
-
[62]
Provisional,
Singh, Shivansh and Blomqvist, Kirsimarja , month = jan, year =. Provisional,
-
[63]
Software: Practice and Experience , author =
Generative. Software: Practice and Experience , author =. 2025 , note =. doi:https://doi.org/10.1002/spe.70005 , abstract =
-
[64]
Kempe, Evelyn and Massey, Aaron , month = sep, year =. Perspectives on. 2021. doi:10.1109/RE51729.2021.00012 , abstract =
-
[65]
The ethical principles of research with human participants and ethical review in the human sciences in
TENK, Finnish National Board on Research Integrity , year =. The ethical principles of research with human participants and ethical review in the human sciences in
-
[66]
Science of Computer Programming , author =
Theorizing about software development practices , volume =. Science of Computer Programming , author =. 2015 , keywords =. doi:10.1016/j.scico.2014.11.012 , abstract =
-
[67]
Journal of Information Technology Theory and Application (JITTA) , author =
A model for analyzing changes in systems development practices , volume =. Journal of Information Technology Theory and Application (JITTA) , author =
-
[68]
Flyvbjerg, Bent , editor =. Case. The. 2011 , pages =
2011
-
[69]
Paré, Guy and Elam, J. J. , editor =. Using. Information. 1997 , pages =
1997
-
[70]
Empirical Software Engineer- ing 14, 131–164
Guidelines for conducting and reporting case study research in software engineering , volume =. Empirical Software Engineering , author =. 2009 , keywords =. doi:10.1007/s10664-008-9102-8 , abstract =
-
[71]
Factors affecting software testing time schedule , doi =
Taipale, O and Smolander, K and Kalviainen, H , year =. Factors affecting software testing time schedule , doi =. Australian
-
[72]
, year =
Webster, Jane and Watson, Richard T. , year =. Analyzing the. MIS Quarterly , publisher =
-
[73]
doi:10.1002/9781118715598.ch21 , abstract =
Booth, Andrew and Hannes, Karin and Harden, Angela and Noyes, Jane and Harris, Janet and Jaure, Allison , month = aug, year =. doi:10.1002/9781118715598.ch21 , abstract =
-
[74]
doi:10.48550/arXiv.2409.03838 , abstract =
Pereira, André and Lima, Bruno and Faria, João Pascoal , month = sep, year =. doi:10.48550/arXiv.2409.03838 , abstract =
-
[75]
Barradas, Thiago and Paes, Aline and Neves, Vânia de Oliveira , month = sep, year =. Combining. doi:10.48550/arXiv.2509.05540 , abstract =
-
[76]
doi:10.48550/arXiv.2511.18038 , abstract =
Han, Xiaoke and Zhu, Hong , month = nov, year =. doi:10.48550/arXiv.2511.18038 , abstract =
-
[77]
Verifying. IEEE Access , author =. 2023 , keywords =. doi:10.1109/ACCESS.2023.3320576 , abstract =
-
[78]
Melegati, Jorge and Wang, Xiaofeng , month = may, year =. Surfacing. 2021. doi:10.1109/CHASE52884.2021.00013 , abstract =
-
[79]
De Morais Leça, Matheus and Valença, Lucas and Santos, Reydne and De Souza Santos, Ronnie , month = may, year =. Applications and. 2025. doi:10.1109/WSESE66602.2025.00013 , abstract =
-
[80]
Marecek, Jeanne , month = apr, year =. Numbers and interpretations:. Theory & Psychology , publisher =. doi:10.1177/0959354310391353 , abstract =
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.