Pith. sign in

REVIEW 5 major objections 7 minor 63 references

Requirements Elicitation Follow-Up Question Generation

T0 review · 5 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that GPT-4o, guided by a catalog of 14 common interviewer mistakes, generates follow-up interview questions that human raters judge to be more relevant, clearer, and more informative than human-authored questions.

desk verdict Solid, well-scoped empirical paper with a useful new dataset; the headline LLM 'outperform' claim is real but narrower than the abstract implies because the human baseline is a single analyst. read the letter →

arxiv 2507.02858 v1 pith:2H3WY36J submitted 2025-07-03 cs.SE cs.CL

Yuchen Shen , Anmol Singhal , Travis Breaux This is my paper
classification cs.SEcs.CL
keywords requirementselicitationinterviewfollow-upquestionslargelanguagemodelsGPT-4ointerviewermistakesquestionqualitypairedcomparisontextualentailment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether a large language model can compose good follow-up questions during a requirements-elicitation interview, in real time, from the interviewee's speech. Its central claim is that GPT-4o, prompted with a catalog of 14 common interviewer mistakes to avoid, generates follow-up questions that blind raters judge as more relevant, clearer, and more informative than questions written by a human analyst, and that it wins 68% of paired comparisons. A secondary claim is that with minimal guidance, GPT-4o questions are neither better nor worse than human-authored ones. The work matters because interviews are a costly, hard-to-scale elicitation method in which interviewers must listen and formulate questions under cognitive load; an effective real-time assistant could make elicitation cheaper and more consistent.

What carries the argument

The load-bearing mechanism is a catalog of 14 interviewer mistake types, down-selected from 28 criteria synthesized from prior literature on requirements elicitation interviews. The catalog is split into follow-up-mistake types, such as failing to elicit tacit assumptions, failing to consider alternatives, not asking for clarification when statements are unclear or contradictory, and failing to elicit tacit knowledge, plus question-framing mistake types, such as asking generic, jargon-laden, technical, vague, or solution-seeking questions. The paper uses these criteria in two ways: as a classification prompt to decide whether a human-authored question commits the mistake, and as a generation prompt telling GPT-4o to write a question that avoids it. The evaluation machinery is a mixed-effect Bradley-Terry model on paired comparisons plus a mixed-effect ordinal logistic regression on 1-to-5 ratings of relevancy, clarity, and informativeness, with raters as random effects.

What would settle it

Re-run the main paired-comparison study with comparison questions authored by several trained, experienced requirements interviewers who are blind to the LLM-generated answers, keeping the same 30 interview contexts and 14 mistake types. The central claim is falsified if GPT-4o no longer wins a majority of paired comparisons or if the mean ratings no longer differ from the human questions at $p<0.05$.

Watch

Extended reading notes

Core claim

The paper's central discovery is a demonstrated use of LLMs at the moment of elicitation rather than before or after the interview. Using transcripts of 14 interviews in four directory-service domains, the authors show that a follow-up question can usually be generated from the immediately preceding speech: 70% of 146 questions needed zero or one prior turn, and 98% needed at most four turns. When GPT-4o was given only those prior turns and asked for a follow-up, its questions were statistically indistinguishable from the interviewers' own questions on relevancy, clarity, and informativeness. When generation was guided by a synthesized framework of 14 mistake types, such as failing to consider alternatives, not clarifying contradictions, or asking vague, technical, or solution-seeking questions, GPT-4o's questions were preferred in 87 of 128 paired comparisons (68%) and received higher mean ratings on all three quality dimensions, with $p<0.05$. The authors interpret this as evidence that mistake-type guidance, not general language ability, is what pushes LLM-generated questions past a human baseline.

Load-bearing premise

The headline result assumes that one human analyst's questions, written after reading a mistake catalog, are an adequate stand-in for what skilled practicing interviewers would write; if expert interviewers authored the comparison questions, the GPT-4o advantage could shrink or vanish.

Editorial extensions

If this is right

  • A real-time interview assistant could present mistake-avoiding follow-up questions to an interviewer during the interview, using only the last few speaker turns as context.
  • Because 98% of observed follow-up questions depend on at most four prior turns, a lightweight context window is enough for practical support tools.
  • The seven-question typology gives a way to classify and route follow-up question generation by required context length.
  • Prompting an LLM to avoid specific, named mistake types may transfer to other quality-sensitive dialogue tasks where generic prompts underperform.
  • Mistake-guided generation raises quality scores across all three evaluated dimensions simultaneously, not just on one metric.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the effect is causal and not just an artifact of the single human baseline, the same mistake-catalog prompting could be tested on other LLMs, and the source list of 28 criteria could become a reusable checklist for question-generation systems.
  • The authors' side study suggests a concrete bottleneck: when asked to avoid all 14 mistakes at once, GPT-4o produced only 1 of 30 questions that avoided every mistake, so multi-constraint instruction is the next target for improvement.
  • A stronger test would swap the single human analyst for several trained interviewers; if their questions closed the gap, the practical takeaway would narrow from 'LLMs write better questions' to 'LLMs help novices write expert-like questions.'
  • The framework could also be applied in reverse, as a training or self-assessment checklist for human interviewers, since the same 14 criteria are usable for classification.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 7 minor

Summary. The paper investigates whether GPT-4o can generate follow-up questions for requirements elicitation interviews in real time. It contributes a framework of 14 common interviewer mistake types, a taxonomy of follow-up question types derived from 14 interview transcripts, and three studies: (1) a minimally guided generation comparison between GPT-4o and trained interviewers using Grice's maxims of relevancy, clarity, and informativeness; (2) a mistake-classification agreement study between GPT-4o and a single human analyst; and (3) a mistake-guided generation comparison in which GPT-4o-generated questions are preferred over human-analyst-authored questions in paired comparisons. The abstract claims that minimally guided LLM questions are no worse than human questions and that mistake-guided LLM questions outperform human-authored questions.

Significance. If the results hold, this work offers a practical method for LLM-assisted requirements elicitation and a useful synthesis of interviewer mistake types. The paper's strengths include a replication package on Zenodo, blinded survey designs, a power analysis, and the use of mixed-effects models for paired comparisons. The falsifiable hypotheses and explicit evaluation criteria based on Grice's maxims are commendable. However, the comparative claims currently rest on a single-analyst baseline and on interpreting non-significance as equivalence, so the significance is conditional on strengthening these points.

major comments (5)
  1. [§III-B, Table Ia] Study 1 tests H1 ('not different') with two-tailed Student's t-tests and interprets the non-significant p-values (0.17, 0.08, 0.10) as evidence that GPT-4o-generated questions are 'no worse' than human-authored questions. Failure to reject the null does not establish equivalence; the paper needs equivalence testing (e.g., TOST) with pre-specified bounds or confidence intervals for the mean differences. As written, the 'no worse' conclusion in the abstract and Section V-C is not supported by the analysis.
  2. [§III-D, Table Ib] The RQ3 agreement rate of 81% is computed against labels from a single human analyst, with no second annotator and no inter-rater reliability statistic (e.g., Cohen's kappa). The human reference is thus unvalidated, and the 128-pair dataset for Study 3 is built from agreement between GPT-4o and this same analyst. The paper should either add a second annotator and report agreement, or justify why a single analyst's labels are a sufficient reference for the subsequent comparison.
  3. [§III-E, Table Ic] The main 'outperform' claim rests on a comparison against a single human analyst, not the trained interviewers from Study 1. The paper's own threats section (Section VI) concedes that human performance in RQ2 may not reflect expert practitioners. Because the baseline is a single analyst familiar with the mistake catalog, the abstract's unqualified claim that LLM-generated questions outperform human-authored questions is overgeneralized. The authors should recruit multiple human writers (ideally expert interviewers) or restrict the claim to 'outperform the study analyst' and discuss the baseline's limitations.
  4. [§IV-C] The paper reports that the probability GPT-4o generates a better question is 'about 2.662/(1 + 2.662) = 93.5%' based on the Bradley-Terry odds ratio of 2.662. This arithmetic is incorrect: 2.662/3.662 = 72.7%. The 93.5% figure overstates the effect, and no confidence interval is reported for the 68% win rate or the odds ratio. Please correct the calculation and report an interval estimate for the effect size.
  5. [§III-E] The 128 pairs used in Study 3 consist only of instances where both GPT-4o and the human analyst agreed that the original interviewer question exhibited a mistake. This filtering excludes cases of disagreement, which may bias the comparison in favor of GPT-4o. The authors should either analyze the full set of 420 classification instances or report how the conclusions change under alternative inclusion criteria.
minor comments (7)
  1. [§III-B] The typology in Figure 2b is based on open coding by a single author with no reported validation; the threats section acknowledges this, but a second coder would strengthen the taxonomy.
  2. [§III-E] Typo: 'we recruited 32 participant' should be '32 participants'.
  3. [§IV-A] The phrase 'On an average' should be 'On average'.
  4. [Abstract] The phrase 'when guided by common mistakes types' contains a grammatical error; it should be 'common mistake types'.
  5. [§II-B] Reference [2] contains typos: 'Missing Software Reporistories' should be 'Mining Software Repositories'.
  6. [§V-A] The authors describe reformulating mistake descriptions from negative to positive framing in the prompts, but the mistake criteria in Section III-C are all stated negatively; including the positive prompt variants (e.g., in an appendix) would aid reproducibility.
  7. [§III-E] The power analysis says 128 records exceed the required number, but the earlier description of 30 pairs makes the path to 128 records unclear; please clarify how multiple mistake types per pair produced 128 records.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the main comparisons are empirical and use external criteria; the single-analyst baseline is a validity threat, not a circular reduction.

full rationale

The paper's derivation chain is empirical rather than definitional. Study 1 evaluates GPT-4o questions against human questions from the same interview transcripts using Grice's maxims (relevancy, clarity, informativeness), which are external to the model and not fitted to the outcome. Studies 2 and 3 use a mistake catalog synthesized from 14 prior publications, with each criterion cited to external sources; the catalog is an input prompt, not a fitted parameter. In Study 3, both GPT-4o and the human analyst are given the same mistake-avoidance instruction, so the paired comparison is symmetric with respect to the treatment. The authors' self-citations (refs [48] and [49]) appear only in related-work and definitional contexts and do not carry the load-bearing claims. The paper candidly notes in Section VI that the human baseline is a single analyst 'who was not engaged in the interview and who had access to the mistake catalog during question formulation' and that 'human performance observed in response to RQ2 may not reflect the capabilities of expert practitioners, thus overstating the relative performance of LLM-generated questions.' These are genuine threats to construct and external validity and could affect the effect size, but they do not make the claimed result equivalent to its inputs by construction. No fitted parameter, self-citation chain, or definitional identity between the independent and dependent measures was identified, so no circular step can be exhibited with the required quote-and-reduction evidence.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper does not introduce new physical or conceptual entities. Its claims rest on assumptions about the validity of Gricean quality criteria, the completeness of the literature-derived mistake catalog, and the reliability of crowd ratings.

assumptions (3)
  • domain assumption Grice's maxims of relevancy, clarity, and informativeness are valid measures of follow-up question quality
    Used to define the evaluation criteria in Section III-B; no validation that these correlate with actual elicitation success.
  • domain assumption The 14 mistake criteria, synthesized from 28 criteria extracted from 14 papers, cover the relevant interviewer mistakes
    Authors down-selected to two categories (follow-up and framing); the set may not be complete or representative.
  • domain assumption Prolific crowd workers can reliably rate question quality on the provided scales
    Raters were not trained in discourse analysis; see Section VI external validity threats.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Requirements Elicitation Follow-Up Question Generation." pith.science (2026). https://pith.science/paper/2H3WY36J

@misc{pith2026250702858,
  author       = {Pith},
  title        = {Pith review of: Requirements Elicitation Follow-Up Question Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2H3WY36J}},
  note         = {Machine review of arXiv:2507.02858}
}
read the original abstract

Interviews are a widely used technique in eliciting requirements to gather stakeholder needs, preferences, and expectations for a software system. Effective interviewing requires skilled interviewers to formulate appropriate interview questions in real time while facing multiple challenges, including lack of familiarity with the domain, excessive cognitive load, and information overload that hinders how humans process stakeholders' speech. Recently, large language models (LLMs) have exhibited state-of-the-art performance in multiple natural language processing tasks, including text summarization and entailment. To support interviewers, we investigate the application of GPT-4o to generate follow-up interview questions during requirements elicitation by building on a framework of common interviewer mistake types. In addition, we describe methods to generate questions based on interviewee speech. We report a controlled experiment to evaluate LLM-generated and human-authored questions with minimal guidance, and a second controlled experiment to evaluate the LLM-generated questions when generation is guided by interviewer mistake types. Our findings demonstrate that, for both experiments, the LLM-generated questions are no worse than the human-authored questions with respect to clarity, relevancy, and informativeness. In addition, LLM-generated questions outperform human-authored questions when guided by common mistakes types. This highlights the potential of using LLMs to help interviewers improve the quality and ease of requirements elicitation interviews in real time.

Figures

Figures reproduced from arXiv: 2507.02858 by the authors.

Figure 1
Figure 1. LLM Prompts Used in the Studies The surveys were published using the Qualtrics platform 2 . We recruited 32 survey participants to rate the questions in each survey (64 participants in each group) using the Prolific platform 3 . To enroll in the experiment, participants first signed an informed consent form, which included information about their rights and confidentiality protection. In addition, the participants m… view at source ↗
Figure 2
Figure 2. Speaker Turn and Follow-up Question Type Distri [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

63 extracted references · 56 canonical work pages

  1. [1]

    Elicitron: An LLM Agent-Based Simulation Framework for Design Requirements Elicitation,

    M. Ataei, H. Cheong, D. Grandi, Y . Wang et al., “Elicitron: An LLM Agent-Based Simulation Framework for Design Requirements Elicitation,” arXiv preprint arXiv:2404.16045 , 2024

    arXiv 2024

  2. [2]

    Can LLMs replace man- ual annotation of software engineering artifacts?

    T. Ahmed, P. Devanbu, C. Treude, M. Pradel, “Can LLMs replace man- ual annotation of software engineering artifacts?”. 22nd International Conference on Missing Software Reporistories (MSR25) , 2025

    work page 2025

  3. [3]

    A survey of paraphrasing and textual entailment methods

    I. Androutsopoulos, P. Malakasiotis. “A survey of paraphrasing and textual entailment methods.” Journal of Artificial Intelligence Research, 38, pp.135-187, 2010. 6https://zenodo.org/records/15793870

    work page arXiv 2010

  4. [4]

    Scenario Co-Creation Cards: A Culturally Sensitive Tool for Eliciting Values,

    T. Alshehri, R. Kirkham, and P. Olivier, “Scenario Co-Creation Cards: A Culturally Sensitive Tool for Eliciting Values,” Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–14 , 2020

    work page 2020

  5. [5]

    Using recommendation agents to cope with information overload

    M. Aljukhadar, S. Senecal, C. Daoust. “Using recommendation agents to cope with information overload.” International Journal of Electronic Commerce 17, no. 2: 41-70 , 2012

    work page 2012

  6. [6]

    Knowledge Acquisition Using Structured Interviewing: An Empirical Investigation,

    R. Agarwal, M. R. Tanniru, “Knowledge Acquisition Using Structured Interviewing: An Empirical Investigation,” Journal of Management Information Systems 7 (1): 123–40. doi:10.1080/07421222.1990.11517884, 1990

    arXiv 1990

  7. [7]

    Elicitation Techniques: Getting People to Talk About Ideas They Don’t usually talk about

    K.Barton, “Elicitation Techniques: Getting People to Talk About Ideas They Don’t usually talk about”, Theory and Research in Social Educa- tion, 43(2), 179–205 , 2015

    work page 2015

  8. [8]

    MUCE: a multilingual use case model extractor using GPT-3

    D. Bajaj, A. Goel, S. Gupta, and H. Batra. “MUCE: a multilingual use case model extractor using GPT-3.” International Journal of Information Technology 14, no. 3: 1543-1554 , 2022

    work page 2022

Show all 63 references
  1. [9]

    Language models are few-shot learners

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J.D. Kaplan, P. Dhariwal, A. Neelakantan et al. “Language models are few-shot learners.” Advances in neural information processing systems 33: 1877-1901 , 2020

    1901

  2. [10]

    Rank Analysis of Incomplete Block Designs: I. The Method of Paired Comparisons

    R. A. Bradley and M. E. Terry. “Rank Analysis of Incomplete Block Designs: I. The Method of Paired Comparisons.” Biometrika 39, no. 3/4 (1952): 324–45., 1952

    1952

  3. [11]

    Science-based interviewing: Information elicitation

    S. Brandon, S. Wells, C.Seale. “Science-based interviewing: Information elicitation.” Journal of Investigative Psychology and Offender Profiling 15, no. 2: 133-148 , 2018

    2018

  4. [12]

    Learning from mistakes: An empirical study of elicitation interviews performed by novices,

    M. Bano, D. Zowghi, A. Ferrari, P. Spoletini, and B. Donati, “Learning from mistakes: An empirical study of elicitation interviews performed by novices,” 2018 IEEE 26th International Requirements Engineering Conference (RE). IEEE, 2018, pp. 182–193 , 2018

    2018

  5. [13]

    On the use of GPT- 4 for creating goal models: an exploratory study

    B. Chen, K. Chen, S. Hassani, Y . Yang et. al. “On the use of GPT- 4 for creating goal models: an exploratory study.” In 2023 IEEE 31st International Requirements Engineering Conference Workshops (REW), pp. 262-271. IEEE , 2023

    2023

  6. [14]

    Systematizing requirements elicitation technique selection,

    D. Carrizo, O. Dieste, and N. Juristo, “Systematizing requirements elicitation technique selection,” Inf. Softw. Technol. 56, 6 (June, 2014), 644–669, 2014

    2014

  7. [15]

    Mapping cognitive structures and processes through verbal content: The thought-listing technique,

    J. T. Cacioppo, W. von Hippel, J. M. Ernst, “Mapping cognitive structures and processes through verbal content: The thought-listing technique,” Journal of Consulting and Clinical Psychology, 65(6), 928–940. , 1997

    1997

  8. [16]

    Data quality in on- line human-subjects research: Comparisons between MTurk, Prolific, CloudResearch, Qualtrics, and SONA

    B. Douglas, P.J. Ewell, and M. Brauer. “Data quality in on- line human-subjects research: Comparisons between MTurk, Prolific, CloudResearch, Qualtrics, and SONA.” PLoS ONE 18(3): e0279720 , 2023

    2023

  9. [17]

    Common mistakes of student analysts in requirements elicitation interviews,

    B. Donati, A. Ferrari, P. Spoletini, and S. Gnesi, “Common mistakes of student analysts in requirements elicitation interviews,” International Working Conference on Requirements Engineering: Foundation for Software Quality. Springer, 2017, pp. 148–164 , 2017

    2017

  10. [18]

    Recognizing textual entailment: Models and applications

    I. Dagan, D. Roth, F. Zanzotto and M. Sammons. “Recognizing textual entailment: Models and applications”, Springer Nature, 2022

    2022

  11. [19]

    Smarter faster better: The transformative power of real productivity

    C. Duhigg, “Smarter faster better: The transformative power of real productivity”, Random House Publishing, New York, NY , 2016

    2016

  12. [20]

    Detecting hallucinations in large language models using semantic entropy

    S. Farquhar, J. Kossen, L. Kuhn, et al. “Detecting hallucinations in large language models using semantic entropy”. Nature 630, 625–630 , 2024

    2024

  13. [21]

    Learning Require- ments Elicitation Interviews with Role-Playing, Self-Assessment and Peer-Review,

    A. Ferrari, P. Spoletini, M. Bano and D. Zowghi, “Learning Require- ments Elicitation Interviews with Role-Playing, Self-Assessment and Peer-Review,”2019 IEEE 27th International Requirements Engineering Conference (RE), Jeju, Korea (South) , 2019

    2019

  14. [22]

    How do requirements evolve during elicitation? An empirical study combining interviews and app store analysis

    A. Ferrari, P. Spoletini, and S. Debnath. “How do requirements evolve during elicitation? An empirical study combining interviews and app store analysis.” Requirements Engineering 27, no. 4: 489-519 , 2022

    2022

  15. [23]

    Ambiguity and tacit knowledge in requirements elicitation interviews

    A. Ferrari, P. Spoletini, and S. Gnesi. “Ambiguity and tacit knowledge in requirements elicitation interviews.” Requirements Engineering 21, no. 3: 333-355, 2016

    2016

  16. [24]

    Generating Requirements Elicitation In- terview Scripts with Large Language Models,

    B. G ¨orer and F. B. Aydemir, “Generating Requirements Elicitation In- terview Scripts with Large Language Models,” 2023 IEEE 31st Interna- tional Requirements Engi- neering Conference Workshops (REW), Han- nover, Germany, 2023, pp. 44-51, doi: 10.1109/REW57809.2023.00015., 2023, 2023

    2023

  17. [25]

    GPT-Powered Elicitation Interview Script Generator for Requirements Engineering Training

    B. G ¨orer and F. B. Aydemir, “GPT-Powered Elicitation Interview Script Generator for Requirements Engineering Training”, arXiv:2406.11439, 2024

    2024 arXiv

  18. [26]

    Openagi: When llm meets domain experts

    Y . Ge, W. Hua, K. Mei, J. Tan, S. Xu, Z. Li, Y . Zhang. “Openagi: When llm meets domain experts”. Advances in Neural Information Processing Systems, 2023

    2023

  19. [27]

    On Non-Functional Requirements,

    M. Glinz, “On Non-Functional Requirements,” 15th IEEE International Requirements Engineering Conference (RE 2007), Delhi, India, pp. 21- 26, doi: 10.1109/RE.2007.45 , 2007

    2007 doi

  20. [28]

    Personas, Participatory Design and Product Devel- opment: An Infrastructure for Engagement

    J. Grudin, J. Pruitt, “Personas, Participatory Design and Product Devel- opment: An Infrastructure for Engagement”, Proceedings of Participa- tion and Design Conference (PDC2002), page 144-161 , 2002

    2002

  21. [29]

    Groen; N

    E. Groen; N. Seyff; R. Ali, F. Dalpiaz et. al., The Crowd in Requirements Engineering: The Landscape and Challenges, IEEE Software, Volume: 34, Issue: 2, Mar.-Apr. 2017 , 2017

    2017

  22. [30]

    The effects of cognitive load for investigative interviewers

    P. Hanway, “The effects of cognitive load for investigative interviewers.” PhD diss., University of Portsmouth , 2020

    2020

  23. [31]

    Elicitation technique selection: how do experts do it?

    A. Hickey, A. Davis. “Elicitation technique selection: how do experts do it?.” In Proceedings. 11th IEEE International Requirements Engineering Conference, 2003., pp. 169-178. IEEE , 2003

    2003

  24. [32]

    The role of domain knowledge in requirements elicitation via interviews: an exploratory study

    I. Hadar, P. Soffer, and K. Kenzi. “The role of domain knowledge in requirements elicitation via interviews: an exploratory study.” Require- ments Engineering 19: 143-159 , 2014

    2014

  25. [33]

    Designing Effective Interview Chatbots: Automatic Chatbot Profiling and Design Suggestion Generation for Chatbot Debugging,

    X. Han, M. Zhou, M. J. Turner, and T. Yeh, “Designing Effective Interview Chatbots: Automatic Chatbot Profiling and Design Suggestion Generation for Chatbot Debugging,” Proceedings of the 2021 CHI Con- ference on Human Factors in Computing Systems (CHI ’21). Association for Co...

    2021

  26. [34]

    The World and the Machine

    M. Jackson, “The World and the Machine.” In Proc. 17th International Conference on Software Engineering , pp. 283–292, 1995

    1995

  27. [35]

    Elicitation Techniques for Interviewing

    J. Johnson, S. Weller, “Elicitation Techniques for Interviewing”, In Handbook of Interview Research, 491-514, SAGE Publications, Inc., ISBN: 9780761919513, 2011

    2011

  28. [36]

    System- atic methodological review: developing a framework for a qualitative semi-structured interview guide

    H. Kallio, A.M. Pietil ¨a, M. Johnson, and M. Kangasniemi, “System- atic methodological review: developing a framework for a qualitative semi-structured interview guide”. Journal of advanced nursing, 72(12), pp.2954-2965, 2016

    2016

  29. [37]

    Negated and Misprimed Probes for Pre- trained Language Models: Birds Can Talk, But Cannot Fly

    N. Kassner and H. Sch ¨utze. “Negated and Misprimed Probes for Pre- trained Language Models: Birds Can Talk, But Cannot Fly.” In Proceed- ings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7811–7818, Online. Association for Computational Li...

    2020

  30. [38]

    Retrieval-augmented generation for knowledge-intensive nlp tasks

    P. Lewis, E. Perez, A. Piktus et al. “Retrieval-augmented generation for knowledge-intensive nlp tasks”. Advances in neural information processing systems. 2020;33:9459-74 , 2020

    2020

  31. [39]

    Textual Entailment based Question Generation

    T. Matsumoto, K. Hasegawa, Y . Yamakawa, & T. Mitamura. “Textual Entailment based Question Generation.” Workshop on Intelligent Inter- active Systems and Language Generation (2IS&NLG), pp. 15-19, 2018

    2018

  32. [40]

    Acquiring Creative Require- ments from the Crowd

    P.K. Murukannaiah, N.Ajmeri, M.P. Singh. “Acquiring Creative Require- ments from the Crowd.”IEEE 24th Int’l Reqts. Engr. Conf., pp. 176-185, 2016

    2016

  33. [41]

    When Users Become Collaborators: Towards Continuous and Context-Aware User Input

    W. Maalej, H. Happel, A. Rashid, “When Users Become Collaborators: Towards Continuous and Context-Aware User Input”, Proc. 24th ACM SIGPLAN Conf. Object-Oriented Programming Systems Languages and Applications (OOPSLA 09), pp. 981-990 , 2009

    2009

  34. [42]

    Bug report, feature request, or simply praise? On automatically classifying app reviews,

    W. Maalej, H. Nabil, “Bug report, feature request, or simply praise? On automatically classifying app reviews,” 2015 IEEE 23rd Interna- tional Requirements Engineering Conference (RE), pp. 116-125, doi: 10.1109/RE.2015.7320414, 2015

    2015

  35. [43]

    Training language models to follow instructions with human feedback

    L. Ouyang, J. Wu, X. Jiang et al. “Training language models to follow instructions with human feedback”. Advances in neural information processing systems, 2022 Dec 6;35:27730-44 , 2022

    2022

  36. [44]

    The state-of-practice in requirements elicitation: an extended interview study at 12 companies

    C. Palomares, Z. Franch, C. Quer et al. “The state-of-practice in requirements elicitation: an extended interview study at 12 companies”. Requirements Eng 26, 273–299 , 2021

    2021

  37. [45]

    A survey on Recognizing Textual Entailment as an NLP Evaluation

    A. Poliak. “A survey on Recognizing Textual Entailment as an NLP Evaluation.” 1st Workshop on Evaluation and Comparison of NLP Systems, pp. 92–109, 2020

    2020

  38. [46]

    Inquiry-based requirements analysis,

    C. Potts, K. Takahashi and A. I. Anton, “Inquiry-based requirements analysis,” IEEE Software, vol. 11, no. 2, pp. 21-32, March 1994, doi: 10.1109/52.268952, 1994

    1994 doi

  39. [47]

    Saldan ˜a

    J. Saldan ˜a. The Coding Manual for Qualitative Researchers, SAGE Publications, 2012

    2012

  40. [48]

    Generative goal modeling,

    A. Sharfuddin, T. D. Breaux. “Generative goal modeling,” IEEE 33rd International Conference on Requirements Engineering , 2025

    2025

  41. [49]

    Stakeholder Preference Extraction From Sce- narios

    Y . Shen and T. Breaux, “Stakeholder Preference Extraction From Sce- narios” in IEEE Transactions on Software Engineering, vol. 50, no. 01, pp. 69-84, 2024

    2024

  42. [50]

    Quantifying language models’ sensitivity to spurious features in prompt design or: How I learned to start worrying about prompt formatting

    M. Sclar, Y . Choi, Y . Tsvetkov, and A. Suhr. “Quantifying language models’ sensitivity to spurious features in prompt design or: How I learned to start worrying about prompt formatting.” In arXiv preprint arXiv:2310.11324, 2023

    arXiv

  43. [51]

    Interview review: An empirical study on detecting ambiguities in requirements elicitation interviews

    P. Spoletini, A. Ferrari, M. Bano, D. Zowghi, and S. Gnesi. “Interview review: An empirical study on detecting ambiguities in requirements elicitation interviews.” In Requirements Engineering: Foundation for Software Quality: 24th International Working Conference, REFSQ 2018, ...

    2018

  44. [52]

    Singh, K

    S. Singh, K. Jiang, K. Bhasin et. al., RACER: An LLM-powered Methodology for Scalable Analysis of Semi-structured Mental Health Interviews, arXiv:2402.02656, 2024

    2024 arXiv

  45. [53]

    Focus groups: Theory and practice

    D. Stewart, P. Shamdasani. “Focus groups: Theory and practice”, Vol

  46. [54]

    Requirements Engineering: From System Goals to UML Models to Software Specifications

    A. van Lamsweerde, “Requirements Engineering: From System Goals to UML Models to Software Specifications”, Wiley, 2009

    2009

  47. [55]

    Sage publications , 2014

    2014

  48. [56]

    A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT

    J. White, Q. Fu, S. Hays, M. Sandborn, C. Olea, H. Gilbert, A. Elnashar, J. Spencer-Smith, D. C. Schmidt, “A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT”, arXiv preprint arXiv:2302.11382 , 2023

    2023 arXiv

  49. [57]

    Large Language Models Still Can’t Plan (A Benchmark for LLMs on Planning and Reasoning about Change)

    K. Valmeekam, A. Olmo, S. Sreedharan, S. Kambhampati, “Large Language Models Still Can’t Plan (A Benchmark for LLMs on Planning and Reasoning about Change)”, NeurIPS 2022 Foundation Models for Decision Making Workshop, 2022

    2022

  50. [58]

    Case study research: Design and methods

    R. K. Yin, “Case study research: Design and methods”, vol. 5. Sage , 2009

    2009

  51. [59]

    Efficient continual pre-training for building domain specific large language models

    Y . Xie, K. Aggarwal, A. Ahmad. “Efficient continual pre-training for building domain specific large language models”. In Findings of the Association for Computational Linguistics ACL 2024 2024 Aug (pp. 10184-10201), 2024

    2024

  52. [60]

    Judging llm-as-a-judge with mt- bench and chatbot arena

    L. Zheng, W. Chiang, Y . Sheng et al. “Judging llm-as-a-judge with mt- bench and chatbot arena”. Advances in Neural Information Processing Systems. 2023 Dec 15;36:46595-623 , 2023

    2023

  53. [61]

    Requirements Elicitation: A Survey of Tech- niques, Approaches, and Tools,

    D. Zowghi, C. Coulin, “Requirements Elicitation: A Survey of Tech- niques, Approaches, and Tools,”Aurum, A., Wohlin, C. (eds) Engineering and Managing Software Requirements. Springer, Berlin, Heidelberg , 2005

    2005

  54. [63]

    Requirements Elicita- tion Follow-Up Question Generation

    Y . Shen, A. Singhal, T. Breaux (2025). “Requirements Elicita- tion Follow-Up Question Generation”. In IEEE International Re- quirements Engineering Conference. IEEE International Require- ments Engineering Conference 2025 (RE), Valencia, Spain . Zenodo. https://doi.org/10.528...

    2025 doi

  55. [118]

    Springer International Publishing , 2018

    2018

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.