Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

Interaction Configurations and Prompt Guidance in Conversational AI for Question Answering in Human-AI Teams

T0 review · 3 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Human-AI question answering only improves quality when the interaction is guided, and suggested prompts are the lever that does the guiding.

desk verdict Interaction-level findings are credible and useful; the headline quality claim rests on a biased post hoc top-7 comparison and should not be taken at face value without a reanalysis. read the letter →

arxiv 2505.01648 v2 pith:ODKBW32N submitted 2025-05-03 cs.HC

Jaeyoon Song , Zahra Ashktorab , Qian Pan , Casey Dugan , Werner Geyer , Thomas W. Malone This is my paper · ORCID
classification cs.HC
keywords human-AIcollaborationconversationalagentsquestionansweringlargelanguagemodelspromptguidancequeryshortcutsresponsequalityevaluationcustomersupport
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that conversational AI can improve customer-support-style question answering only when humans and AI interact successfully, and that a simple interface feature—suggested messages to send to the AI—can make that success more likely. The authors created two prompt-guidance designs (Nudging and Highlight) and tested them against human-only, conversational-AI, and AI-only baselines in two controlled experiments with 137 crowd workers. Across all responses, AI-only output was rated highest on average, and simply pairing a human with AI produced no overall quality gain. But when the best seven responses in each condition were compared, responses built in the Nudging and Conversation conditions were rated significantly higher than AI-only responses. If the finding holds, message-suggestion guidance is a cheap, practical design lever for real-world question-answering systems.

What carries the argument

The load-bearing mechanism is the Nudging configuration's query shortcuts: pre-written, clickable messages—'Ask Agent what it can do to help', 'Paraphrase the question', and 'Summarize the reference document'—inserted at the top of the chat input box. Clicking a button fills the input box with a full prompt (editable before sending), which changes what people ask the AI and measurably raises the number of successful AI responses. The second design, Highlight, supplies key sentences from the reference document and serves as a contrast; the evaluation apparatus is an ELO-style pairwise comparison by crowd raters, checked against the reference documents, with ratings converted to scores. The query shortcuts carry the argument because they are the only manipulated feature that consistently changed behavior and improved interaction success.

What would settle it

Re-evaluate the collected responses using reference documents created from actual primary sources rather than generated by an AI model; if the Nudging and Conversation top-7 advantage over AI-only disappears, the headline result depends on the synthetic ground truth rather than on the interaction design.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that the value of human-AI collaboration in question answering is conditional on the interaction succeeding, and that query shortcuts are an effective way to make it succeed. In Study 1, 31 participants answered questions about iPhone usage under four conditions: human-only, conversation with a GPT-4 agent, Nudging (with suggested messages), and Highlight (with key sentences emphasized). In Study 2, 106 raters blind-compared the resulting 131 responses, including GPT-4's own answers, using reference documents as the accuracy standard. On average, AI-only responses received the highest ratings and no condition differed significantly from the others; however, comparing the best seven responses per condition, Nudging and Conversation responses were rated significantly higher than AI-only responses, both before and after the response source was revealed. The Nudging condition also produced significantly more successful AI responses than the Highlight or Conversation conditions, and clicking the message suggestions correlated with successful AI responses. The authors conclude that human-AI collaboration can beat AI alone, but only when the human's interaction with the AI is successful.

Load-bearing premise

The central claim rests on the premise that the GPT-4-generated reference documents, after author review, are accurate and complete enough to serve as ground truth for rating response quality; if those documents are wrong, incomplete, or stylistically closer to the AI model's own output than to good human answers, the comparison between AI-only and human-involved responses is distorted.

Editorial extensions

If this is right

  • Deploying message-suggestion buttons in customer-support QA tools should increase the share of successful AI responses and raise the ceiling on final response quality, compared with giving users an AI chatbot and no guidance.
  • Teams should not expect quality gains from simply adding an LLM to a human workflow; the interaction must be guided for the collaboration to pay off.
  • The best human-AI outputs can outperform the best AI-only outputs, so high-quality response selection or skilled-worker filtering is worth designing for.
  • Designers should support meta-prompting (asking the AI what it can do) and summarization requests, because these behaviors predict successful AI responses.
  • The gap between raters' stated preference for human-AI text and their actual higher average rating of AI-only text means self-reported preferences are not reliable predictors of response quality.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The three specific nudges chosen here are likely not the only effective ones: an obvious extension is adaptive or context-aware suggestions that change as the conversation proceeds, which the authors note as future work.
  • If the effect is driven by teaching users a successful interaction pattern, the same benefit might be obtained more cheaply with a tutorial or example rather than persistent buttons; that is a testable alternative explanation the paper does not rule out.
  • A stronger test of the synthetic-ground-truth threat would be to have domain experts rewrite the reference documents from primary sources and repeat Study 2; this would decouple the interaction-design effect from stylistic similarity between model-generated documents and model-generated answers.
  • The top-7 comparison is implicitly a worker-screening argument: in practice, organizations would need to identify high performers or iterate on drafts to realize the human-AI advantage, since average responses do not beat AI alone.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper investigates how different interaction configurations affect the quality of responses produced in human-AI question answering. In Study 1, 31 crowd workers each constructed customer-support responses to Stack Overflow questions under four within-subjects conditions: Human only, Conversation with an AI agent, Nudging with suggested prompts, and Highlight with key reference sentences; 7 additional AI-only responses were generated by GPT-4. In Study 2, 106 raters performed pairwise comparisons of the resulting 131 responses, blind to source and then with source labels. The overall ANOVA found no significant condition differences, but a post hoc analysis selecting the best seven responses per condition reports that Nudging and Conversation responses are rated significantly higher than AI-only responses. The paper concludes that successful human-AI collaboration can improve quality, that Nudging query shortcuts are a practical design lever, and that merely combining human and AI effort does not guarantee improvement.

Significance. If the central quality claim were sound, the message-suggestion design would be a practical, low-cost intervention for customer-support QA, and the paper would make a useful contribution to human-AI interaction research. The study has genuine strengths: a controlled within-subjects design, detailed logging of human-AI interactions, independently coded interaction categories with reported inter-rater reliability, and a separate rater pool with attention checks. The process-level findings, such as the Nudging condition producing more successful AI responses and more productive message types, are informative and less vulnerable to the selection issue. However, the headline quality claim rests on the post hoc best-seven analysis, and the evaluation ground truth is itself generated by the same model family as the AI-only responses being rated, so the central claim is not yet established at the level claimed in the abstract and conclusion.

major comments (3)
  1. [Section 7.2] The best-seven comparison is structurally biased and does not support the quality claim as stated. The human-involved conditions each contribute 31 responses (one per participant), whereas the AI-only condition contains exactly 7 responses (one per question), so comparing the mean of the top seven ELO-rated responses from each 31-response condition with the mean of all seven AI-only responses compares an upper order statistic of 31 draws with the full sample of 7 draws. Under a null model in which all conditions are drawn from the same distribution, the expected mean of the top 7 of 31 exceeds the expected mean of all 7, so the observed Nudging advantage (M=1645.034 vs M=1591.730) can arise without any true condition effect. Moreover, the same ELO ratings are used both to select the top seven and in the subsequent ANOVA and Tukey HSD tests, so the p-values do not test a pre-specified hypothesis on independent data. Section 8.4 acknowledges this limitation but the abstract, Section 7.2, and Section 9 still advance the quality claim. Please add a selection-null test (for example, a permutation test that repeatedly draws the top 7 of 31 responses under the null and compares them with 7 AI-only responses), and if the effect does not survive, revise the abstract and conclusions accordingly.
  2. [Sections 3.2 and 6.2.2] The evaluation standard for response quality is vulnerable to circularity. The reference documents used as the accuracy benchmark were generated by GPT-4 and then edited by the authors, while the AI-only responses being rated were also generated by GPT-4 from the same model family, and raters were explicitly instructed to fact-check against these reference documents. Stylistic or factual alignment between GPT-4-generated references and GPT-4-generated responses could therefore inflate the perceived quality of AI-only outputs relative to human-involved responses, distorting the headline comparisons. The authors acknowledge the synthetic origin of the documents in Section 8.4, but the manuscript does not quantify the risk or provide an external check. Please report a robustness analysis using an independent ground truth (for example, the original Stack Overflow accepted answers) or separate accuracy and style ratings, so the relative quality claim does not rest entirely on a same-family evaluation standard.
  3. [Section 7.4 and 8.2.1] The causal language used for the Nudging effect on interaction success is stronger than the evidence warrants. The claim that query shortcuts "led to successful interactions" is supported by correlations within the Nudging condition, such as r=0.626 between clicks on suggestions and the number of successful AI responses. Because participants chose whether and how often to click, this correlation may reflect participant diligence or engagement rather than a causal effect of the shortcut design itself. An intention-to-treat analysis comparing the Nudging and Conversation conditions on interaction success, or an analysis that controls for total interaction effort, would better support the causal statements in Sections 7.4 and 8.2.1.
minor comments (6)
  1. [Section 6.1.1] The reported average task duration of 73.277 minutes is difficult to reconcile with the 30-minute task description used for recruitment and payment; please clarify whether this includes waiting time or is a typo.
  2. [Section 6.1.1] The participant recruitment text appears twice nearly verbatim in the same subsection; please remove the duplicate.
  3. [Section 7.2] The term "class imbalance" is used for unequal response counts across conditions, but the cited references [19,31] concern imbalanced learning in classification; please use a different term or cite a more directly relevant source.
  4. [Section 3.2] The average reference document length is reported as "3308.429 letters long" with a standard deviation; presumably the intended unit is characters, and the unit should be stated consistently.
  5. [Table 3] The high-ranking Highlight response contains typos (for example, "app cons" instead of "app icon") and inconsistent punctuation; please proofread the example responses.
  6. [Section 7.4] The sentence "more participants asked the AI to paraphrase the reference document" should likely say "paraphrase the question" based on the nudge buttons and Table 5; please clarify.

Circularity Check

2 steps flagged · score 6.0 of 10

The headline claim that the Nudging configuration improves response quality vs. AI alone is based on a top-7 selection made from the same ELO ratings that are then re-tested, making the comparison partly constructed from its own outcome data; GPT-4-generated reference documents add a partial measurement circularity.

  1. fitted input called prediction [Section 7.2 (Quality of Responses by Study Condition), Figure 8; acknowledged in Section 8.4]
    "By selecting the top seven responses in each condition based on their ELO ratings, we could address this imbalance, allowing for a balanced comparison of the highest-performing outputs across conditions. ... Post-hoc comparisons using Tukey’s HSD test indicated that both the Nudging and Conversation conditions were rated significantly higher than GPT-4 responses, both before and after seeing the source labels (p < 0.05)."

    The Nudging and Conversation means in this test are not independent condition estimates: they are order statistics. Study 1 produced 31 responses per human-involved condition (31 participants, one response per condition), while the AI-only condition has exactly 7 responses, one per question. Taking the best 7 of 31 ELO-rated responses and then running ANOVA/Tukey on those same ELO ratings guarantees a high mean for the selected subset even under a null hypothesis of identical conditions, so the observed Nudging advantage over AI alone (M=1645.034 vs M=1591.730) is partly forced by the selection rule.

  2. other [Sections 3.2, 6.1.2, 6.2.2; acknowledged in Section 8.4]
    "We used prompt engineering with GPT-4 to create high-quality reference documents. ... The generated reference documents served two purposes ... the documents served as a standard for raters in Study 2 to evaluate response accuracy. ... The responses for the AI condition were generated using GPT-4, using the prompt in Figure 6."

    The accuracy benchmark (reference documents) and the AI-only system being benchmarked are both outputs of GPT-4, with the AI-only prompt containing the same reference document as context. Raters were instructed to fact-check responses against that GPT-4-authored document, so responses that closely track GPT-4's phrasing or coverage may be rewarded for consistency with the evaluation standard rather than for independent quality. The authors' manual review reduces but does not eliminate this shared-source bias. This is a partial measurement circularity, distinct from the top-7 selection problem, and the paper itself flags it in Section 8.4 ('Another notable limitation is that we synthetically generated the reference documents using GPT-4').

full rationale

The paper's most robust, non-circular results are behavioral: the Nudging configuration increases the number of summarization/paraphrase requests and successful AI responses (Section 7.4), and these findings are not derived from the response-quality ratings. The headline quality claim, however, rests on the top-7 comparison in Section 7.2. Because the top-7 subset is selected using the same ELO ratings that are subsequently entered into the ANOVA and Tukey tests, the Nudging/Conversation advantage over AI-only is at least partly an artifact of comparing the best 7 of 31 draws with all 7 draws of the AI condition. This is a fitted-input-called-prediction pattern rather than a full definitional identity, so it constitutes partial circularity. The GPT-4-generated reference documents add a second, lesser circularity: the accuracy criterion and the AI-only system share a generator, so part of the measured AI advantage may be model self-consistency. The paper acknowledges both limitations in Section 8.4, but the abstract and conclusion still state the quality claim without the selection caveat. No load-bearing self-citation or imported uniqueness theorem was found; the behavioral interaction claims remain independent of the ratings and are not circular.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical or formal entities. The central measurement assumptions are the synthetic reference documents used as ground truth, the validity of crowd rater Elo judgments, the customer-support simulation setting, and the representativeness of GPT-4. The only hand-chosen numerical knob affecting the central claim is the top-7 threshold used for the best-responses comparison.

free parameters (1)
  • top-k threshold (k=7) = 7
    The comparison that produces the headline result uses the seven highest-rated responses per condition, chosen because the AI-only condition had seven responses; this selection is post hoc and discards the bottom responses, and it affects the central claim.
assumptions (4)
  • domain assumption GPT-4-generated reference documents, after author review, contain accurate and complete answers to the questions.
    Invoked in Section 3.2 and used as the accuracy benchmark for raters in Section 6.2; if the synthetic documents are wrong or stylistically biased, response quality rankings are distorted.
  • domain assumption Crowd raters' pairwise comparisons, aggregated with Elo ratings, validly measure response quality.
    Sections 6.2 and 7.2 treat Elo ratings as the dependent variable; no validation against expert ratings is reported, and raters were incentivized to match unspecified expert ratings.
  • domain assumption The Stack Overflow iPhone questions and reference documents simulate a customer-support question-answering environment.
    Sections 3.1 and 8.4 state this simulation assumption; generalizability to other QA domains is asserted, not demonstrated.
  • domain assumption The behavior of GPT-4 in this setup represents conversational AI assistance more broadly.
    Section 8.4 acknowledges only one LLM was used; conclusions about human-AI collaboration are drawn from a single model.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Interaction Configurations and Prompt Guidance in Conversational AI for Question Answering in Human-AI Teams." pith.science (2026). https://pith.science/paper/ODKBW32N

@misc{pith2026250501648,
  author       = {Pith},
  title        = {Pith review of: Interaction Configurations and Prompt Guidance in Conversational AI for Question Answering in Human-AI Teams},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ODKBW32N}},
  note         = {Machine review of arXiv:2505.01648}
}
read the original abstract

Understanding the dynamics of human-AI interaction in question answering is crucial for enhancing collaborative efficiency. Extending from our initial formative study, which revealed challenges in human utilization of conversational AI support, we designed two configurations for prompt guidance: a Nudging approach, where the AI suggests potential responses for human agents, and a Highlight strategy, emphasizing crucial parts of reference documents to aid human responses. Through two controlled experiments, the first involving 31 participants and the second involving 106 participants, we compared these configurations against traditional human-only approaches, both with and without AI assistance. Our findings suggest that effective human-AI collaboration can enhance response quality, though merely combining human and AI efforts does not ensure improved outcomes. In particular, the Nudging configuration was shown to help improve the quality of the output when compared to AI alone. This paper delves into the development of these prompt guidance paradigms, offering insights for refining human-AI collaborations in conversational question-answering contexts and contributing to a broader understanding of human perceptions and expectations in AI partnerships.

Figures

Figures reproduced from arXiv: 2505.01648 by the authors.

Figure 1
Figure 1. Experiment Design and Study Conditions for Study 1 [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The detailed prompt provided to the AI agent for generating reference documents, including the title [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. The structured prompt used to provide the AI agent with a question and its corresponding reference [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Comparison of the Nudging and Highlight configurations in AI Assistant design. Nudging prompts users with additional cues to guide their interactions, while Highlight emphasizes specific parts of the reference document to aid user understanding. Design 1. Nudging Desig…
Figure 5
Figure 5. Figure 5: The prompt used to generate highlights of the reference document [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: The prompt used to generate responses for the AI-only condition [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 7
Figure 7. Figure 7: Experimental system interfaces used for each study condition [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: Distribution of ratings before and after showing sources for various study conditions (AI, Human, [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Between Knowledge and Care: A Mixed-Methods Evaluation of Generative AI for T2DM Self-Management from Patient and Physician Perspectives

    cs.HC 2026-07 conditional novelty 6.0 of 10

    Generative AI aids T2DM self-management on facts and lifestyle but fails on meds and emotion; patients and physicians converge on role limits, emotional gaps, and personalization needs, informing four design directions.

Reference graph

Works this paper leans on

66 extracted references · 52 canonical work pages · cited by 1 Pith paper

  1. [1]

    Martin Adam, Michael Wessel, and Alexander Benlian. 2021. AI-based chatbots in customer service and their effects on user compliance. Electronic Markets 31, 2 (2021), 427–445

    work page 2021

  2. [2]

    Ali Mohamed Nabil Allam and Mohamed Hassan Haggag. 2012. The question answering systems: A survey. International Journal of Research and Reviews in Information Sciences (IJRRIS) 2, 3 (2012)

    work page 2012

  3. [3]

    Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi Iqbal, Paul N Bennett, Kori Inkpen, et al. 2019. Guidelines for human-AI interaction. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–13

    work page 2019

  4. [4]

    Zahra Ashktorab, Mohit Jain, Q Vera Liao, and Justin D Weisz. 2019. Resilient chatbots: Repair strategy preferences for conversational breakdowns. In Proceedings of the 2019 CHI conference on human factors in computing systems. 1–12

    work page 2019

  5. [5]

    Zahra Ashktorab, Q Vera Liao, Casey Dugan, James Johnson, Qian Pan, Wei Zhang, Sadhana Kumaravel, and Murray Campbell. 2020. Human-ai collaboration in a cooperative game setting: Measuring social perception and outcomes. Proceedings of the ACM on Human-Computer Interaction 4, CSCW2 (2020), 1–20

    work page 2020

  6. [6]

    Hubert Baniecki, Bartlomiej Sobieski, Przemysław Bombiński, Patryk Szatkowski, and Przemysław Biecek. 2023. Hospital Length of Stay Prediction Based on Multi-modal Data Towards Trustworthy Human-AI Collaboration in Radiomics. In International Conference on Artificial Intelligence in Medicine. Springer, 65–74

    work page 2023

  7. [7]

    Madalina Busuioc. 2021. Accountable artificial intelligence: Holding algorithms to account. Public Administration Review 81, 5 (2021), 825–836

    work page 2021

  8. [8]

    Ángel Alexander Cabrera, Adam Perer, and Jason I Hong. 2023. Improving human-AI collaboration with descriptions of AI behavior. arXiv preprint arXiv:2301.06937 (2023)

    work page Pith review arXiv 2023

Show all 66 references
  1. [9]

    Andres Campero, Michelle Vaccaro, Jaeyoon Song, Haoran Wen, Abdullah Almaatouq, and Thomas W Malone. 2022. A test for evaluating performance in human-computer systems. arXiv preprint arXiv:2206.12390 (2022). Proc. ACM Hum.-Comput. Interact., Vol. 9, No. 7, Article CSCW305. Pub...

    2022 arXiv

  2. [10]

    Tara Capel and Margot Brereton. 2023. What is Human-Centered about Human-Centered AI? A Map of the Research Landscape. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–23

    2023

  3. [11]

    Ajay Chatterjee and Shubhashis Sengupta. 2020. Intent Mining from past conversations for Conversational Agent. In Proceedings of the 28th International Conference on Computational Linguistics. 4167–4178

    2020

  4. [12]

    Eunsol Choi, He He, Mohit Iyyer, Mark Yatskar, Wen-tau Yih, Yejin Choi, Percy Liang, and Luke Zettlemoyer. 2018. QuAC: Question answering in context. arXiv preprint arXiv:1808.07036 (2018)

    2018 arXiv

  5. [13]

    Alton YK Chua and Snehasish Banerjee. 2015. Answers or no answers: Studying question answerability in stack overflow. Journal of Information Science 41, 5 (2015), 720–731

    2015

  6. [14]

    Fatima Dakalbab, Manar Abu Talib, Omnia Abu Waraga, Ali Bou Nassif, Sohail Abbas, and Qassim Nasir. 2022. Artificial intelligence & crime prediction: A systematic literature review.Social Sciences & Humanities Open 6, 1 (2022), 100342

    2022

  7. [15]

    Paul Denny, Viraj Kumar, and Nasser Giacaman. 2023. Conversing with copilot: Exploring prompt engineering for solving cs1 problems using natural language. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1. 1136–1142

    2023

  8. [16]

    Anne R Diekema, Ozgur Yilmazel, and Elizabeth D Liddy. 2004. Evaluation of restricted domain question-answering systems. (2004)

    2004

  9. [17]

    Arpad E Elo. 1967. The proposed uscf rating system, its development, theory, and applications. Chess Life 22, 8 (1967), 242–247

    1967

  10. [18]

    Thomas Erickson, Catalina M Danis, Wendy A Kellogg, and Mary E Helander. 2008. Assistance: the work practices of human administrative assistants and their implications for it and organizations. In Proceedings of the 2008 ACM conference on Computer supported cooperative work. 609–618

    2008

  11. [19]

    Andy Hector, Stefanie Von Felten, and Bernhard Schmid. 2010. Analysis of variance with unbalanced data: an update for ecology & evolution. Journal of animal ecology 79, 2 (2010), 308–316

    2010

  12. [20]

    Noel L Hillman. 2019. The use of artificial intelligence in gauging the risk of recidivism. Judges J. 58 (2019), 36

    2019

  13. [21]

    Kenneth Holstein, Maria De-Arteaga, Lakshmi Tumati, and Yanghuidi Cheng. 2023. Toward supporting perceptual com- plementarity in human-AI collaboration via reflection on unobservables.Proceedings of the ACM on Human-Computer Interaction 7, CSCW1 (2023), 1–20

    2023

  14. [22]

    Abayomi Oluwaseyi Ikumoro and M. Jawad. 2019. Intention to Use Intelligent Conversational Agents in e-Commerce among Malaysian SMEs: An Integrated Conceptual Framework Based on Tri-theories including Unified Theory of Acceptance, Use of Technology (UTAUT), and T-O-E. Social Sc...

    2019

  15. [23]

    Kori Inkpen, Shreya Chappidi, Keri Mallari, Besmira Nushi, Divya Ramesh, Pietro Michelucci, Vani Mandava, Libuše Hannah Vepřek, and Gabrielle Quinn. 2022. Advancing Human-AI Complementarity: The Impact of User Expertise and Algorithmic Tuning on Joint Decision Making. ACM Tran...

    2022

  16. [24]

    Rune Møberg Jacobsen, Lukas Bjørn Leer Bysted, Patrick Skov Johansen, Eleftherios Papachristos, and Mikael B Skov

  17. [25]

    Chaitanya K Joshi, Fei Mi, and Boi Faltings. 2017. Personalization in goal-oriented dialog. arXiv preprint arXiv:1706.07503 (2017)

    2017 arXiv

  18. [26]

    Samia Kabir, David N Udo-Imeh, Bonan Kou, and Tianyi Zhang. 2024. Is stack overflow obsolete? an empirical study of the characteristics of chatgpt answers to stack overflow questions. In Proceedings of the CHI Conference on Human Factors in Computing Systems. 1–17

    2024

  19. [27]

    Subbarao Kambhampati. 2020. Challenges of human-aware ai systems: Aaai presidential address. AI Magazine 41, 3 (2020), 3–17

    2020

  20. [28]

    Diane Kelly, Karl Gyllstrom, and Earl W Bailey. 2009. A comparison of query and term suggestion features for interactive searching. In Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval. 371–378

    2009

  21. [29]

    Anjali Khurana, Hariharan Subramonyam, and Parmit K Chilana. 2024. Why and when llm-based assistants can go wrong: Investigating the effectiveness of prompt-based interactions for software help-seeking. In Proceedings of the 29th International Conference on Intelligent User In...

    2024

  22. [30]

    Najoung Kim, Ellie Pavlick, Burcu Karagol Ayan, and Deepak Ramachandran. 2021. Which linguist invented the lightbulb? presupposition verification for question-answering. arXiv preprint arXiv:2101.00391 (2021)

    2021 arXiv

  23. [31]

    Sotiris Kotsiantis, Dimitris Kanellopoulos, Panayiotis Pintelas, et al. 2006. Handling imbalanced datasets: A review. GESTS international transactions on computer science and engineering 30, 1 (2006), 25–36

    2006

  24. [32]

    Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. 2019. Natural questions: a benchmark for question answering research. Transactions of the Association for C...

    2019

  25. [33]

    Yi Lai, Atreyi Kankanhalli, and Desmond Ong. 2021. Human-AI collaboration in healthcare: A review and research agenda. (2021)

    2021

  26. [34]

    Yuping Liu-Thompkins, Shintaro Okazaki, and Hairong Li. 2022. Artificial empathy in marketing interactions: Bridging the human-AI gap in affective and social customer experience. Journal of the Academy of Marketing Science 50, 6 (2022), 1198–1218

    2022

  27. [35]

    Jon McCormack, Toby Gifford, Patrick Hutchings, Maria Teresa Llano Rodriguez, Matthew Yee-King, and Mark d’Inverno. 2019. In a silent way: Communication between ai and improvising musicians beyond sound. In Proceedings of the 2019 chi conference on human factors in computing s...

    2019

  28. [36]

    Piotr Mirowski, Kory W Mathewson, Jaylen Pittman, and Richard Evans. 2023. Co-Writing Screenplays and Theatre Scripts with Language Models: Evaluation by Industry Professionals. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–34

    2023

  29. [37]

    Diego Mollá and José Luis Vicedo. 2007. Question answering in restricted domains: An overview. Computational Linguistics 33, 1 (2007), 41–61

    2007

  30. [38]

    Mulesa, Mykola Kotsipak, Sergey Dolgikh, Y

    O. Mulesa, Mykola Kotsipak, Sergey Dolgikh, Y. Bilak, T. Radivilova, and Oleksii Baranovskyi. 2022. Collaborative Human-AI Decision-Making Systems with Numerical Channels. Blekinge Institute of Technology (2022). http: //bth.diva-portal.org/smash/get/diva2:1711729/FULLTEXT01

    2022

  31. [39]

    Imani Munyaka, Zahra Ashktorab, Casey Dugan, J Johnson, and Qian Pan. 2023. Decision Making Strategies and Team Efficacy in Human-AI Teams. Proceedings of the ACM on Human-Computer Interaction 7, CSCW1 (2023), 1–24

    2023

  32. [40]

    Nicolescu and Monica Teodora Tudorache

    L. Nicolescu and Monica Teodora Tudorache. 2022. Human-Computer Interaction in Customer Service: The Experience with AI Chatbots—A Systematic Literature Review. Symmetry 12, 12 (2022), 2045

    2022

  33. [41]

    Stefan Palan and Christian Schitter. 2018. Prolific. ac—A subject pool for online experiments. Journal of Behavioral and Experimental Finance 17 (2018), 22–27

    2018

  34. [42]

    Anastasia Pampouchidou, Panagiotis G Simos, Kostas Marias, Fabrice Meriaudeau, Fan Yang, Matthew Pediaditis, and Manolis Tsiknakis. 2017. Automatic assessment of depression based on visual cues: A systematic review. IEEE Transactions on Affective Computing 10, 4 (2017), 445–470

    2017

  35. [43]

    Luca Pappalardo, Paolo Cintia, Paolo Ferragina, Emanuele Massucco, Dino Pedreschi, and Fosca Giannotti. 2019. PlayeRank: data-driven performance evaluation and player ranking in soccer via a machine learning approach. ACM Transactions on Intelligent Systems and Technology (TIS...

    2019

  36. [44]

    Sun Young Park, Pei-Yi Kuo, Andrea Barbarin, Elizabeth Kaziunas, Astrid Chow, Karandeep Singh, Lauren Wilcox, and Walter S Lasecki. 2019. Identifying challenges and opportunities in human-AI collaboration in healthcare. In Conference Companion Publication of the 2019 on Comput...

    2019

  37. [45]

    Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018. Know what you don’t know: Unanswerable questions for SQuAD. arXiv preprint arXiv:1806.03822 (2018)

    2018 arXiv

  38. [46]

    Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250 (2016)

    2016 arXiv

  39. [47]

    Hima Bindu Sadashiva Reddy, Roopesh Reddy Sadashiva Reddy, and Ratnaditya Jonnalagadda. 2022. Literature review process: Measuring the effective usage of knowledge management systems in customer support organizations. Journal homepage: www. ijrpr. com ISSN 2582 (2022), 7421

    2022

  40. [48]

    Reverberi, T

    C. Reverberi, T. Rigon, A. Solari, C. Hassan, P. Cherubini, et al. 2022. Experimental evidence of effective human–AI collaboration in medical decision-making. Scientific Reports 12, 1 (2022), 1–14. https://www.nature.com/articles/ s41598-022-18751-2.pdf

    2022

  41. [49]

    Inger Roos, Martin Löfgren, and Bo Edvardsson. 2013. Customer-support service from a relationship perspective: Best practice for Telecom. Management Research and Practice 5, 2 (2013), 4

    2013

  42. [50]

    Neil Savage. 2023. Synthetic data could be better than real data. Nature (2023). doi:10.1038/d41586-023-01445-8

    2023 doi

  43. [51]

    Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2020. A human-centered review of algorithms used within the US child welfare system. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–15

    2020

  44. [52]

    Hanieh Shakeri, Carman Neustaedter, and Steve DiPaola. 2021. Saga: Collaborative storytelling with gpt-3. In Companion Publication of the 2021 Conference on Computer Supported Cooperative Work and Social Computing. 163–166

    2021

  45. [53]

    Anja Thieme, Maryann Hanratty, Maria Lyons, Jorge Palacios, Rita Faia Marques, Cecily Morrison, and Gavin Doherty

  46. [54]

    Qiongjie Tian, Peng Zhang, and Baoxin Li. 2013. Towards predicting the best answers in community-based question- answering services. In Proceedings of the International AAAI Conference on Web and Social Media, Vol. 7. 725–728. Proc. ACM Hum.-Comput. Interact., Vol. 9, No. 7, A...

    2013

  47. [55]

    Toader, G

    D. Toader, G. Boca, Rita Toader, Mara Macelaru, Cezar Toader, D. Ighian, and A. Rădulescu. 2019. The Effect of Social Presence and Chatbot Errors on Trust. Sustainability 11, 1 (2019), 256

    2019

  48. [56]

    Dakuo Wang, Elizabeth Churchill, Pattie Maes, Xiangmin Fan, Ben Shneiderman, Yuanchun Shi, and Qianying Wang

  49. [57]

    Dakuo Wang, Justin D Weisz, Michael Muller, Parikshit Ram, Werner Geyer, Casey Dugan, Yla Tausczik, Horst Samulowitz, and Alexander Gray. 2019. Human-ai collaboration in data science: Exploring data scientists’ perceptions of automated ai. Proceedings of the ACM on human-compu...

    2019

  50. [58]

    Chengyuan Xu, Kuo-Chin Lien, and Tobias Höllerer. 2023. Comparing Zealous and Restrained AI Recommendations in a Real-World Human-AI Collaboration Task. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–15

    2023

  51. [59]

    In Extended abstracts of the 2020 CHI conference on human factors in computing systems

    From human-human collaboration to Human-AI collaboration: Designing AI systems that can work together with people. In Extended abstracts of the 2020 CHI conference on human factors in computing systems. 1–6

    2020

  52. [60]

    Wei Xu, Marvin J Dainoff, Liezhong Ge, and Zaifeng Gao. 2023. Transitioning to human interaction with AI systems: New challenges and opportunities for HCI professionals to enable human-centered AI. International Journal of Human–Computer Interaction 39, 3 (2023), 494–518

    2023

  53. [61]

    Yingzi Xu, Chih-Hui Shieh, Patrick van Esch, and I-Ling Ling. 2020. AI customer service: Task complexity, problem- solving ability, and usage intention. Australasian marketing journal 28, 4 (2020), 189–199

    2020

  54. [62]

    Wei Xu, Marvin J Dainoff, Liezhong Ge, and Zaifeng Gao. 2021. From human-computer interaction to human-AI Interaction: new challenges and opportunities for enabling human-centered AI. arXiv preprint arXiv:2105.05424 5 (2021)

    2021 arXiv

  55. [63]

    JD Zamfirescu-Pereira, Richmond Y Wong, Bjoern Hartmann, and Qian Yang. 2023. Why Johnny can’t prompt: how non-AI experts try (and fail) to design LLM prompts. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Received July 2024; revised De...

    2023

  56. [65]

    Rui Yan and Dongyan Zhao. 2018. Smarter Response with Proactive Suggestion: A New Generative Neural Conversation Paradigm.. In IJCAI. 4525–4531

    2018

  57. [2020]

    In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems

    Perceived and measured task effectiveness in human-AI collaboration. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems. 1–9

    2020

  58. [2023]

    ACM Transactions on Computer-Human Interaction 30, 2 (2023), 1–50

    Designing human-centered AI for mental health: Developing clinically relevant applications for online CBT treatment. ACM Transactions on Computer-Human Interaction 30, 2 (2023), 1–50

    2023

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.