REVIEW 4 cited by
Prompting and Evaluating Large Language Models for Proactive Dialogues: Clarification, Target-guided, and Non-collaboration
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Conversational systems based on Large Language Models (LLMs), such as ChatGPT, show exceptional proficiency in context understanding and response generation. However, despite their impressive capabilities, they still possess limitations, such as providing randomly-guessed answers to ambiguous queries or failing to refuse users' requests, both of which are considered aspects of a conversational agent's proactivity. This raises the question of whether LLM-based conversational systems are equipped to handle proactive dialogue problems. In this work, we conduct a comprehensive analysis of LLM-based conversational systems, specifically focusing on three aspects of proactive dialogue systems: clarification, target-guided, and non-collaborative dialogues. To trigger the proactivity of LLMs, we propose the Proactive Chain-of-Thought prompting scheme, which augments LLMs with the goal planning capability over descriptive reasoning chains. Empirical findings are discussed to promote future studies on LLM-based proactive dialogue systems.
Forward citations
Cited by 4 Pith papers
-
ProEvent: An Event-centric Benchmark for Proactive Agents
ProEvent is a benchmark showing LLM agents keep a user's event timetable from chats poorly, with the best fully-correct score at 27.2%.
-
ProactiveEval: A Unified Evaluation Framework for Proactive Dialogue Agents
A unified evaluation framework for proactive dialogue agents, built with 328 synthetic environments across six domains, shows that thinking modes improve target planning but not dialogue guidance in a 22-model comparison.
-
Referential ambiguity and clarification requests: comparing human and LLM behaviour
Humans seldom ask clarification questions for referential ambiguity, while LLMs ask them more often, and reasoning prompts increase LLM question frequency and relevance.
-
Spiritual-LLM : Gita Inspired Mental Health Therapy In the Era of LLMs
A Gita-based mental-health dialogue dataset helps small LLMs score higher on spirituality-oriented metrics, but the evaluation loop is largely self-referential.
Discussion (0). Sign in to comment.