DialogXpert: Driving Intelligent and Emotion-Aware Conversations through Online Value-Based Reinforcement Learning with LLM Priors

Ambuj Mehrish; Lay-Ki Soon; Soujanya Poria; Tazeek Bin Abdur Rakib; Wern Han Lim

REVIEW 4 major objections 6 minor 16 references

DialogXpert: Driving Intelligent and Emotion-Aware Conversations through Online Value-Based Reinforcement Learning with LLM Priors

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read DialogXpert claims that a frozen LLM proposing top-k actions, scored by a small Q-network trained online with emotion tracking, makes goal-driven dialogues finish in under three turns with over 94% success, and that scaling the LLM prior…

desk verdict A clean and honest architecture for LLM-prior dialogue planning, but the headline 94–97% success rates are unvalidated because the same LLM critic that supplies training rewards also judges task success. read the letter →

arxiv 2505.17795 v1 pith:O2GXYW24 submitted 2025-05-23 cs.CL cs.AI

Tazeek Bin Abdur Rakib , Ambuj Mehrish , Lay-Ki Soon , Wern Han Lim , Soujanya Poria This is my paper · ORCID

classification cs.CLcs.AI

keywords dialogueplanninglargelanguagemodelsreinforcementlearningQ-learningemotion-awareLLMpriorsself-playgoal-drivenconversation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

DialogXpert claims that proactive dialogue does not need expensive full-tree search: a frozen LLM proposes a small set of candidate actions each turn, and a compact Q-network trained with temporal-difference learning selects among them, with the user's inferred emotion folded into the state. Across negotiation, emotional support, and tutoring benchmarks, the paper reports average conversations ending in under three turns with success rates above 94%, and above 97% when the LLM prior is scaled up. This matters because LLM agents are otherwise reactive or myopic, and prior planners either act greedily or pay the latency of Monte Carlo tree search. If true, the framework offers real-time, goal-driven, emotionally aware conversation planning at scale.

What carries the argument

The central object is the LLM Action Prior, a frozen LLM that takes the serialized case information, conversation history, and emotion history together with the full action inventory and returns a free-form continuation that is deterministically projected onto a valid action, implicitly defining a normalized prior over the action set. Beam search approximates this prior and yields the top-k candidate actions for the turn. A lightweight Q-network, built from a fixed BERT encoder followed by a three-layer MLP, scores those candidates, and softmax over the Q-values selects the executed action. The Q-network is updated online through temporal-difference Q-learning with Bellman backups; all LLM components and BERT remain frozen, so only the small value network changes during training.

What would settle it

Run the same DialogXpert policy against human users on negotiation and emotional support, counting actual task completion and user-rated satisfaction; if the sub-3-turn, above-94% success pattern does not appear with real users while the self-play critic still reports success, the reward and user-simulation chain is the point of failure. A cheaper probe is to change the CIMA reward mapping, for example correcting the 0.5 reward for partial translation, and check whether reported success changes materially.

Watch

Extended reading notes

Core claim

The central claim is that the usual trade-off between myopic generation and deep search can be broken by making a frozen LLM the action proposer rather than the decision-maker. DialogXpert prompts the LLM with the case, history, and accumulated user emotions, extracts a top-k set of semantically coherent actions via a free-form plus projection prior, and lets a BERT-embedded Q-network pick the best one using Q-learning with an epsilon-greedy policy. An LLM critic supplies scalar rewards for each transition, and an emotion tracker injects the inferred user feeling into the state representation. The paper shows that removing either the LLM prior or the emotion tracker degrades every metric, and that the Q-network trained on ExTES transfers to ESConv without fine-tuning, which argues the learned policy captures reusable dialogue skill rather than dataset memorization.

Load-bearing premise

The load-bearing premise is that the LLM critic's scalar rewards and the LLM-simulated user provide a valid and stable measurement of dialogue success and user satisfaction, so if either is biased the reported success rates may not transfer to real users.

Editorial extensions

If this is right

Under the reported protocol, planning needs only four LLM calls per turn instead of roughly thirty from MCTS rollouts, so the method is fast enough for real-time deployment.
Scaling the frozen LLM prior from 13B to 14B raises success from above 94% to above 97% and improves negotiation outcomes, making prior-model scale a direct lever.
Ablations show the LLM prior and emotion tracker are each load-bearing: dropping the prior raises average turns and lowers success, while dropping emotion tracking lowers success and, in negotiation, cuts the sale-to-list ratio from 0.4389 to 0.3156.
The ExTES-trained Q-network transfers to ESConv without fine-tuning, suggesting the learned state-action values generalize across similar task environments.
Human raters prefer DialogXpert over a supervised planner on emotional support and tutoring, so the efficiency gains are accompanied by perceived quality gains.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper's own limitations section concedes the critic "can behave inconsistently, sometimes terminating too early (e.g., in ESConv) or failing to end dialogues when goals are met (e.g., in CIMA)," so the reported turn counts and success rates partly inherit the critic's termination behavior rather than measuring a fixed task objective.
The same "frozen LLM proposes, small Q-network selects" pattern could be applied to other finite-action sequential decision problems, such as web navigation or tool use, wherever top-k sampling is cheap.
The emotion tracker's open-ended emotion space is an implementation choice; a lightweight classifier over a fixed emotion set would likely be more stable and cheaper, as the paper's own limitation discussion hints.
The reported four LLM calls per turn may understate runtime cost since the paper notes its dynamic state-action space prevents caching, so real-world latency depends on the full per-turn prompt cost.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

DialogXpert is a clean engineering combination: a frozen LLM proposes top-k dialogue actions, a small BERT-based Q-network scores them via TD learning, and an LLM emotion tracker feeds the state. The components are known, but the packaging is new and the code and prompts are released, which is real. The paper is honest enough to list limitations, including that the critic LLM can terminate early or miss goal attainment.

The soft spot is the evaluation. Success rate and average turns are measured in a closed self-play loop where the same LLM family acts as policy, user, and critic. The critic supplies the training reward and also the terminal 'Completed' judgment, so the reported 94–97% SR may mostly reflect policy-critic agreement, not real task success. The paper's own limitations concede the critic is inconsistent. The human evaluation only compares 20 dialogues against PPDPP on win/tie/loss; it does not validate the absolute numbers. There are also reporting errors: the 'Impact of Emotions' text gives CIMA numbers that don't match Table 1, and the 'Impact of LLM Prior' section cites the -w/o RL row as the -w/o LLM-Prior effect. No error bars, no seeds, and the test-time epsilon of 0.5 (50% random actions) is an unusual protocol that needs justification.

The central architecture is worth engaging with; the evaluation protocol needs external validation with human users or at least a held-out, independently labeled terminal state. The paper deserves peer review but with major revisions expected on the evaluation and reporting. I'd bring it to a reading group as a case study in self-play eval pitfalls.

Referee Report

4 major / 6 minor

Summary. The paper proposes DialogXpert, a dialogue planning framework that combines a frozen LLM action prior (top-k candidate actions per turn) with a lightweight Q-network trained via online temporal-difference learning, and an emotion-tracking module that feeds inferred user emotions into the state representation. The method is evaluated with self-play on negotiation (CraigslistBargain), emotional support (ESConv, ExTES, P4G), and tutoring (CIMA) benchmarks, reporting average turns below 3, success rates above 94%, and improved sale-to-list ratios. The paper also provides ablations of the emotion tracker, top-k size, and epsilon-greedy exploration, plus a cross-dataset generalization experiment from ExTES to ESConv.

Significance. If the reported results are valid, DialogXpert offers a computationally attractive alternative to MCTS-based planners, using only a handful of LLM calls per turn and a small trainable Q-network. The paper includes several strengths: the method is clearly described, the code is released, and the ablation study (e.g., Table 1 rows for -w/o Emotion, -w/o LLM-Prior) helps identify the contribution of each component. However, the headline claims hinge on the validity of an LLM-based critic that simultaneously provides training rewards and success-rate labels in a closed self-play loop. The paper's own Limitations section concedes that this critic is inconsistent. Without independent terminal-state labels or a human-validated success metric, the absolute SR and AT numbers remain unverified for real users, and the claimed superiority over baselines is therefore conditional on the self-play protocol being a faithful proxy for true task success.

major comments (4)

[§3.4, §5.2, Appendix B] The training reward and the reported success rate come from the same Critic LLM prompt family that also supplies the system and user roles during self-play. Section 5.2 explicitly states the LLM is 'shared across system, user, and critic roles.' This means the Q-network is trained to maximize rewards from this critic, and success is then measured by that same critic. Such circularity can inflate SR: the policy may learn to satisfy the critic's termination criterion rather than achieve genuine task goals. The paper's Limitations section admits the critic 'can behave inconsistently, sometimes terminating too early (e.g., in ESConv) or failing to end dialogues when goals are met (e.g., in CIMA).' The human evaluation in Appendix B only reports pairwise win/tie/loss on 20 dialogues and does not validate absolute SR or AT values. To support the abstract's claims, the authors should provide an independent ground-truth evaluation (e.g., human judges or rule-based terminal-state checks) for at least a subset of dialogues, and report SR and AT on those instances.
[§5.1, Table 1] No error bars, confidence intervals, or multiple-seed results are reported anywhere in Tables 1-4 or Figure 2. All numbers are point estimates from a single run (or an unreported number of runs). Given the stochasticity of LLM sampling, the simulated user, and the epsilon-greedy exploration (epsilon=0.5 at evaluation), the differences between configurations—e.g., SR 0.9876 vs 0.9805 for Qwen-2.5-14B vs Qwen-1.8B on ESConv—may be within noise. The paper should report means and variances over at least 3 seeds and state the number of evaluation episodes used for AT/SR/SL.
[§5.1, Figure 2, Table 1 footnote] The test-time epsilon is fixed at 0.5, meaning half of the actions are chosen uniformly at random among the top-4 candidates (unless the footnote implies epsilon applies only during training; the text is ambiguous). If 50% random actions still yield SR above 0.97, this raises questions about the sensitivity of the success metric to action quality. The paper should clarify whether epsilon is applied during evaluation, and should report results under pure exploitation (epsilon=0). If epsilon is applied at evaluation, the reported 'success' is not the success of the learned policy, but of a stochastic mixture that includes random actions.
[§3.2, Appendix A] The claim that the LLM prior's top-k set is sufficient for good performance is not verified. The paper approximates pproj(a|s_t) via beam search and extracts the top-k (k=4) actions, but never measures how often the optimal action (e.g., the action selected by the full prior or by an oracle) falls outside this candidate set. Without such a coverage analysis, the strong results only show that the Q-network selects well within the top-4, not that the top-4 always contains a good action. A concrete test would be to run an ablation with k equal to the full action set and compare SR/AT; this would separate the prior's coverage from the Q-network's selection ability.

minor comments (6)

[Abstract/§1] The phrase 'solving dialogues in under 3 turns' is imprecise because the measured metric is average turns, not the maximum or median; consider phrasing 'average dialogue length below 3 turns.'
[§2, Related Works] Typo: 'V oyager' should be 'Voyager' (also in the reference list).
[§4.3 / Appendix E] The reward mappings in Appendix E are not derived from any user study or calibration; the claim that the mapping is 'consistent with prior works' is not supported by direct citations to those mappings. Please provide the exact prompts and mapping tables used for PPDPP/DPDP/LDPP for comparability.
[§5.1, Table 2] The table reports LDPP results on ExTES (SR 0.903, AT 4.132) but the text says 'LDPP relies heavily on RoBERTa-based encoders/decoders'; it would be helpful to clarify the backbone used for LDPP in the table, since the original paper uses Qwen1-1.8B as the policy backbone.
[Appendix B] The text says 'four expert annotators' and that they were given '40 dialogue contexts in total' (20 ESConv and 20 CIMA), but the results section only reports ESConv in the main text; please include the CIMA human-evaluation results in the main paper or move them to a separate appendix with clear interpretation.
[§5.1, Generalization Test] The generalization result (ExTES-trained model evaluated on ESConv) is reported without any statistical significance test or comparison of the same critic distribution shift; please report the number of test dialogues and include a baseline of the critic's own agreement on the same dialogues.

Circularity Check

2 steps flagged · score 7.0 of 10

Success rates and negotiation gains are measured by the same LLM critic that supplies training rewards, so the headline 94–97% SR and <3-turn claims are not independent of the training signal.

self definitional [Section 3.4; Section 5.2; Appendix C]
"We then execute at in the environment, observe the next state st+1, and solicit a scalar reward rt from the Critic LLM, which assesses the transition (st, at, st+1) in terms of task effectiveness and emotional alignment. ... The same LLM is shared across system, user, and critic roles during self-play, ensuring stable memory usage and training efficiency."

The Q-network is trained on scalar rewards from the Critic LLM, and the same frozen LLM serves as system, user, and critic during self-play. Appendix C defines terminal states as 'Completed: the goal of the conversation is achieved' or 'Failed: the maximum number of turns are reached without the goal being completed,' and Appendix E maps the critic's categorical judgments to the scalar rewards used in Eq. (3). SR and AT therefore count how often the critic emits the 'Completed' label, exactly the signal the policy was optimized to produce. The Limitations concede the critic 'can behave inconsistently,' so the reported 94–97% SR and <3-turn values reflect policy-critic agreement rather than independent task success.
fitted input called prediction [Appendix E; Section 4.3; Abstract]
"CraigslistBargain (CB): If a deal is reached, we compute the sale-to-list price ratio as the reward. If no deal is made, the reward is set to 0. ... For the CraigslistBargain (CB) dataset, we also report the Sale-to-List Ratio (SL), indicating negotiation quality from the buyer's perspective—higher SL values represent better deals, while failed negotiations receive an SL of zero."

For CraigslistBargain, the training reward is defined as the sale-to-list price ratio computed by the Critic LLM when a deal is reached, and the reported evaluation metric SL is defined identically: higher SL = better deals, failed negotiations = 0. The abstract's claim of 'markedly improving negotiation outcomes' is therefore not an independent outcome measurement; it is the optimized objective itself, computed by the same critic that assigns the reward. Because the critic also decides whether a deal occurred and extracts the final price, the reported SL/SR gains can be inflated by critic leniency or extraction errors—a risk the Limitations acknowledge for the other datasets.

full rationale

DialogXpert's architecture is a legitimate engineering contribution: a frozen LLM action prior, a compact BERT-embedding Q-network, and an emotion tracker are combined with standard Q-learning and ablated across tasks. The algorithm's derivation is not circular, and no load-bearing self-citation chain is present: the cited planning baselines (PPDPP, DPDP, LDPP/UDP) are external works, and the comparisons are concrete. The circularity is located in the evaluation protocol. Section 3.4 trains the Q-network on Critic-LLM rewards; Appendix E maps the critic's categorical judgments to those rewards; Appendix C and Section 5.2 show the same LLM family, shared across system, user, and critic roles, supplies the terminal 'Completed/Failed' judgments that define SR and AT. Consequently the paper's strongest quantitative claims—SR >94% (over 97% with Qwen 2.5 14B) and AT under 3 turns—measure how well the policy satisfies the same critic that shaped it, not an independently verified notion of task success. The human evaluation only reports pairwise preferences on 20 ESConv and 20 CIMA dialogues and does not validate the absolute SR/AT numbers, and the Limitations explicitly admit the critic 'can behave inconsistently.' For CraigslistBargain, the reported SL improvement is the optimized reward reported under the same critic-extracted price. This is partial circularity: the method is real and the ablation logic is sound, but the headline numbers are not independent of the training signal.

Assumptions & free parameters 4 free parameters · 4 assumptions · 2 invented entities

The central claims rest on a set of hand-chosen hyperparameters (epsilon, top-k), hand-set reward mappings, and three domain assumptions: the MDP abstraction, the coverage of the LLM prior, and the validity of the LLM self-play loop. No totally new physical or mathematical entities are introduced; the Emotion Tracker is a prompt-based module without independent evidence.

free parameters (4)

epsilon (exploration rate) = 0.5 (evaluation), scheduled 1.0 to 0.1 (training)
Selected in Section 5.1 preliminaries as optimal and fixed for all evaluations; affects exploration-exploitation trade-off.
top-k candidate count = 4
Chosen empirically via Table 4 sweep; controls the size of the action candidate set.
reward mapping values per dataset = ESConv: -1.0,-0.5,0.5,1.0; CIMA: -1.0,-0.5,0.5,1.0; CB: sale-to-list ratio; P4G: -1.0,-0.5,0.1,1.0; ExTES: -1.0,0.5,1.0
Hand-specified in Appendix E; the paper notes these mappings are subjective and can misrepresent task success (Limitations).
discount factor gamma = 0.999
Set in Appendix C; standard RL hyperparameter, not fitted to data but affects credit assignment.

assumptions (4)

domain assumption Dialogue planning can be modeled as a Markov Decision Process with state, actions, rewards, and transitions as defined in Section 3.1.
Assumes the dialogue state is Markovian and that reward from the critic reflects goal achievement.
ad hoc to paper The LLM prior's top-k proposals contain the optimal action with sufficient probability.
The Q-network only selects among top-k candidates; if the optimal action is excluded, performance is capped. See Section 3.2 and the w/o LLM-Prior ablation.
domain assumption LLM-simulated users and the LLM critic faithfully represent real user behavior and task success.
Training and evaluation rely on self-play with LLM user and critic; Limitations admit critic inconsistency.
domain assumption Fixed BERT embeddings of serialized state and action tokens preserve enough information for value estimation.
The Q-network uses fixed BERT features with a 3-layer MLP; no fine-tuning, so representational sufficiency is assumed.

invented entities (2)

Emotion Tracker
purpose: Infers the user's emotional state each turn and folds it into the state representation for the Q-network.
Presented as a frozen LLM prompt (Section 3.3); its contribution is only evidenced through ablations within this paper.
LLM-Prior Planning Paradigm
purpose: A framework conjoining a frozen LLM action proposer, top-k candidate selection, and lightweight value learning.
A named combination of existing components (free-form + projection from Yan et al. 2024, Q-learning, BERT embeddings); no external validation of the paradigm beyond reported benchmarks.

how reviews work

0 comments

Cite this review

Pith. "Pith review of DialogXpert: Driving Intelligent and Emotion-Aware Conversations through Online Value-Based Reinforcement Learning with LLM Priors." pith.science (2026). https://pith.science/paper/O2GXYW24

@misc{pith2026250517795,
  author       = {Pith},
  title        = {Pith review of: DialogXpert: Driving Intelligent and Emotion-Aware Conversations through Online Value-Based Reinforcement Learning with LLM Priors},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/O2GXYW24}},
  note         = {Machine review of arXiv:2505.17795}
}

abstract

Large-language-model (LLM) agents excel at reactive dialogue but struggle with proactive, goal-driven interactions due to myopic decoding and costly planning. We introduce DialogXpert, which leverages a frozen LLM to propose a small, high-quality set of candidate actions per turn and employs a compact Q-network over fixed BERT embeddings trained via temporal-difference learning to select optimal moves within this reduced space. By tracking the user's emotions, DialogXpert tailors each decision to advance the task while nurturing a genuine, empathetic connection. Across negotiation, emotional support, and tutoring benchmarks, DialogXpert drives conversations to under $3$ turns with success rates exceeding 94\% and, with a larger LLM prior, pushes success above 97\% while markedly improving negotiation outcomes. This framework delivers real-time, strategic, and emotionally intelligent dialogue planning at scale. Code available at https://github.com/declare-lab/dialogxpert/

Figures

Figures reproduced from arXiv: 2505.17795 by the authors.

**Figure 1.** DialogXpert pipeline: case information and dialogue history drive user/system LLMs and an emotion tracker; a frozen LLM generates a prior over candidate actions, the top-k are evaluated by a Q-network and executed by the system LLM; a critic LLM provides reward signals to train the Q-network. logues. Specifically, two distinct LLM agents are used: one represents the user and the other the dialogue system, as illust… view at source ↗

**Figure 2.** Exploration vs. Exploitation: We use the Qwen 2.5 14B prior with top- [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

**Figure 3.** Win/tie/loss percentages for DialogXpert vs. PPDPP on ESConv across Identification, Comforting, Suggestion and Overall metrics. Human Evaluation To ensure a fair comparison, both DialogXpert and PPDPP were run with the same Vicuna-13B backbone on 20 ESConv [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Win/tie/loss percentages for DialogXpert vs. PPDPP on the CIMA tutoring dataset across Hint, Identification and Overall metrics [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

16 extracted references · 10 canonical work pages

[1]

Identification: Which response best ac- knowledges and accurately reflects the patient’s feelings?

work page
[2]

Comforting: Which response provides greater emotional support and reassur- ance?

work page
[3]

Suggestion: Which response offers more helpful and appropriate guidance?

work page
[4]

Shibo Hao, Yi Gu, Haodi Ma, Joshua Jiahua Hong, Zhen Wang, Daisy Zhe Wang, and Zhiting Hu

Chatgpt outperforms crowd workers for text-annotation tasks.Proceedings of the National Academy of Sciences, 120(30):e2305016120. Shibo Hao, Yi Gu, Haodi Ma, Joshua Jiahua Hong, Zhen Wang, Daisy Zhe Wang, and Zhiting Hu. 2023. Reasoning with language model is planning with world model.arXiv preprint arXiv:2305.14992. Peter E Hart, Nils J Nilsson, and Bert...

arXiv 2023
[5]

David Silver, Aja Huang, Christopher J

Direct preference optimization: Your language model is secretly a reward model.arXiv preprint arXiv:2305.18290. David Silver, Aja Huang, Christopher J. Maddison, Arthur Guez, Laurent Sifre, George van den Driess- che, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, I...

arXiv 2016
[7]

Conversational Tree Search: A New Hybrid Dialog Task

Conversational tree search: A new hybrid dialog task.arXiv preprint arXiv:2303.10227. Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Man- dlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and An- ima Anandkumar. 2023a. V oyager: An open-ended embodied agent with large language models.arXiv preprint arXiv:2305.16291. Lingzhi Wang, Mrinmaya Sachan, Xingshan Zeng, and ...

work page Pith review arXiv 2023
[8]

Xiao Yu, Maximillian Chen, and Zhou Yu

Tree of thoughts: Deliberate problem solving with large language models.Advances in neural information processing systems, 36:11809–11822. Xiao Yu, Maximillian Chen, and Zhou Yu. 2023. Prompt-based monte-carlo tree search for goal- oriented dialogue policy planning. InConference on Empirical Methods in Natural Language Process- ing. Ceyao Zhang, Kaijie Ya...

arXiv 2023
[12]

Your responses are auto-saved after each item

Overall: Which response do you find more effective overall? All four selections are required before moving on. Your responses are auto-saved after each item. If you exit, simply log back in with the sameUser IDto resume where you left off. CIMA Instructions You will see translation-tutoring dialogues. For each item, compareResponse Aand Response Bon these...

work page

Show all 16 references

[13]

Hint: Which assistant gives more help- ful hints for correct translation?
[14]

Identification: Which assistant better spots the student’s translation errors?
[15]

Responses are auto-saved; log back in with the sameUser IDto resume

Overall: Which assistant teaches more effectively? All three selections are required before moving on. Responses are auto-saved; log back in with the sameUser IDto resume. B.2 Results: CIMA On the CIMA tutoring task, we asked four anno- tators to compare DialogXpert and PPDPP ...

2024
[16]

Policy LLM for {dataset}

that generates natural language feedback at each turn. This textual evaluation is parsed and mapped into scalar rewards to supervise policy learning. Our reward design is consistent with prior works such as PPDPP, DPDP, LDPP, and UDP, en- suring comparability across benchmarks...
[2017]

InNeural Information Processing Systems

Thinking fast and slow with deep learning and tree search. InNeural Information Processing Systems. Nabiha Asghar, Ivan Kobyzev, Jesse Hoey, Pas- cal Poupart, and Muhammad Bilal Sheikh. 2020. Generating emotionally aligned responses in dia- logues using affect control theory.a...

2020 arXiv
[2020]

InProceedings of the Fifteenth Work- shop on Innovative Use of NLP for Building Educa- tional Applications, BEA@ACL 2020, pages 52–64

CIMA: A large open access dialogue dataset for tutoring. InProceedings of the Fifteenth Work- shop on Innovative Use of NLP for Building Educa- tional Applications, BEA@ACL 2020, pages 52–64. Gerald Tesauro and 1 others. 1995. Temporal difference learning and td-gammon.Communi...

2020 arXiv
[2022]

CoRR, abs/2210.04242

Improving multi-turn emotional support dia- logue generation with lookahead strategy planning. CoRR, abs/2210.04242. Yang Deng, Wenqiang Lei, Lizi Liao, and Tat-Seng Chua. 2023a. Prompting and evaluating large lan- guage models for proactive dialogues: Clarification, target-gu...

arXiv
[2023]

arXiv preprint arXiv:2305.10142

Improving language model negotiation with self-play and in-context learning from ai feedback. arXiv preprint arXiv:2305.10142. Chongming Gao, Wenqiang Lei, Xiangnan He, Maarten De Rijke, and Tat-Seng Chua. 2021. Advances and challenges in conversational recommender systems: A ...

2021 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Identification: Which response best ac- knowledges and accurately reflects the patient’s feelings?

work page

[2] [2]

Comforting: Which response provides greater emotional support and reassur- ance?

work page

[3] [3]

Suggestion: Which response offers more helpful and appropriate guidance?

work page

[4] [4]

Shibo Hao, Yi Gu, Haodi Ma, Joshua Jiahua Hong, Zhen Wang, Daisy Zhe Wang, and Zhiting Hu

Chatgpt outperforms crowd workers for text-annotation tasks.Proceedings of the National Academy of Sciences, 120(30):e2305016120. Shibo Hao, Yi Gu, Haodi Ma, Joshua Jiahua Hong, Zhen Wang, Daisy Zhe Wang, and Zhiting Hu. 2023. Reasoning with language model is planning with world model.arXiv preprint arXiv:2305.14992. Peter E Hart, Nils J Nilsson, and Bert...

arXiv 2023

[5] [5]

David Silver, Aja Huang, Christopher J

Direct preference optimization: Your language model is secretly a reward model.arXiv preprint arXiv:2305.18290. David Silver, Aja Huang, Christopher J. Maddison, Arthur Guez, Laurent Sifre, George van den Driess- che, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, I...

arXiv 2016

[6] [7]

Conversational Tree Search: A New Hybrid Dialog Task

Conversational tree search: A new hybrid dialog task.arXiv preprint arXiv:2303.10227. Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Man- dlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and An- ima Anandkumar. 2023a. V oyager: An open-ended embodied agent with large language models.arXiv preprint arXiv:2305.16291. Lingzhi Wang, Mrinmaya Sachan, Xingshan Zeng, and ...

work page Pith review arXiv 2023

[7] [8]

Xiao Yu, Maximillian Chen, and Zhou Yu

Tree of thoughts: Deliberate problem solving with large language models.Advances in neural information processing systems, 36:11809–11822. Xiao Yu, Maximillian Chen, and Zhou Yu. 2023. Prompt-based monte-carlo tree search for goal- oriented dialogue policy planning. InConference on Empirical Methods in Natural Language Process- ing. Ceyao Zhang, Kaijie Ya...

arXiv 2023

[8] [12]

Your responses are auto-saved after each item

Overall: Which response do you find more effective overall? All four selections are required before moving on. Your responses are auto-saved after each item. If you exit, simply log back in with the sameUser IDto resume where you left off. CIMA Instructions You will see translation-tutoring dialogues. For each item, compareResponse Aand Response Bon these...

work page