Pith. sign in

REVIEW 4 major objections 5 minor 48 references

Simulation-Free Hierarchical Latent Policy Planning for Proactive Dialogues

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read LDPP shows that proactive dialogue agents can discover fine-grained latent policies directly from raw, unlabeled dialogue records and learn to plan them offline, outperforming predefined-policy baselines and even ChatGPT when paired with…

desk verdict The latent-policy offline RL pipeline is a genuine contribution, but the headline numbers are all measured with the same ChatGPT critic used for training, so the size of the claimed win over ChatGPT is not yet established. read the letter →

arxiv 2412.14584 v1 pith:JJ2W5FOZ submitted 2024-12-19 cs.CL

Tao He , Lizi Liao , Yixin Cao , Yuanxing Liu , Yiheng Sun , Zerui Chen , Ming Liu , Bing Qin This is my paper · ORCID
classification cs.CL
keywords proactivedialoguelatentpolicyofflinereinforcementlearninghierarchicalRLVQ-VAEemotionsupportpersuasionP-Former
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that proactive dialogue systems—agents that must steer conversations toward goals such as emotional support or persuasion—do not need hand-crafted strategy taxonomies or simulated online training. It introduces LDPP, which mines fine-grained 'latent policies' directly from raw dialogue transcripts using a vector-quantized autoencoder, then learns to plan those policies with an offline hierarchical reinforcement learning algorithm. A frozen 1.8B-parameter LLM, guided by a small trainable adapter called P-Former, follows the latent policies to generate responses. On ExTES, ESConv, and P4G, LDPP reports success rates above all baselines, including a ChatGPT prompt baseline, suggesting that small models can surpass much larger ones when given a good policy planner.

What carries the argument

The load-bearing object is the learned latent policy codebook $\mathcal{Z} = \{Z_k\}_{k=1}^K$: a set of $K$ continuous policy vectors that, combined through a soft weighted sum of the encoder's distribution, represent fine-grained and mixed dialogue strategies. The P-Former (a stack of transformer layers with learnable policy tokens and cross-attention) converts a latent policy vector into tokens the frozen LLM can read, giving the generator latent-policy-following ability. The optimization machinery is offline hierarchical RL: at the high level, implicit Q-learning (IQL) with expectile regression learns $Q_\alpha$ and $V_\beta$ from ChatGPT rewards and weights the planner's updates by the advantage $\exp(Q_\alpha(h,z)-V_\beta(h))$; at the low level, REINFORCE applies the same advantage as the terminal reward for token-level generation with the LLM frozen.

What would settle it

Run LDPP and the strongest baselines in a user study with human participants (or with an independently calibrated, non-ChatGPT reward model) and compare success rates; if LDPP's advantage over the standard prompt shrinks or reverses, the latent-policy planning is optimizing the ChatGPT critic rather than genuinely better proactive behavior.

Watch

Extended reading notes

Core claim

LDPP's central claim is that dialogue policies can be represented as continuous latent vectors and discovered automatically from raw, unlabeled system utterances. A VQ-VAE-style encoder compresses each utterance into a distribution over a codebook and forms a policy vector as a weighted sum, allowing mixtures of strategies; the generator must reconstruct the utterance from history plus this vector, which forces the latent space to capture policy content. These vectors annotate the training dialogues, a policy planner is initialized by distilling the encoder and pre-trained with implicit Q-learning on ChatGPT-assigned per-turn rewards, and offline hierarchical RL (IQL at the policy level, REINFORCE at the token level) jointly improves the planner and response generation. The paper reports SSR/SR of 0.723/0.903 on ExTES, 0.651/0.781 zero-shot on ESConv, and 0.733/0.795 on P4G, surpassing all baselines in Table 1, including a standard ChatGPT prompt, with human evaluation agreeing on the overall quality.

Load-bearing premise

The ChatGPT critic's four-state judgments with hand-set rewards (worse/same/better/solved mapped to -1, -0.5, 0.1, 1.0) are assumed to be a valid reward signal for RL training and a valid measure of dialogue success, and the same critic is used for both, so if this proxy diverges from real user outcomes the reported gains may not transfer.

Editorial extensions

If this is right

  • Dialogue agents for emotional support and persuasion can be built from existing corpora rather than expert-defined strategy sets, removing a major bottleneck for new proactive tasks.
  • Small frozen LLMs with a lightweight planner can outperform much larger prompted models, cutting the cost of deployment.
  • The same planner trained on ExTES transfers zero-shot to ESConv, suggesting discovered latent policies capture task-general supportive behaviors.
  • Offline training from static transcripts avoids the sim-to-real gap and API expense of online self-play used by prior methods.
  • Because latent policies are combinations of codebook entries, the representation can express mixed strategies that coarse discrete taxonomies miss.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The ChatGPT critic is used both as the RL reward and as the evaluation judge, so the reported advantage may partly reflect overfitting to that critic's notion of 'solved'; an evaluation with human judges or an independent reward model would test whether the gains persist outside the LLM-simulated loop.
  • The latent policies are currently opaque; pairing each codebook entry with natural-language descriptions generated from its top utterances (as the paper's Table 12 begins to do) is a natural step toward explainable, auditable dialogue strategies.
  • The framework's recipe—autoencoded action space, offline hierarchical RL, frozen text generator plus adapter—is generic and could be applied to other sequential text tasks with incomplete policy taxonomies, such as tutoring or negotiation, which are also characterized by proactive goals.
  • One testable implication is that the planner should be able to trade off exploration of rare latent policies against exploitation, which could be checked by measuring success as the codebook size $K$ is varied and the reward threshold is moved.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes LDPP, a three-stage framework for proactive dialogue. Stage 1 discovers fine-grained latent policies from raw system utterances with a VQ-VAE-style encoder and codebook, using a P-Former to inject the continuous policy into a frozen LLM. Stage 2 distills the encoder into a policy planner and pre-trains Q/V networks with IQL, filtering low-reward samples. Stage 3 applies offline hierarchical RL, using IQL-style advantage-weighted updates for the planner and REINFORCE with an exponential-advantage final reward for the generator. Experiments on ExTES, ESConv, and P4G report SSR/SR improvements over predefined-policy, prompt-based, and fine-tuning baselines, including a claim of surpassing ChatGPT with a 1.8B Qwen backbone, along with ablations, human evaluation, and a second-critic check in the appendix.

Significance. If the reported results hold, the paper makes a useful contribution: it removes the need for manually defined policy taxonomies and online interaction for policy planning, and it demonstrates that latent-policy planning can be learned offline and transferred zero-shot. The framework is accompanied by code, an ablation of each stage, sensitivity analyses, and a worked REINFORCE derivation for the token-level MDP. However, the central empirical claim currently rests on a self-play evaluation whose user and critic are from the same ChatGPT family that also supplies training rewards; the paper's own second-critic result does not break that coupling. The significance of the results for real deployment therefore remains to be established.

major comments (4)
  1. [Section 'Evaluation Methods', 'Critic model', and Eqs. (2), (3), (5), (6)] The same model, gpt-3.5-turbo-0613, is used to assign per-turn rewards r_t on the static dataset and to compute success judgments in the self-play evaluation, and the user simulator is also ChatGPT. Because the reward filter (Eq. 2), the IQL targets (Eq. 3), and the advantage weights (Eqs. 5-6) all maximize this critic's rating, the reported SSR/SR gains in Table 1 may partly reflect overfitting to this specific critic rather than general dialogue quality. Table 6 replaces the critic with gpt-3.5-turbo-0125, but that is still the same model family and the same self-play protocol; it does not provide an independent gold standard. I ask for an evaluation with a critic from a different family or with human-annotated turn-level outcomes, and for a report of the correlation between the critic and human judgments.
  2. [Section 'Main Results' and Section 'Evaluation Methods'] The paper reports that main experiments were run 'at least twice' and averages, but no variance, standard errors, or significance tests are given anywhere. Given the stochasticity of LLM self-play and the fact that each critic score is an average of 10 classifications, the gaps in Table 1 (e.g., LDPP SSR 0.723 vs Standard+ChatGPT 0.650) need error bars and a paired significance test before 'outperforms all baselines significantly' is supportable. Please provide per-run results and bootstrap or similar confidence intervals.
  3. [Appendix, 'Data Augmentation for P4G', and Abstract] The paper repeatedly claims to be 'simulation-free' and to avoid 'dynamic interactions with simulated environments,' yet the P4G training set is expanded from 817 to 5,579 dialogues by ChatGPT self-play that completes partial dialogues (first 2-8 turns). Since the P4G test evaluation also uses ChatGPT self-play, the reported P4G gains (Table 1) may partly reflect alignment with the simulation protocol rather than the offline algorithm. Either report P4G results without the synthetic augmentation, or revise the simulation-free claim and discuss the effect of the augmentation.
  4. [Section 'Human Evaluation' and Table 2] The human study is limited to 50 dialogues per task and to two baselines (PPDPP and LoRA), with no comparison against the ChatGPT prompt baseline that is central to the abstract's 'surpassing ChatGPT' claim, and no significance testing or inter-annotator agreement reported. Moreover, on ExTES, LDPP is not preferred over PPDPP on Identification (8% vs 8%) and is only modestly preferred on Comforting, so the human evidence is weaker than the self-play numbers and should be presented more cautiously.
minor comments (5)
  1. [Eq. (5)] The term p_phi(h_t | z_t) should almost certainly be p_phi(z_t | h_t); as written, the high-level objective uses a reversed conditional that is inconsistent with the planner's input. Please correct this notation.
  2. [Figure 2] The right panel of Figure 2 labels the method 'LDDP'; this should be 'LDPP'.
  3. [Appendix, 'Proof and Explanation of Token-level MDP'] The sentence 'According to Section ,' contains an empty cross-reference; please fill in the intended section number.
  4. [Section 'Baselines'] The phrase 'shorted as LoRA' should be 'shortened as LoRA'; the same typo appears in the Human Evaluation appendix.
  5. [Table 1 caption] The caption says ESConv results use the planner trained on ExTES; since the column header already says 'Generalization to ESConv', please make the zero-shot nature of this column explicit to avoid confusion.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity; central derivation is self-contained, with a minor self-citation and a critic-coupling validity caveat.

full rationale

The paper's derivation chain is not circular. Stage 1 discovers latent policies with a VQ-VAE-style reconstruction objective, so the 'policies' are defined by their ability to reconstruct observed utterances, not by the evaluation metric; this is self-supervised and independent of the headline success measure. Stages 2-3 train the planner, Q/V networks, and P-Former with IQL and REINFORCE using rewards rt supplied by ChatGPT, and the headline SSR/SR in Table 1 are also computed from the same critic's four-state judgments (e.g., 'We also use ChatGPT to assess dialogue completion status following PPDPP'; 'we employ this method to preprocess the static dataset D ... We evaluated each dialogue turn in D to determine the reward'). This same-critic training/evaluation coupling is a genuine external-validity limitation—the paper even admits that SR evaluation rewards quick suggestions over thorough inquiry—but it is not a logical reduction: test dialogues are held out, all baselines are scored by the same critic, and the paper provides a second critic (gpt-3.5-turbo-0125, Table 6) and a human evaluation (Table 2) as checks. No equation or predicted quantity is equivalent to its input by construction. The only self-citation (He et al. 2024) is used for the human-evaluation protocol and is not load-bearing for the main claims.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

LDPP's central claim rests on domain assumptions that (a) reconstruction-based latent policies capture actionable strategies, (b) ChatGPT's state classification is a faithful reward/outcome signal, and (c) self-play with a ChatGPT user approximates real users. These are not validated against real user outcomes. The RL derivations themselves are standard.

free parameters (3)
  • Success threshold eta = 0.6
    Used to binarize ChatGPT critic rewards into success/failure for training rewards and evaluation. The paper changes it from PPDPP's 0.1, affecting reported SR.
  • Distillation reward threshold delta = 0.1 (ExTES), -1.1 (P4G)
    Filters which utterances are used to distill the policy planner in Stage 2 (Eq. 2).
  • Critic state reward values = [-1, -0.5, 0.1, 1.0]
    Hand-assigned rewards to the four dialogue states used to compute per-turn rewards rt.
assumptions (4)
  • domain assumption Latent dialogue policies can be discovered by reconstructing system utterances from a weighted combination of codebook vectors given dialogue history (Stage 1 premise).
    The VQ-VAE variant assumes the reconstruction objective yields policies that are meaningfully aligned with dialogue strategies; the paper validates this only via utterance clustering visualization (Fig. 3).
  • domain assumption ChatGPT's judgment of dialogue state is a valid reward signal for training and for evaluating success.
    The whole RL pipeline (Stage 2 filtering, Q/V targets, and final SR/SSR metrics) relies on ChatGPT's four-state classification as ground-truth-like reward and outcome. This is an unvalidated proxy for real user outcomes.
  • domain assumption Self-play with a ChatGPT user simulator approximates real user behavior closely enough to measure task success.
    Evaluation uses two LLMs playing system and user; the resulting success rates are treated as the headline comparison (Table 1). The paper notes the gap between simulated and real interactions for prior methods but applies the same self-play protocol for evaluation.
  • standard math The token-level MDP formulation with a single terminal reward exp(A(h,z)) and gamma=1 yields a valid policy gradient objective.
    The appendix derives Llow from REINFORCE; this is a correct algebraic reduction under the stated reward scheme.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Simulation-Free Hierarchical Latent Policy Planning for Proactive Dialogues." pith.science (2026). https://pith.science/paper/JJ2W5FOZ

@misc{pith2026241214584,
  author       = {Pith},
  title        = {Pith review of: Simulation-Free Hierarchical Latent Policy Planning for Proactive Dialogues},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JJ2W5FOZ}},
  note         = {Machine review of arXiv:2412.14584}
}
read the original abstract

Recent advancements in proactive dialogues have garnered significant attention, particularly for more complex objectives (e.g. emotion support and persuasion). Unlike traditional task-oriented dialogues, proactive dialogues demand advanced policy planning and adaptability, requiring rich scenarios and comprehensive policy repositories to develop such systems. However, existing approaches tend to rely on Large Language Models (LLMs) for user simulation and online learning, leading to biases that diverge from realistic scenarios and result in suboptimal efficiency. Moreover, these methods depend on manually defined, context-independent, coarse-grained policies, which not only incur high expert costs but also raise concerns regarding their completeness. In our work, we highlight the potential for automatically discovering policies directly from raw, real-world dialogue records. To this end, we introduce a novel dialogue policy planning framework, LDPP. It fully automates the process from mining policies in dialogue records to learning policy planning. Specifically, we employ a variant of the Variational Autoencoder to discover fine-grained policies represented as latent vectors. After automatically annotating the data with these latent policy labels, we propose an Offline Hierarchical Reinforcement Learning (RL) algorithm in the latent space to develop effective policy planning capabilities. Our experiments demonstrate that LDPP outperforms existing methods on two proactive scenarios, even surpassing ChatGPT with only a 1.8-billion-parameter LLM.

Figures

Figures reproduced from arXiv: 2412.14584 by the authors.

Figure 1
Figure 1. The training process of the LDPP framework. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Performance comparison as the LLM size and [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Visualization of latent policies for utterances be [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

48 extracted references · 25 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Bai, J.; Bai, S.; Chu, Y.; Cui, Z.; Dang, K.; Deng, X.; Fan, Y.; Ge, W.; Han, Y.; Huang, F.; Hui, B.; Ji, L.; Li, M.; Lin, J.; Lin, R.; Liu, D.; Liu, G.; Lu, C.; Lu, K.; Ma, J.; Men, R.; Ren, X.; Ren, X.; Tan, C.; Tan, S.; Tu, J.; Wang, P.; Wang, S.; Wang, W.; Wu, S.; Xu, B.; Xu, J.; Yang, A.; Yang, H.; Yang, J.; Yang, S.; Yao, Y.; Yu, B.; Yuan, H.; Yuan,...

    arXiv 2023

  4. [4]

    Bao, S.; He, H.; Wang, F.; and Wu, H. 2019. PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variable. In Annual Meeting of the Association for Computational Linguistics

    work page 2019

  5. [5]

    R.; Vilnis, L.; Vinyals, O.; Dai, A

    Bowman, S. R.; Vilnis, L.; Vinyals, O.; Dai, A. M.; J \'o zefowicz, R.; and Bengio, S. 2015. Generating Sentences from a Continuous Space. In Conference on Computational Natural Language Learning

    work page 2015

  6. [6]

    Chen, M.; Yu, X.; Shi, W.; Awasthi, U.; and Yu, Z. 2023. Controllable mixed-initiative dialogue generation through prompting. arXiv preprint arXiv:2305.04147

    work page Pith review arXiv 2023

  7. [7]

    Chen, W.; Gong, Y.; Wang, S.; Yao, B.; Qi, W.; Wei, Z.; Hu, X.-M.; Zhou, B.; Mao, Y.; Chen, W.; Cheng, B.; and Duan, N. 2022. DialogVED: A Pre-trained Latent Variable Encoder-Decoder Model for Dialog Response Generation. In Annual Meeting of the Association for Computational Linguistics

    work page 2022

  8. [8]

    Cheng, Y.; Liu, W.; Li, W.; Wang, J.; Zhao, R.; Liu, B.; Liang, X.; and Zheng, Y. 2022. Improving multi-turn emotional support dialogue generation with lookahead strategy planning. arXiv preprint arXiv:2210.04242

    arXiv 2022

Show all 48 references
  1. [9]

    T.; Ouyang, Y.; Li, W.; Wu, X.; and Zheng, Y

    Cheng, Y.; Liu, W.; Wang, J.; Leong, C. T.; Ouyang, Y.; Li, W.; Wu, X.; and Zheng, Y. 2024. Cooper: Coordinating specialized agents towards a complex dialogue goal. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 17853--17861

    2024

  2. [10]

    Cho, I.; Takahashi, R.; Yanase, Y.; and Saito, H. 2023. Deep RL with Hierarchical Action Exploration for Dialogue Generation. ArXiv, abs/2303.13465

    2023 arXiv

  3. [11]

    Deng, Y.; Lei, W.; Liao, L.; and Chua, T.-S. 2023 a . Prompting and Evaluating Large Language Models for Proactive Dialogues: Clarification, Target-guided, and Non-collaboration. arXiv preprint arXiv:2305.13626

    2023 arXiv

  4. [12]

    Deng, Y.; Zhang, W.; Lam, W.; Ng, S.-K.; and Chua, T.-S. 2023 b . Plug-and-play policy planner for large language model powered dialogue agents. In The Twelfth International Conference on Learning Representations

    2023

  5. [13]

    Fu, Y.; Peng, H.; Khot, T.; and Lapata, M. 2023. Improving language model negotiation with self-play and in-context learning from ai feedback. arXiv preprint arXiv:2305.10142

    2023 arXiv

  6. [14]

    He, H.; Chen, D.; Balakrishnan, A.; and Liang, P. 2018. Decoupling Strategy and Generation in Negotiation Dialogues. ArXiv, abs/1808.09637

    2018 arXiv

  7. [15]

    He, T.; Liao, L.; Cao, Y.; Liu, Y.; Liu, M.; Chen, Z.; and Qin, B. 2024. Planning Like Human: A Dual-process Framework for Dialogue Planning. arXiv preprint arXiv:2406.05374

    2024 arXiv

  8. [16]

    E.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; and Chen, W

    Hu, J. E.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; and Chen, W. 2021. LoRA: Low-Rank Adaptation of Large Language Models. ArXiv, abs/2106.09685

    2021 arXiv

  9. [17]

    T.; and Hooi, B

    Hu, Z.; Feng, Y.; Deng, Y.; Li, Z.; Ng, S.-K.; Luu, A. T.; and Hooi, B. 2023. Enhancing Large Language Model Induced Task-Oriented Dialogue Systems Through Look-Forward Motivated Goals. arXiv preprint arXiv:2309.08949

    2023 arXiv

  10. [18]

    Kahneman, D. 2003. Maps of Bounded Rationality: Psychology for Behavioral Economics. The American Economic Review, 93: 1449--1475

    2003

  11. [19]

    Kang, D.; Kim, S.; Kwon, T.; Moon, S.; Cho, H.; Yu, Y.; Lee, D.; and Yeo, J. 2024. Can Large Language Models be Good Emotional Supporter? Mitigating Preference Bias on Emotional Support Conversation. arXiv preprint arXiv:2402.13211

    2024 arXiv

  12. [20]

    P.; and Welling, M

    Kingma, D. P.; and Welling, M. 2013. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114

    2013 arXiv

  13. [21]

    Kostrikov, I.; Nair, A.; and Levine, S. 2021. Offline Reinforcement Learning with Implicit Q-Learning. ArXiv, abs/2110.06169

    2021 arXiv

  14. [22]

    Li, J.; Li, D.; Savarese, S.; and Hoi, S. C. H. 2023. BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models. In International Conference on Machine Learning

    2023

  15. [23]

    P.; Dieskau, J.; Hunermund, M.; Mostaghim, S.; and Lucas, S

    Liebana, D. P.; Dieskau, J.; Hunermund, M.; Mostaghim, S.; and Lucas, S. M. M. 2015. Open Loop Search for General Video Game Playing. Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation

    2015

  16. [24]

    Liu, A.; Wang, B.; Tan, Y.; Zhao, D.; Huang, K.; He, R.; and Hou, Y. 2023. MTGP: Multi-turn Target-oriented Dialogue Guided by Generative Global Path with Flexible Turns. In Findings of the Association for Computational Linguistics: ACL 2023, 259--271

    2023

  17. [25]

    Liu, J.; Pan, F.; and Luo, L. 2020. GoChat: Goal-oriented Chatbots with Hierarchical Reinforcement Learning. Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval

    2020

  18. [26]

    Liu, S.; Zheng, C.; Demasi, O.; Sabour, S.; Li, Y.; Yu, Z.; Jiang, Y.; and Huang, M. 2021 a . Towards emotional support dialog systems. arXiv preprint arXiv:2106.01144

    2021 arXiv

  19. [27]

    Liu, S.; Zheng, C.; Demasi, O.; Sabour, S.; Li, Y.; Yu, Z.; Jiang, Y.; and Huang, M. 2021 b . Towards Emotional Support Dialog Systems. In Annual Meeting of the Association for Computational Linguistics

    2021

  20. [28]

    My nose is running

    Liu, W.; Cheng, Y.; Wang, H.; Tang, J.; Liu, Y.; Zhao, R.; Li, W.; Zheng, Y.; and Liang, X. 2022. " My nose is running."" Are you also coughing?": Building A Medical Diagnosis Agent with Interpretable Inquiry Logics. arXiv preprint arXiv:2204.13953

    2022 arXiv

  21. [29]

    Liu, Y.; Ott, M.; Goyal, N.; Du, J.; Joshi, M.; Chen, D.; Levy, O.; Lewis, M.; Zettlemoyer, L.; and Stoyanov, V. 2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach. ArXiv, abs/1907.11692

    2019 arXiv

  22. [30]

    Lubis, N.; Geishauser, C.; Heck, M.; Lin, H.-C.; Moresi, M.; van Niekerk, C.; and Gavsi'c, M. 2020. LAVA: Latent Action Spaces via Variational Auto-encoding for Dialogue Policy Optimization. ArXiv, abs/2011.09378

    2020 arXiv

  23. [31]

    Mesnard, G. T. T.; Hardin, C.; Dadashi, R.; Bhupatiraju, S.; Pathak, S.; Sifre, L.; Riviere, M.; Kale, M.; Love, J. C.; Tafti, P. D.; Hussenot, L.; Chowdhery, A.; Roberts, A.; Barua, A.; Botev, A.; Castro-Ros, A.; Slone, A.; H'eliou, A.; Tacchetti, A.; Bulanova, A.; Paterson, ...

    2024 arXiv

  24. [32]

    L.; Mishkin, P.; Zhang, C.; Agarwal, S.; Slama, K.; Ray, A.; Schulman, J.; Hilton, J.; Kelton, F.; Miller, L

    Ouyang, L.; Wu, J.; Jiang, X.; Almeida, D.; Wainwright, C. L.; Mishkin, P.; Zhang, C.; Agarwal, S.; Slama, K.; Ray, A.; Schulman, J.; Hilton, J.; Kelton, F.; Miller, L. E.; Simens, M.; Askell, A.; Welinder, P.; Christiano, P. F.; Leike, J.; and Lowe, R. J. 2022. Training langu...

    2022 arXiv

  25. [33]

    M.; Mishra, K.; Firdaus, M.; and Ekbal, A

    Samad, A. M.; Mishra, K.; Firdaus, M.; and Ekbal, A. 2022. Empathetic persuasion: reinforcing empathy and persuasiveness in dialogue systems. In Findings of the Association for Computational Linguistics: NAACL 2022, 844--856

    2022

  26. [34]

    R.; Kumar, N.; and De Choudhury, M

    Song, I.; Pendse, S. R.; Kumar, N.; and De Choudhury, M. 2024. The typing cure: Experiences with large language model chatbots for mental health support. arXiv preprint arXiv:2401.14362

    2024 arXiv

  27. [35]

    S.; McAllester, D

    Sutton, R. S.; McAllester, D. A.; Singh, S.; and Mansour, Y. 1999. Policy Gradient Methods for Reinforcement Learning with Function Approximation. In Neural Information Processing Systems

    1999

  28. [36]

    van den Oord, A.; Kalchbrenner, N.; and Kavukcuoglu, K. 2016. Pixel Recurrent Neural Networks. In International Conference on Machine Learning

    2016

  29. [37]

    Wang, J.; Zhang, Y.; Kim, T.-K.; and Gu, Y. 2020. Modelling Hierarchical Structure between Dialogue Policy and Natural Language Generator with Option Framework for Task-oriented Dialogue System. ArXiv, abs/2006.06814

    2020 arXiv

  30. [39]

    J.; Yang, S.; Zhang, J.; and Yu, Z

    Wang, X.; Shi, W.; Kim, R.; Oh, Y. J.; Yang, S.; Zhang, J.; and Yu, Z. 2019 b . Persuasion for Good: Towards a Personalized Persuasive Dialogue System for Social Good. ArXiv, abs/1906.06725

    2019 arXiv

  31. [40]

    Xiang, J.; Liu, Z.; Liu, H.; Bai, Y.; Cheng, J.; and Chen, W. 2024. DiffusionDialog: A Diffusion Model for Diverse Dialog Generation with Latent Space. In International Conference on Language Resources and Evaluation

    2024

  32. [41]

    Yang, Y.; Li, Y.; and Quan, X. 2021. Ubar: Towards fully end-to-end task-oriented dialog system with gpt-2. In Proceedings of the AAAI conference on artificial intelligence, volume 35, 14230--14238

    2021

  33. [42]

    Yu, X.; Chen, M.; and Yu, Z. 2023. Prompt-Based Monte-Carlo Tree Search for Goal-Oriented Dialogue Policy Planning. In Conference on Empirical Methods in Natural Language Processing

    2023

  34. [43]

    Zhang, Q.; Naradowsky, J.; and Miyao, Y. 2023. Ask an Expert: Leveraging Language Models to Improve Strategic Reasoning in Goal-Oriented Dialogue Models. In Annual Meeting of the Association for Computational Linguistics

    2023

  35. [44]

    Zhang, T.; Huang, C.; Deng, Y.; Liang, H.; Liu, J.; Wen, Z.; Lei, W.; and Chua, T.-S. 2024. Strength Lies in Differences! Towards Effective Non-collaborative Dialogues via Tailored Strategy Planning. arXiv preprint arXiv:2403.06769

    2024 arXiv

  36. [45]

    Zhao, T.; Zhao, R.; and Esk \'e nazi, M. 2017. Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders. In Annual Meeting of the Association for Computational Linguistics

    2017

  37. [46]

    Zhao, W.; Zhao, Y.; Lu, X.; Wang, S.; Tong, Y.; and Qin, B. 2023. Is ChatGPT equipped with emotional dialogue capabilities? arXiv preprint arXiv:2304.09582

    2023 arXiv

  38. [48]

    Zheng, Z.; Liao, L.; Deng, Y.; and Nie, L. 2023 b . Building emotional support chatbots in the era of llms. arXiv preprint arXiv:2308.11584

    2023 arXiv

  39. [49]

    W.; and Tsvetkov, Y

    Zhou, Y.; He, H.; Black, A. W.; and Tsvetkov, Y. 2019. A dynamic strategy coach for effective negotiation. arXiv preprint arXiv:1909.13426

    2019 arXiv

  40. [50]

    Zhou, Y.; Zanette, A.; Pan, J.; Levine, S.; and Kumar, A. 2024. ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RL. arXiv preprint arXiv:2402.19446

    2024 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.