Pith. sign in

REVIEW 3 major objections 6 minor 35 references

Deep Reinforcement Learning for Chatbots Using Clustered Actions and Human-Likeness Rewards

T0 review · 3 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a chatbot trained by deep reinforcement learning with 100 clustered actions and a +1/-1 human-likeness reward learns reasonable policies on familiar dialogues, while generalization to unseen dialogues remains unsolved.

desk verdict Honest, incremental paper on DQN for chitchat; the central reward-validation is circular and the action-collision issue is unmeasured, but the core training claim holds. read the letter →

arxiv 1908.10331 v1 pith:FNPYGVT4 submitted 2019-08-27 cs.AI cs.CLcs.LGcs.NE

Heriberto Cuayáhuitl , Donghyeon Lee , Seonghan Ryu , Sungja Choi , Inchul Hwang , Jihie Kim This is my paper · ORCID
classification cs.AIcs.CLcs.LGcs.NE
keywords deepreinforcementlearningchatbotsclusteredactionshuman-likenessrewardsdialoguerewardpredictionsentenceembeddingschitchatactionclustering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that a chatbot can be trained end-to-end with deep reinforcement learning on raw chitchat text, with no manual annotations, if the infinite space of possible replies is first reduced to a finite set of clustered actions and rewards are derived automatically from human-human versus randomly sampled sentences. The agent is tasked with choosing the true human reply out of a small candidate set, receiving +1 for the human response and -1 for a random distractor. On dialogues similar to its training clusters, the agent's average reward improves over time; on completely unseen test dialogues, performance drops substantially. The paper also argues that 100- and 300-dimensional sentence embeddings perform equivalently on test data, and that the proposed reward signal can be predicted from long dialogue histories of at least ten sentences. If true, this offers a low-cost route to reward specification for open-ended dialogue agents, while exposing generalization as the remaining obstacle.

What carries the argument

The machinery is the paired reduction of two reinforcement-learning difficulties. Clustered actions replace the infinite space of possible sentences with $k=100$ cluster IDs obtained by K-Means++ on mean word vectors, making the output layer of the Q-network finite. The human-likeness reward replaces hand-crafted rewards with a binary signal: $+1$ for the true human response and $-1$ for a randomly sampled sentence from another dialogue. The state representation is a GRU encoding of up to 50 mean word vectors of the dialogue history. This combination lets a DQN optimize expected cumulative reward without a user simulator or manual labels.

What would settle it

Take a held-out set of dialogue turns, sample random sentences from other dialogues as distractors, and ask human judges whether each distractor is a coherent response in context. If a substantial fraction, for example more than a quarter, are judged coherent, then the binary reward mislabels good replies as bad, and the reward function cannot be said to measure human-likeness. Alternatively, retrain the same agent with rewards replaced by human coherence judgments and compare action selections; divergence would show the proxy, not the property, is what was learned.

Watch

Extended reading notes

Core claim

The central claim is that value-based deep reinforcement learning can be applied to chitchat dialogue by treating sentence clusters as actions and a simple human-versus-random binary reward as the training signal. Concretely, sentences are embedded as mean word vectors, K-Means++ groups them into 100 clusters, and each cluster ID becomes an action; the agent's state is a sequence of sentence embeddings of the dialogue history encoded by GRUs. At each turn the agent must pick the cluster containing the true next human sentence among candidates that include randomly sampled sentences. Rewards are +1 for the true response and -1 for any random distractor, under the assumption that random sentences from other dialogues are incoherent in context. The paper reports three main findings: agents improve on training and familiar test dialogues; test performance on unseen dialogues stays negative; and reward prediction models achieve strong correlation with the human-likeness scores only when the dialogue history has at least ten sentences.

Load-bearing premise

The argument relies on the premise that a sentence sampled at random from another dialogue is generally incoherent in the current context, so that rewarding the agent with -1 for choosing it teaches human-likeness; if random distractors are often fluent or contextually plausible, the reward no longer tracks conversational quality.

Editorial extensions

If this is right

  • Chatbot policies can be learned with a fraction of the action space, since 100 clusters capture greeting, question, and statement types well enough for training.
  • Training on automatically discovered subsets of dialogues can outperform training on the full data, suggesting that domain structure matters more than data volume in this setting.
  • Sentence embeddings of 100 dimensions are sufficient for test performance, cutting parameter count from 12.1 million to 4.4 million without significant loss.
  • Human-likeness rewards are reliable only with long dialogue history; short contexts of fewer than ten sentences cannot support useful reward prediction.
  • The approach generalizes poorly to unseen dialogues, so the learned policies are environment-specific rather than broadly conversational.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the reward is defined entirely by the distractor assumption, a direct human-judgment study on random distractors would be a natural next test; such a test is not part of this paper.
  • The same clustered-action machinery could be transferred to task-oriented dialogue with large slot-value combinations, where finite but large action sets are common.
  • The finding that clustered dialogue splits train better than the full set hints that a curriculum that gradually mixes splits may give the generalization the paper could not obtain.
  • One could make the reward self-improving: use the trained reward predictor on longer histories to filter distractors, keeping only those predicted to be contextually plausible as hard negatives.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes ChatDQN, a DQN-based chatbot trained on raw chitchat text without manual annotations. Actions are obtained by k-means clustering sentence embeddings into 100 clusters, and rewards are +1 for selecting the true next human utterance and -1 for selecting a randomly sampled distractor sentence from another dialogue. Experiments on Persona-Chat across 20 automatically discovered dialogue splits plus the full training set compare sentence embedding sizes of 100 and 300 dimensions. The authors report that agents improve on training dialogues they have seen, that performance drops substantially on held-out dialogues, that the two embedding sizes are not significantly different on the test set, and that training on automatically discovered splits outperforms training on the full data. A separate analysis trains GRU regression models to predict the +1/-1 dialogue rewards and reports strong Pearson correlations for dialogue histories of at least 10 sentences.

Significance. The main conceptual contribution is interesting: replacing the infinite action space of open-domain dialogue with clustered actions and using a simple self-supervised reward could make value-based deep RL applicable to chitchat without manual labels. The paper is also honest about its negative test-set results, which is a useful counterpoint to overly optimistic generative chatbot evaluations. If the reward function were independently validated and the action-space bottleneck quantified, the approach would be a meaningful step. As it stands, the evidence supports only the narrow claim that DQN can optimize the hand-defined reward on familiar dialogues; the stronger claim that the proposed rewards measure human-likeness is not yet established.

major comments (3)
  1. [III-B and V] The reward function and its validation are circular. Section III-B defines the +1/-1 labels by assuming that a randomly sampled sentence from another dialogue is incoherent, and Section V validates the reward by predicting exactly those self-generated labels using the distortion algorithm of [32]. A strong Pearson correlation between true and predicted labels shows only that the labels are learnable from long dialogue contexts; it does not show that the labels correspond to human judgments of coherence or appropriateness. The conclusion in Section VI that the proposed rewards 'can be used for training future neural-based chatbots' therefore overreaches the evidence. I would like to see either a human annotation study on a sample of true versus distractor responses, or an independent coherence metric, together with a report of how well the +1/-1 labels agree with that external signal.
  2. [IV-B and Table II] The action-space collision problem is acknowledged but never measured. With k=100 clusters, the three candidate responses (one true response and two distractors) are mapped to cluster IDs, and if two candidates share a cluster the agent cannot distinguish them even with an optimal Q-function (Algorithm 1, lines 8-10). The paper itself notes that a small number of clusters 'could result in actions being assigned to potentially the same cluster,' but it does not report how often this happens. Table II shows average training rewards around 3.3 against an upper bound of 7.18, and the gap cannot be interpreted without collision statistics. I request the frequency of collisions among the three candidates on both training and test dialogues, and an analysis of the per-turn maximum achievable reward after removing indistinguishable turns, since this directly affects the central claim that agents learn reasonable policies.
  3. [IV-C and Table II] The only quantitative performance metric reported is the same hand-defined reward that the agents are trained to optimize. Because that reward is based on an unvalidated assumption about distractor incoherence, the numerical results do not by themselves measure dialogue quality. In addition, no baselines are provided against which 'reasonable policies' can be calibrated; the comparison is only among differently trained ChatDQN agents and summary random-selection numbers. A comparison with at least one non-RL baseline (for example, always selecting the most frequent cluster, or a supervised classifier of the true response) or an external human evaluation would be needed to support the abstract's first claim that the agents learn reasonable policies.
minor comments (6)
  1. [IV-C] The statistical claim about embedding sizes is under-reported: the text says a two-tailed Wilcoxon signed-rank test at p=0.05 was used, but Table II marks significance only with a dagger and gives no p-values or test statistics. Please report the actual p-values and clarify whether the test is across the 20 split agents or per-split.
  2. [III-B] The second case in the reward equation reads 'if a is human but randomly chosen (incoherent)', which is ambiguous; it should say 'if a is a randomly chosen human sentence from a different dialogue', because the current wording suggests the same action is both human and random.
  3. [Algorithm 1] Line 12 leaves unspecified how the reward is computed when the selected cluster contains both the true response and a distractor sentence; this is exactly the collision case discussed above and should be defined explicitly.
  4. [Table II] The column heading 'Testing on the Training Set' is confusing because it does not clearly indicate evaluation on training dialogues under the learned policy; a rename such as 'Evaluation on Training Dialogues' would improve readability.
  5. [Figure 5] The bars report the mean Pearson correlation over 10 runs without error bars or standard deviations; adding them would help assess the stability of the history-length effect, which is a central piece of evidence for the reward-prediction claim.
  6. [V] The distortion algorithm of [32] is central to the reward-prediction experiment but is described only by reference; please summarize how the distorted dialogues and target scores are generated so that the reader can assess the validity of the label-generation process.

Circularity Check

2 steps flagged · score 6.0 of 10

Reward validation predicts self-generated labels from the authors' own [32], making the 'reasonable rewards' claim partly circular.

  1. self definitional [Section V, 'Analysis of Human-Likeness Rewards', Fig. 5; conclusion in Section VI]
    "Assuming that vector Y ={y1,...,y|Y|} is the set of target labels—generated as described in the dialogue generation algorithm of [32], and using the same test data as the previous section."

    The validation claims the rewards are reasonable because predicted rewards correlate with 'true' dialogue rewards on test data. But the 'true' labels are not independent: they are generated by the authors' own dialogue-distortion algorithm [32], which encodes the same human-versus-random intuition as the reward definition in Section III-B ('-1, if a is human but randomly chosen (incoherent)'). Section VI states the paper's assumption that such random sentences 'are generally incoherent throughout the dialogue history'. A neural regressor that learns to predict these self-generated labels only shows that the generator is learnable; it does not show that the rewards reflect human-likeness against any external judgment.

  2. self citation load bearing [Section V, first sentence; reference [32]]
    "We employ the algorithm of [32] for extending a dataset of human-human dialogues with distorted dialogues. The latter include varying amounts of distortions, i.e. different degrees of human-likeness."

    Reference [32] is the authors' own prior workshop paper (Cuayáhuitl, Ryu, Lee, and Kim, 2018). The only evidence offered for the load-bearing claim that 'human-likeness rewards are reasonable' is the correlation of predicted rewards with labels generated by this cited algorithm. Because the cited algorithm is the source of the ground-truth labels and is not independently verified, machine-checked, or compared with external human judgments, the argument's central support is a self-citation chain rather than evidence outside the present paper's own generated values. This is load-bearing self-citation, not merely incidental citation.

full rationale

Score 6 reflects partial circularity. The DQN training and clustering pipeline themselves are not circular: the agent optimizes the stated +1/-1 reward with standard DQN, and the 100-vs-300 embedding comparison is a straightforward empirical benchmark. The circularity is concentrated in the validation of the reward function. Section V builds the target labels with the authors' own distortion algorithm [32]—the same random-distractor assumption that motivates the reward in Section III-B—and then reports that neural regressors can predict those labels. High Pearson correlation with self-generated labels demonstrates internal consistency, not that the rewards correspond to human-likeness; the conclusion in Section VI that the rewards 'can be used for training future neural-based chatbots' is therefore supported by a self-citation chain and by predicting the paper's own labels. The acknowledged cluster-collision issue in Section IV-B is a separate correctness and robustness limitation, not itself a circularity, so it does not further raise the score.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

No fundamentally new theoretical entities are introduced. The main load-bearing choices are the action clustering (k=100) and the binary human-likeness reward, both hand-defined and not independently validated. All other hyperparameters are standard training choices.

free parameters (5)
  • Number of action clusters k = 100
    Hand-chosen; Section IV-B says 'we thus use a sentence clustering model with k=100 derived from our training data and prior to reinforcement learning'; manual inspection motivated the choice, with no tuning procedure described.
  • Number of dialogue clusters for data splits = 20
    Used to split training dialogues into groups; chosen without a stated criterion in Section IV-B, and the exact features for dialogue clustering are not specified.
  • Candidate response set size = 3
    Listed as a hyperparameter in Section IV-B; affects action selection difficulty and was not varied.
  • Discount factor = 0.99
    Set hyperparameter in Section IV-B; standard but hand-chosen.
  • Maximum dialogue history length = 50 sentence vectors
    Zero-padded maximum in the network input; listed in the hyperparameters in Section IV-B.
assumptions (4)
  • domain assumption Randomly sampled sentences from different dialogues are non-human-like and incoherent in the current context.
    Section III-B assigns r=-1 to 'human but randomly chosen (incoherent)' responses; Section VI restates 'we assume that the latter are generally incoherent throughout the dialogue history.' This premise is load-bearing for the reward signal.
  • domain assumption Mean word vectors of sentences (Deep Averaging Networks) capture enough semantic similarity for clustering to form meaningful action types.
    Section III-A represents each sentence as the mean of GloVe word vectors; the clustering and state representation rely on this.
  • domain assumption K-means++ clusters correspond to dialogue action types such as greetings and questions.
    Section III-A and Section IV-B use k=100 clusters as the action set, so cluster IDs must be semantically coherent actions for the policy to be meaningful.
  • domain assumption Replacing one human in a human-human dialogue with the learned agent creates a valid training environment without a user simulator.
    Section IV-B states no simulator is used; the agent learns to select the human sentence among candidates within real dialogues, which assumes this setup provides useful learning signal.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Deep Reinforcement Learning for Chatbots Using Clustered Actions and Human-Likeness Rewards." pith.science (2026). https://pith.science/paper/FNPYGVT4

@misc{pith2026190810331,
  author       = {Pith},
  title        = {Pith review of: Deep Reinforcement Learning for Chatbots Using Clustered Actions and Human-Likeness Rewards},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FNPYGVT4}},
  note         = {Machine review of arXiv:1908.10331}
}
read the original abstract

Training chatbots using the reinforcement learning paradigm is challenging due to high-dimensional states, infinite action spaces and the difficulty in specifying the reward function. We address such problems using clustered actions instead of infinite actions, and a simple but promising reward function based on human-likeness scores derived from human-human dialogue data. We train Deep Reinforcement Learning (DRL) agents using chitchat data in raw text---without any manual annotations. Experimental results using different splits of training data report the following. First, that our agents learn reasonable policies in the environments they get familiarised with, but their performance drops substantially when they are exposed to a test set of unseen dialogues. Second, that the choice of sentence embedding size between 100 and 300 dimensions is not significantly different on test data. Third, that our proposed human-likeness rewards are reasonable for training chatbots as long as they use lengthy dialogue histories of >=10 sentences.

Figures

Figures reproduced from arXiv: 1908.10331 by the authors.

Figure 1
Figure 1. High-level architecture of the proposed deep reinforcement learning [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Example clusters of our training data using Principal Component Analysis [31] for visualisations in 2D – black dots represent sentences or dialogues [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Training performance of ChatDQN agents using different data splits of dialogues—see text for details [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Training performance of our ChatDQN agents using all training [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Bar plot showing the performance of our dialogue reward predictors [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 32 canonical work pages

  1. [32]

    A study on dialogue reward prediction for open-ended conversational agents,

    Heriberto Cuay ´ahuitl, Seonghan Ryu, Donghyeon Lee, and Jihie Kim, “A study on dialogue reward prediction for open-ended conversational agents,” in NeurIPS Workshop on Conversational AI: “Today’s Practice and Tomorrow‘s Potential”, 2018

    work page 2018

  2. [1]

    Attention, intentions, and the structure of discourse,

    Barbara J. Grosz and Candace L. Sidner, “Attention, intentions, and the structure of discourse,” Computational Linguistics , vol. 12, no. 3, pp. 175–204, 1986

    work page 1986

  3. [2]

    Sutton and Andrew G

    Richard S. Sutton and Andrew G. Barto, Reinforcement learning - an introduction, Adaptive computation and machine learning. MIT Press, 2nd edition edition, 2018

    work page 2018

  4. [3]

    Friedman, The elements of statistical learning: data mining, inference, and prediction, 2nd Edition, Springer series in statistics

    Trevor Hastie, Robert Tibshirani, and Jerome H. Friedman, The elements of statistical learning: data mining, inference, and prediction, 2nd Edition, Springer series in statistics. Springer, 2009

    work page 2009

  5. [4]

    Deep learning,

    Yann LeCun, Yoshua Bengio, and Geoffrey E. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436–444, 2015

    work page 2015

  6. [5]

    Feu- dal reinforcement learning for dialogue management in large domains,

    I ˜nigo Casanueva, Pawel Budzianowski, Pei-Hao Su, Stefan Ultes, Lina Maria Rojas-Barahona, Bo-Hsiang Tseng, and Milica Gasic, “Feu- dal reinforcement learning for dialogue management in large domains,” in NAACL-HLT, 2018

    work page 2018

  7. [6]

    SimpleDS: A Simple Deep Reinforcement Learning Dialogue System

    Heriberto Cuay ´ahuitl, “SimpleDS: A simple deep reinforcement learning dialogue system,” CoRR, vol. abs/1601.04574, 2016

    work page Pith review arXiv 2016

  8. [7]

    Scaling up deep reinforcement learning for multi-domain dialogue systems,

    Heriberto Cuay ´ahuitl, Seunghak Yu, Ashley Williamson, and Jacob Carse, “Scaling up deep reinforcement learning for multi-domain dialogue systems,” in IJCNN, 2017

    work page 2017

Show all 35 references
  1. [8]

    Deep reinforcement learning of dialogue policies with less weight updates,

    Heriberto Cuay ´ahuitl and Seunghak Yu, “Deep reinforcement learning of dialogue policies with less weight updates,” in INTERSPEECH, 2017

    2017

  2. [9]

    Hybrid code networks: practical and efficient end-to-end dialog control with supervised and reinforcement learning,

    Jason D. Williams, Kavosh Asadi, and Geoffrey Zweig, “Hybrid code networks: practical and efficient end-to-end dialog control with supervised and reinforcement learning,” in ACL, 2017

    2017

  3. [10]

    Composite task-completion dia- logue policy learning via hierarchical deep reinforcement learning,

    Baolin Peng, Xiujun Li, Lihong Li, Jianfeng Gao, Asli C ¸ elikyilmaz, Sungjin Lee, and Kam-Fai Wong, “Composite task-completion dia- logue policy learning via hierarchical deep reinforcement learning,” in EMNLP, 2017

    2017

  4. [11]

    Deep reinforcement learning for dialogue generation,

    Jiwei Li, Will Monroe, Alan Ritter, Dan Jurafsky, Michel Galley, and Jianfeng Gao, “Deep reinforcement learning for dialogue generation,” in EMNLP, 2016

    2016

  5. [12]

    Adversarial learning for neural dialogue generation,

    Jiwei Li, Will Monroe, Tianlin Shi, S ´ebastien Jean, Alan Ritter, and Dan Jurafsky, “Adversarial learning for neural dialogue generation,” in EMNLP, 2017

    2017

  6. [13]

    A deep rein- forcement learning chatbot (short version),

    Iulian Vlad Serban, Chinnadhurai Sankar, Mathieu Germain, Saizheng Zhang, Zhouhan Lin, Sandeep Subramanian, Taesup Kim, Michael Pieper, Sarath Chandar, Nan Rosemary Ke, Sai Rajeswar, Alexandre de Br ´ebisson, Jose M. R. Sotelo, Dendi Suhubdy, Vincent Michalski, Alexandre Nguye...

    2018 arXiv

  7. [14]

    Modeling non-goal oriented dialog with discrete attributes,

    Chinnadhurai Sankar and Sujith Ravi, “Modeling non-goal oriented dialog with discrete attributes,” in NeurIPS Workshop on Conversational AI: “Today’s Practice and Tomorrow‘s Potential”, 2018

    2018

  8. [15]

    Scalable sentiment for sequence- to-sequence chatbot response with performance analysis,

    Chih-Wei Lee, Yau-Shian Wang, Tsung-Yuan Hsu, Kuan-Yu Chen, Hung-yi Lee, and Lin-Shan Lee, “Scalable sentiment for sequence- to-sequence chatbot response with performance analysis,” CoRR, vol. abs/1804.02504, 2018

    2018 arXiv

  9. [16]

    A neural conversational model,

    Oriol Vinyals and Quoc V . Le, “A neural conversational model,” CoRR, vol. abs/1506.05869, 2015

    2015 arXiv

  10. [17]

    A neural network approach to context-sensitive generation of conversational responses,

    Alessandro Sordoni, Michel Galley, Michael Auli, Chris Brockett, Yangfeng Ji, Margaret Mitchell, Jian-Yun Nie, Jianfeng Gao, and Bill Dolan, “A neural network approach to context-sensitive generation of conversational responses,” in HLT-NAACL, 2015

    2015

  11. [18]

    Multiresolution recurrent neural networks: An application to dialogue response genera- tion,

    Iulian Vlad Serban, Tim Klinger, Gerald Tesauro, Kartik Talamadupula, Bowen Zhou, Yoshua Bengio, and Aaron C. Courville, “Multiresolution recurrent neural networks: An application to dialogue response genera- tion,” in AAAI, 2017

    2017

  12. [19]

    A persona-based neural conversation model,

    Jiwei Li, Michel Galley, Chris Brockett, Georgios P. Spithourakis, Jian- feng Gao, and William B. Dolan, “A persona-based neural conversation model,” in ACL, 2016

    2016

  13. [20]

    Chat more: Deepening and widening the chatting topic via a deep model,

    Wenjie Wang, Minlie Huang, Xin-Shun Xu, Fumin Shen, and Liqiang Nie, “Chat more: Deepening and widening the chatting topic via a deep model,” in SIGIR. 2018, ACM

    2018

  14. [21]

    Personalizing dialogue agents: I have a dog, do you have pets too?,

    Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston, “Personalizing dialogue agents: I have a dog, do you have pets too?,” CoRR, vol. abs/1801.07243, 2018

    2018 arXiv

  15. [22]

    ”chitty-chitty-chat bot

    Rui Yan, “”chitty-chitty-chat bot”: Deep learning for conversational AI,” in IJCAI, 2018

    2018

  16. [23]

    How NOT to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation,

    Chia-Wei Liu, Ryan Lowe, Iulian Serban, Michael Noseworthy, Laurent Charlin, and Joelle Pineau, “How NOT to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation,” in EMNLP, 2016

    2016

  17. [24]

    Distributed representations of words and phrases and their compositionality,

    Tomas Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean, “Distributed representations of words and phrases and their compositionality,” in NIPS, 2013

    2013

  18. [25]

    Glove: Global vectors for word representation,

    Jeffrey Pennington, Richard Socher, and Christopher D. Manning, “Glove: Global vectors for word representation,” in EMNLP, 2014

    2014

  19. [26]

    Deep unordered composition rivals syntactic methods for text classification,

    Mohit Iyyer, Varun Manjunatha, Jordan L. Boyd-Graber, and Hal Daum ´e III, “Deep unordered composition rivals syntactic methods for text classification,” in ACL (1), 2015

    2015

  20. [27]

    K-means++: The advantages of careful seeding,

    David Arthur and Sergei Vassilvitskii, “K-means++: The advantages of careful seeding,” in SODA. 2007, SIAM

    2007

  21. [28]

    Human-level control through deep reinforcement learning,

    V olodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin A. Riedmiller, Andreas Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra,...

    2015

  22. [29]

    Parlai: A dialog research software platform,

    Alexander H. Miller, Will Feng, Dhruv Batra, Antoine Bordes, Adam Fisch, Jiasen Lu, Devi Parikh, and Jason Weston, “Parlai: A dialog research software platform,” in EMNLP (System Demonstrations), 2017

    2017

  23. [30]

    Learning phrase representations using RNN encoder–decoder for statistical ma- chine translation,

    Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bah- danau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio, “Learning phrase representations using RNN encoder–decoder for statistical ma- chine translation,” in EMNLP. 2014, Association for Computational Linguistics

    2014

  24. [31]

    Probabilistic principal compo- nent analysis,

    M. E. Tipping and Christopher Bishop, “Probabilistic principal compo- nent analysis,” Journal of the Royal Statistical Society, Series B , vol. 21/3, pp. 611622, January 1999

    1999

  25. [33]

    Batch normalization: Accelerating deep network training by reducing internal covariate shift,

    Sergey Ioffe and Christian Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in Interna- tional Conference on Machine Learning (ICML) , 2015

    2015

  26. [34]

    The Stanford CoreNLP natural language processing toolkit,

    Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven J. Bethard, and David McClosky, “The Stanford CoreNLP natural language processing toolkit,” in Association for Computational Linguistics (ACL) System Demonstrations , 2014

    2014

  27. [35]

    Domain transfer for deep natural language generation from abstract meaning representations,

    Nina Dethlefs, “Domain transfer for deep natural language generation from abstract meaning representations,” IEEE Comp. Int. Mag. , vol. 12, no. 3, pp. 18–28, 2017

    2017

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.