Reward-Driven Interaction: Enhancing Proactive Dialogue Agents through User Satisfaction Prediction

Chuheng Zhang; Wanchun Dou; Wei Shen; Xiaolong Xu; Xiaonan He; Xuyun Zhang

REVIEW 4 major objections 6 minor 42 references

Reward-Driven Interaction: Enhancing Proactive Dialogue Agents through User Satisfaction Prediction

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read By adding two auxiliary training tasks to a transformer-based satisfaction model, this paper shows that an industrial voice assistant can catch more rare ASR errors and long-tail user dissatisfaction, raising online contextual…

desk verdict Real industrial system paper with a sensible motivation, but the evidence for the central claim doesn't survive contact with the evaluation protocol: test-set threshold tuning and a near-circular auxiliary task. read the letter →

arxiv 2505.18731 v1 pith:H5ORLE5U submitted 2025-05-24 cs.LG

Wei Shen , Xiaonan He , Chuheng Zhang , Xuyun Zhang , Xiaolong Xu , Wanchun Dou This is my paper · ORCID

classification cs.LG

keywords usersatisfactionpredictionproactivedialoguecontrastiveself-supervisedlearningdomain-intentclassificationlong-taildistributionASRerrordetectionmulti-taskDuerOS

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the accuracy of user-satisfaction prediction in industrial dialogue systems is limited less by model capacity than by poor representations of rare and low-frequency inputs, a consequence of noisy weak labels and the power-law distribution of user utterances. It proposes adding two auxiliary training tasks to a transformer-based satisfaction model: a contrastive self-supervised objective over ASR query embeddings, and a domain-intent classification over session representations. On an industrial dataset, the combined model improves conditional label accuracy by 5.5% over the previously deployed model, with the largest gains on long-tailed domains, and in an online A/B test it raises contextual user satisfaction from 0.684 to 0.701. If the result holds, dialogue systems can catch more ASR-induced errors and long-tail frustrations without new features or architectural changes.

What carries the argument

The load-bearing mechanism is the joint training objective that combines the main satisfaction loss with a contrastive self-supervised task and a domain-intent classification task. The contrastive task, adapted from SimCSE, feeds the same ASR query (original, n-best, and rewritten) through the encoder twice with independent dropout, treats the two views as a positive pair and all other in-batch queries as negatives, and uses a cross-entropy loss; this is intended to create tight clusters for rare utterances so ASR errors become detectable. The domain-intent task takes the SEP token's representation of the user session and trains a fully connected head to predict the known domain-intent label with cross-entropy, which sharpens session representations for long-tailed domains. The final loss is a weighted sum of the three terms, with the contrastive weight set to $10^{-2}$ and the classification weight to $10^{-1}$.

What would settle it

Recompute the offline comparison with the decision threshold fixed on a separate validation set before scoring the expert-labeled test set, and repeat on a larger expert-labeled sample; if ABM's advantage over TBM-2 on long-tail domains largely disappears, the claimed representation benefit would be undercut. Additionally, inspect whether the Universal QA gain is driven by fewer than a handful of test samples.

Watch

Extended reading notes

Core claim

The paper's central claim is that two auxiliary tasks—contrastive self-supervised learning on rare query embeddings and domain-intent classification on session representations—fix a representation gap that makes satisfaction models miss dissatisfied users on rare utterances and long-tailed domains. The authors build on a three-sub-module transformer that matches ASR n-best queries, query-reply pairs, and multi-turn sessions, and train it with an extra loss that pulls dropout views of the same query together while pushing in-batch negatives apart, plus a cross-entropy head that predicts the domain-intent of the session. In offline tests, the auxiliary-task model (ABM) beats the previous best baseline (TBM-2) by 5.5% in conditional label accuracy, with a 28.8% relative gain on the Universal QA long-tail domain and 45% on other domains; online, CUS rises from 0.684 to 0.701 and ABM recalls 38 of 119 ASR errors versus 30 for TBM-2. The authors take this as evidence that the auxiliary tasks improve representation learning where weak labels and data skew previously hid true dissatisfaction signals.

Load-bearing premise

The reported offline improvement depends on choosing the decision threshold by grid search on the same expert-labeled test set that is used to compute the reported CLA, so some of the gain may be threshold overfitting rather than better representations.

Editorial extensions

If this is right

Because the auxiliary losses only change training, the approach can be applied to existing deployed satisfaction models without adding online latency or new input features.
The largest relative gains appear on long-tailed domains (Universal QA, UDF, other), so similar multi-task objectives should help other low-frequency user behaviors that currently drive the majority of missed errors.
Online, the model recalls 38 of 119 ASR errors and 10 of 61 NLU errors versus 30 and 5 for the baseline, implying that more clarifications are triggered in exactly the failure cases users notice.
The method yields a small but consistent CUS increase from 0.684 to 0.701 on expert-annotated samples, which the authors interpret as a genuine reduction in user dissatisfaction at scale.
The same architecture can be retrained daily (as done in the online experiment) without pre-training costs, making it feasible for production deployment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

We infer that the offline CLA numbers, which rely on a threshold grid-searched on the expert-labeled test set, may overstate real-world gains; a separate validation split for threshold selection would test whether the long-tail improvements persist.
The contrastive task's mechanism suggests a broader principle: in any dialogue reward model, rare inputs that cause errors should be treated as a representation problem, not just a labeling problem, and self-supervised objectives are a cheap way to address it.
A direct extension would be to apply the same two auxiliary tasks to other turn-level prediction targets in dialogue, such as user confusion, intent shift, or churn, where power-law distributions also dominate.
Given that overall error-recognition accuracy was similar (86.5% vs 85%) while recall improved on specific error types, the paper implies that aggregate accuracy is the wrong metric for evaluating proactive dialogue: error-type-specific recall should guide deployment decisions.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Here's my take. The paper is a straightforward industrial application: they take a deployed satisfaction predictor (TBM-2) and add two auxiliary losses—SimCSE-style contrastive learning on query embeddings and a domain-intent classification head—to improve recall of dissatisfied sessions, especially after ASR errors and in long-tail domains. The problem is real, and the case analysis that motivates it is the best part of the paper. Showing that a large share of missed dissatisfaction cases involve ASR errors and that long-tail domains dominate the error budget is genuinely useful, and it comes from a deployed system with 32M training samples.

The method itself is not new: the contrastive loss is the standard InfoNCE/SimCSE objective, and the domain-intent head is a straightforward multi-task classifier. Neither is a contribution in itself; the claimed contribution is that they help this particular model. That claim is not supported by the evidence as presented.

First, the offline CLA metric is selected. Section 5.1.2 says the threshold is grid-searched on the expert-labeled test set to maximize recall at precision above 85%. CLA is therefore an in-sample statistic. The 28.8% relative gain on Universal QA (0.045 to 0.058) is likely a handful of examples; without confidence intervals it is not evidence of generalization. AUC, by contrast, barely moves (0.790 vs 0.792).

Second, the online CUS gain is 0.684 to 0.701 on 1,000 annotated queries per arm. If CUS is a binary satisfaction rate, that difference is roughly 0.8 standard errors. No significance test or CI is reported. The further analysis uses counts like 38 vs 30 and 15 vs 7, again without uncertainty quantification.

Third, the domain-intent auxiliary task is near-circular. The SEP vector comes from the session module, and that module's input includes the domain-intent embedding e_i^n. The classifier can read the answer off its own input. That doesn't necessarily make the loss useless, but it does mean the paper's interpretation—that the task forces better session representations—is not demonstrated.

Net: the motivation and deployment are real, and the paper could be useful as a case study. But the central quantitative claim is not established. If I were editing, I'd send it out—there is enough substance in the industrial problem and the failure-mode analysis—but I'd expect heavy revision: fixed threshold on validation, confidence intervals on CUS, and a control for the input-label leakage in the domain-intent task. As is, I wouldn't cite it for the result.

Referee Report

4 major / 6 minor

Summary. The paper addresses user satisfaction prediction for a proactive clarification mechanism in DuerOS. The authors argue that weak labels and power-law query distributions make it hard to recognize rare ASR-error-induced utterances and long-tail domain errors. They add two auxiliary losses to a transformer-based satisfaction model: a SimCSE-style contrastive loss on query embeddings and a domain-intent classification loss on a session embedding. They report offline AUC/CLA results on an expert-labeled 1k test set and an online A/B test (1000 annotated queries per arm) with CUS 0.684 vs 0.701, and conclude that the auxiliary-task model significantly improves error recognition on rare utterances and long-tail domains.

Significance. If the empirical claims held, the work would be practically valuable: it studies a real deployment in a large industrial dialogue system, uses a 32M-sample training set, and provides both offline and online evidence, including expert annotations. The contrastive auxiliary task is a sensible, low-cost way to address representation learning under noisy weak labels. However, the evaluation as presented does not support the central claim: the offline CLA metric is optimized on the test set, the online gain is within sampling error, and the domain-intent auxiliary task is nearly circular. The paper's useful engineering setting cannot compensate for the missing statistical rigor in the current version.

major comments (4)

[5.1.2, Table 2] The reported CLA improvements are in-sample statistics. Section 5.1.2 states that the threshold is tuned by grid search to maximize recall on the expert-labeled samples, and the 1,000-sample expert-labeled set is the test set (Section 5.1.1). Because CLA is defined as the maximum recall subject to precision ≥ 85%, the threshold selection and the metric are evaluated on the same data. The headline results—5.5% overall CLA gain, 28.8% relative gain on Universal QA (0.045→0.058)—are therefore selected statistics, not estimates of generalization, and the per-domain numbers likely correspond to only one or two test examples. The threshold should be fixed on the validation set (or a separate labeled set), and CLA should be reported with a confidence interval at that fixed threshold.
[6, Figure 4] The online A/B result does not demonstrate a significant improvement. CUS is 0.684 for TBM-2 and 0.701 for ABM on 1,000 expert-annotated queries per arm. For a binary satisfaction rate the standard error of the difference is approximately sqrt(0.684*0.316/1000 + 0.701*0.299/1000) ≈ 0.021, so the observed 0.017 difference is within one standard error. No confidence intervals, p-values, or effect sizes are reported. The further analysis in Figure 4 relies on small counts (38 vs 30 ASR errors, 10 vs 5 NLU errors, 15 vs 7 Universal QA cases) and uses the word 'significantly' without any test; for the ASR-error comparison, for example, the difference is not significant at conventional levels. These results need proper uncertainty quantification before any significance claim can be made.
[3.2, 4.3, Figure 3] The domain-intent classification auxiliary task is nearly circular and cannot provide the intended representation-learning signal. In Section 3.2, the session representation for turn i is composed as e_i_s = concat(e_i_q, e_i_n, e_i_t), where e_i_n is the embedding of the NLU domain/intent label. In Section 4.3, the auxiliary classifier takes the SEP vector of the current turn and is trained to predict the domain-intent label d. The model can therefore solve this auxiliary task almost trivially by reading the label from its own input. Any benefit attributed to this task on long-tailed domains is not explained by the stated mechanism, and the authors should either remove the task from the attribution analysis or redesign it so the target is not an input feature.
[Table 2, Section 5.2] The threshold-independent AUC does not improve: ABM has overall AUC 0.790 versus TBM-2's 0.792, and the same small decrease appears in the media domain (0.793 vs 0.795). Since AUC is not affected by the threshold-selection issue, this undermines the claim that the auxiliary tasks improve the model's overall satisfaction prediction. The paper should report precision-recall curves or a fixed-threshold evaluation, and should explain why a method that improves the selected CLA can slightly reduce AUC; without that, the evidence for the central claim rests entirely on a test-set-optimized statistic.

minor comments (6)

[5.1.3] The model is introduced as 'ATM' but called 'ABM' in Table 2 and elsewhere; use one abbreviation consistently.
[Table 2] The column header 'Univesal QA' should be 'Universal QA'.
[4.1, Figure 2] The text reports 230 unrecognized dissatisfied cases while the caption says '48% of total 220 cases'; reconcile these numbers.
[6] 'DuesOs' should be 'DuerOS'; also fix the typos 'Tabel', 'choosen', 'erros', and 'trys' throughout the manuscript.
[5.1.2] Please define CLA formally, including the exact set of thresholds over which the maximum is taken and whether the 85% precision constraint is required to hold on the test set or on a held-out set.
[Figure 4] The total number of dissatisfied sessions per domain is not given, so the '15 vs 7' Universal QA comparison lacks a denominator and cannot be interpreted without additional context.

Circularity Check

2 steps flagged · score 6.0 of 10

Domain-intent auxiliary task predicts labels already embedded in its input, and offline CLA is optimized on the test set; the central offline evidence is partly circular.

self definitional [Section 3.2 and Section 4.3 (domain-intent classification auxiliary task)]
"each domain-intent and slot are mapped into a unique NLU embedding e_i_n ... e_i_s = Concat(e_i_q, e_i_n, e_i_t) ... we first extract the representation of the user session, pass the vector of the symbol SEP in the current turn to a fully connected layer ... the vector s is transformed as a prediction p. Then we adopt the binary cross entropy loss function to get the final loss on p and labeled domain-intent d: L_cl = CE(p,d)."

The domain-intent classifier predicts the labeled domain-intent d from a representation s derived from the current turn's SEP vector, and that SEP vector is by construction the concatenation that includes e_i_n, the NLU embedding of the domain-intent label itself. The classifier can therefore minimize L_cl = CE(p,d) by a linear readout of an input component that already encodes d. The task is solvable without learning any new long-tail session pattern, so the paper's claim that this auxiliary task improves representation learning on long-tailed domains is not supported: the task's own objective is self-definitional, not an independent learning signal.
fitted input called prediction [Section 5.1.2 (metrics) and Table 2 (reported CLA)]
"CLA is the maximum recall when precision is larger than a specified value (which is set to 85% in this paper). In our experiments, we binary our prediction according to a threshold and then calculate the classification precision given the true labels, where we tune the threshold by grid searching to maximize the recall rate of dissatisfied user sessions."

The expert-labeled 1k test samples are used both to grid-search the decision threshold and to compute the reported CLA, and CLA is itself defined as the maximum recall at a precision bound. Thus the reported offline numbers are in-sample selected maxima: the threshold is fit directly to the test labels that define the metric. The headline gains, including the 28.8% relative Universal QA increase from 0.045 to 0.058, are differences between two in-sample optimized statistics rather than unbiased predictions of deployment performance. This makes the offline evaluation statistic statistically forced by construction.

full rationale

Two circular steps are present. First, the domain-intent auxiliary classifier takes the current turn's SEP vector as input, and that vector by construction contains the NLU embedding e_i_n of the very domain-intent label d it is asked to predict (Section 3.2 vs Section 4.3). The loss L_cl = CE(p,d) can be minimized by copying the e_i_n component through the network, so this auxiliary task is self-definitional and does not demonstrate that long-tail session representations are actually learned. Second, the offline CLA metric is computed by grid-searching the decision threshold on the same 1k expert-labeled test samples whose labels define CLA; the reported gains are in-sample selected maxima, i.e., a fit renamed as a prediction. The online CUS gain (0.684 vs 0.701) and the small-count further analysis are better viewed as statistical concerns than as circularity, but they do not independently rescue the central claim. The self-citations to [29] and [28] are not load-bearing: they provide the baseline model and metric, not the proof of the auxiliary-task benefit. Overall, one of the two proposed auxiliary mechanisms is circular by construction and the primary offline evidence is fitted on the test set, so the headline improvement is only partially established; the contrastive SSL task and the online deployment do provide independent content. Score 6 reflects partial circularity rather than a fully forced result.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on several domain assumptions about weak labels, the CUS metric, and the usefulness of the two auxiliary tasks. The most fragile is the domain-intent classification task, whose target label is also an input feature, making the auxiliary task near-circular.

free parameters (4)

w1 = 1e-2
Weight of the contrastive self-supervised loss in the final loss, chosen by grid search over {1e-3, 1e-2, 1e-1} (Section 5.1.2).
w2 = 1e-1
Weight of the domain-intent classification loss, chosen by grid search (Section 5.1.2).
decision_threshold = 0.78
Threshold on predicted satisfaction probability used to trigger clarification; tuned by grid searching on the expert-labeled test set to maximize recall at precision 85% (Section 5.1.2), biasing CLA.
CLA_precision_floor = 0.85
Hand-chosen precision requirement in the conditional label accuracy (CLA) metric; the threshold is then fit on the test set (Section 5.1.2).

assumptions (4)

domain assumption Weak labels generated by a simple model trained on post-hoc user actions are a usable training signal for user satisfaction.
The paper relies on these labels for 32M training samples and 250K validation samples (Section 5.1.1); if the weak labels are too noisy, the model and auxiliary tasks are trained on unreliable targets.
domain assumption The contextual user satisfaction (CUS) metric from expert annotation of 1000 sampled queries is a valid measure of online user satisfaction.
Section 6 defines CUS and uses it to compare TBM-2 and ABM; the metric's reliability is taken for granted.
domain assumption SimCSE-style contrastive loss improves representation of rare ASR-error utterances.
Section 4.2.1 assumes the dropout-based positive pairs encode useful utterance structure, but no analysis shows the learned embeddings actually cluster rare utterances.
ad hoc to paper The domain-intent classification auxiliary loss is non-trivial and beneficial despite the domain-intent being an input feature.
Section 4.3 and Figure 3 use the SEP vector of the session that contains the domain-intent embedding e_i_n, so the auxiliary task can be solved by copying an input; the paper does not address this leakage.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Reward-Driven Interaction: Enhancing Proactive Dialogue Agents through User Satisfaction Prediction." pith.science (2026). https://pith.science/paper/H5ORLE5U

@misc{pith2026250518731,
  author       = {Pith},
  title        = {Pith review of: Reward-Driven Interaction: Enhancing Proactive Dialogue Agents through User Satisfaction Prediction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/H5ORLE5U}},
  note         = {Machine review of arXiv:2505.18731}
}

read the original abstract

Reward-driven proactive dialogue agents require precise estimation of user satisfaction as an intrinsic reward signal to determine optimal interaction strategies. Specifically, this framework triggers clarification questions when detecting potential user dissatisfaction during interactions in the industrial dialogue system. Traditional works typically rely on training a neural network model based on weak labels which are generated by a simple model trained on user actions after current turn. However, existing methods suffer from two critical limitations in real-world scenarios: (1) Noisy Reward Supervision, dependence on weak labels derived from post-hoc user actions introduces bias, particularly failing to capture satisfaction signals in ASR-error-induced utterances; (2) Long-Tail Feedback Sparsity, the power-law distribution of user queries causes reward prediction accuracy to drop in low-frequency domains. The noise in the weak labels and a power-law distribution of user utterances results in that the model is hard to learn good representation of user utterances and sessions. To address these limitations, we propose two auxiliary tasks to improve the representation learning of user utterances and sessions that enhance user satisfaction prediction. The first one is a contrastive self-supervised learning task, which helps the model learn the representation of rare user utterances and identify ASR errors. The second one is a domain-intent classification task, which aids the model in learning the representation of user sessions from long-tailed domains and improving the model's performance on such domains. The proposed method is evaluated on DuerOS, demonstrating significant improvements in the accuracy of error recognition on rare user utterances and long-tailed domains.

Figures

Figures reproduced from arXiv: 2505.18731 by the authors.

**Figure 2.** The analysis of the online error cases. Among the [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Two auxiliary tasks to enhance the representation learning of the user utterances and user sessions. The left part is a [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: The online analysis of the performance of TBM-2 and ABM. We further compare the performance of these two [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 33 canonical work pages

[1]

Aman Alok, Rahul Gupta, and Shankar Ananthakrishnan. 2020. Design Consid- erations for Hypothesis Rejection Modules in Spoken Language Understanding Systems. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 8049–8053

work page 2020
[2]

Junwen Bai, Bo Li, Yu Zhang, Ankur Bapna, Nikhil Siddhartha, Khe Chai Sim, and Tara N Sainath. 2022. Joint unsupervised and supervised training for multilingual asr. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 6402–6406

work page 2022
[3]

Praveen Kumar Bodigutla, Aditya Tiwari, Josep Vallas Vargas, Lazaros Poly- menakos, and Spyros Matsoukas. 2020. Joint Turn and Dialogue level User Satisfaction Estimation on Multi-Domain Conversations. arXiv preprint arXiv:2010.02495 (2020)

work page Pith review arXiv 2020
[4]

Christian Buck, Jannis Bulian, Massimiliano Ciaramita, Wojciech Gajewski, An- drea Gesmundo, Neil Houlsby, and Wei Wang. 2018. Ask the Right Questions: Active Question Reformulation with Reinforcement Learning. In Proceedings of International Conference on Learning Representations . https://openreview.net/ forum?id=S1CChZ-CZ

work page 2018
[5]

Yuanying Cai, Chuheng Zhang, Wei Shen, Xiaonan He, Xuyun Zhang, and Longbo Huang. 2022. Imitation learning to outperform demonstrators by directly extrap- olating demonstrations. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management . 128–137

work page 2022
[6]

Yuanying Cai, Chuheng Zhang, Wei Shen, Xuyun Zhang, Wenjie Ruan, and Longbo Huang. 2023. RePreM: representation pre-training with masked model for reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 6879–6887

work page 2023
[7]

Yuanying Cai, Chuheng Zhang, Li Zhao, Wei Shen, Xuyun Zhang, Lei Song, Jiang Bian, Tao Qin, and Tieyan Liu. 2022. TD3 with reverse KL regularizer for offline reinforcement learning from mixed datasets. In 2022 IEEE International Conference on Data Mining (ICDM) . IEEE, 21–30

work page 2022
[8]

Giuseppe Castellucci, Valentina Bellomaria, Andrea Favalli, and Raniero Romag- noli. 2019. Multi-lingual intent detection and slot filling in a joint bert-based model. arXiv preprint arXiv:1907.02884 (2019)

arXiv 2019

Show all 42 references

[9]

Wenhu Chen, Jianshu Chen, Pengda Qin, Xifeng Yan, and William Yang Wang
[10]

Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. Simcse: Simple contrastive learning of sentence embeddings. arXiv preprint arXiv:2104.08821 (2021)

2021 arXiv
[11]

Tianyu Guo, Hong Liu, Zhan Chen, Mengyuan Liu, Tao Wang, and Runwei Ding
[12]

Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory.Neural computation 9, 8 (1997), 1735–1780

1997
[13]

Che-Wei Huang, Roland Maas, Sri Harish Mallidi, and Björn Hoffmeister. 2019. A Study for Improving Device-Directed Speech Detection Toward Frictionless Human-Machine Interaction. In Proceedings of Interspeech . 3342–3346

2019
[14]

Kijpokin Kasemsap. 2017. Mastering web mining and information retrieval in the digital age. In Web usage mining techniques and applications across industries . IGI Global, 1–28

2017
[15]

Yoon Kim. 2014. Convolutional neural networks for sentence classification.arXiv preprint arXiv:1408.5882 (2014)

2014 arXiv
[16]

Dimitrios Kollias. 2023. Abaw: Learning from synthetic data & multi-task learning challenges. In European Conference on Computer Vision . Springer, 157–172

2023
[17]

Harshit Kumar, Arvind Agarwal, and Sachindra Joshi. 2019. A Practical Dialogue- Act-Driven Conversation Model for Multi-Turn Response Selection. InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference ...

2019
[18]

Vitaly Kurin, Alessandro De Palma, Ilya Kostrikov, Shimon Whiteson, and Pawan K Mudigonda. 2022. In defense of the unitary scalarization for deep multi-task learning. Advances in Neural Information Processing Systems 35 (2022), 12169–12183

2022
[19]

Shu Li, Wei Shen, Muhammad Bilal, Xiaolong Xu, Wanchun Dou, and Nour Moustafa. 2021. Fair and size-scalable participant selection framework for large- scale mobile crowdsensing. Journal of Systems Architecture 119 (2021), 102273

2021
[20]

Xiujun Li, Yun-Nung Chen, Lihong Li, Jianfeng Gao, and Asli Celikyilmaz. 2017. End-to-end task-completion neural dialogue systems. In Proceedings of the Eighth International Joint Conference on Natural Language Processing . 733–743. https: //www.aclweb.org/anthology/I17-1074.pdf

2017
[21]

Yuliang Liu, Junjie Lu, Zhaoling Chen, Chaofeng Qu, Jason Klein Liu, Chonghan Liu, Zefan Cai, Yunhui Xia, Li Zhao, Jiang Bian, et al. 2025. Adaptivestep: Au- tomatically dividing reasoning step through model confidence. arXiv preprint arXiv:2502.13943 (2025)

2025 arXiv
[22]

Bodhisattwa Prasad Majumder, Sudha Rao, Michel Galley, and Julian McAuley
[23]

Nikola Mrkšić, Diarmuid Ó Séaghdha, Tsung-Hsien Wen, Blaise Thomson, and Steve Young. 2017. Neural Belief Tracker: Data-Driven Dialogue State Tracking. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 1777–1788. https://www.aclweb.org...

2017
[24]

Kainan Peng, Wei Ping, Zhao Song, and Kexin Zhao. 2019. Parallel neural text- to-speech. arXiv preprint arXiv:1905.08459 (2019), 422–425

2019 arXiv
[25]

Pragaash Ponnusamy, Alireza Roshan Ghias, Chenlei Guo, and Ruhi Sarikaya
[26]

Sainath, Rohit Prabhavalkar, Anjuli Kannan, and Ding Zhao

Golan Pundak, Tara N. Sainath, Rohit Prabhavalkar, Anjuli Kannan, and Ding Zhao. 2018. Deep context: end-to-end contextual speech recognition. In IEEE Spoken Language Technology Workshop . 418–425. https://arxiv.org/pdf/1808. 02480.pdf

2018
[27]

Solmaz Sebnem Severcan, Nigmet Uzal, and Kevser Kahraman. 2020. Clarification of Apple Juice Using New Generation Nanocomposite Membranes Fabricated with TiO 2 and Al 2 O 3 Nanoparticles. Food and Bioprocess Technology 13, 3 (2020), 391–403

2020
[28]

Wei Shen, Xiaonan He, Chuheng Zhang, Qiang Ni, Wanchun Dou, and Yan Wang
[29]

Wei Shen, Xiaonan He, Chuheng Zhang, Xuyun Zhang, and Jian Xie. 2022. A Transformer-Based User Satisfaction Prediction for Proactive Interaction Mech- anism in DuerOS. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management . 1777–1786

2022
[30]

Zhulin Tao, Xiaohao Liu, Yewei Xia, Xiang Wang, Lifang Yang, Xianglin Huang, and Tat-Seng Chua. 2022. Self-supervised learning for multimedia recommenda- tion. IEEE Transactions on Multimedia (2022)

2022
[31]

Amrita S Tulshan and Sudhir Namdeorao Dhage. 2019. Survey on virtual assis- tant: Google assistant, siri, cortana, alexa. In Advances in Signal Processing and Intelligent Recognition Systems: 4th International Symposium SIRS 2018, Bangalore, India, September 19–22, 2018, Revis...

2019
[32]

Tsung-Hsien Wen, David Vandyke, Nikola Mrksic, Milica Gasic, Lina M Rojas- Barahona, Pei-Hao Su, Stefan Ultes, and Steve Young. 2016. A network-based end- to-end trainable task-oriented dialogue system. arXiv preprint arXiv:1604.04562 (2016)

2016 arXiv
[33]

In Proceedings of the 29th ACM International Conference on Information & Knowledge Management

Auxiliary-task based deep reinforcement learning for participant selection problem in mobile crowdsourcing. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management . 1355–1364
[34]

Tiansheng Yao, Xinyang Yi, Derek Zhiyuan Cheng, Felix Yu, Ting Chen, Aditya Menon, Lichan Hong, Ed H Chi, Steve Tjoa, Jieqi Kang, et al. 2021. Self-supervised learning for large-scale item recommendations. In Proceedings of the 30th ACM International Conference on Information ...

2021
[35]

Hamed Zamani, Gord Lueck, Everest Chen, Rodolfo Quispe, Flint Luu, and Nick Craswell. 2020. Mimics: A large-scale data collection for search clarification. In Proceedings of the 29th ACM International Conference on Information and Knowledge Management. 3189–3196

2020
[36]

Chenwei Zhang, Yaliang Li, Nan Du, Wei Fan, and Philip S. Yu. 2019. Joint Slot Filling and Intent Detection via Capsule Neural Networks. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics . 5259–5267. https://www.aclweb.org/anthology/P19-1519

2019
[37]

Zhe Zhao, Lichan Hong, Li Wei, Jilin Chen, Aniruddh Nath, Shawn Andrews, Aditee Kumthekar, Maheswaran Sathiamoorthy, Xinyang Yi, and Ed Chi. 2019. Recommending what video to watch next: a multitask ranking system. In Pro- ceedings of the 13th ACM Conference on Recommender Syst...

2019
[38]

Mingkun Yang, Minghui Liao, Pu Lu, Jing Wang, Shenggao Zhu, Hualin Luo, Qi Tian, and Xiang Bai. 2022. Reading and writing: Discriminative and generative modeling for self-supervised text recognition. In Proceedings of the 30th ACM International Conference on Multimedia . 4214–4223

2022
[2019]

In Proceedings of the 57th Conference of the Association for Computational Linguistics

Semantically Conditioned Dialog Response Generation via Hierarchical Dis- entangled Self-Attention. In Proceedings of the 57th Conference of the Association for Computational Linguistics. 3696–3709. https://www.aclweb.org/anthology/ P19-1360/
[2020]

In Proceedings of the AAAI Conference on Artificial Intelligence, Vol

Feedback-based self-learning in large-scale conversational ai agents. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 13180–13187
[2021]

arXiv preprint arXiv:2104.06828 (2021)

Ask what’s missing and what’s useful: Improving Clarification Question Generation using Global Knowledge. arXiv preprint arXiv:2104.06828 (2021)

2021 arXiv
[2022]

In Proceedings of the AAAI Conference on Artificial Intelligence, Vol

Contrastive learning from extremely augmented skeleton sequences for self-supervised action recognition. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 762–770

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Aman Alok, Rahul Gupta, and Shankar Ananthakrishnan. 2020. Design Consid- erations for Hypothesis Rejection Modules in Spoken Language Understanding Systems. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 8049–8053

work page 2020

[2] [2]

Junwen Bai, Bo Li, Yu Zhang, Ankur Bapna, Nikhil Siddhartha, Khe Chai Sim, and Tara N Sainath. 2022. Joint unsupervised and supervised training for multilingual asr. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 6402–6406

work page 2022

[3] [3]

Praveen Kumar Bodigutla, Aditya Tiwari, Josep Vallas Vargas, Lazaros Poly- menakos, and Spyros Matsoukas. 2020. Joint Turn and Dialogue level User Satisfaction Estimation on Multi-Domain Conversations. arXiv preprint arXiv:2010.02495 (2020)

work page Pith review arXiv 2020

[4] [4]

Christian Buck, Jannis Bulian, Massimiliano Ciaramita, Wojciech Gajewski, An- drea Gesmundo, Neil Houlsby, and Wei Wang. 2018. Ask the Right Questions: Active Question Reformulation with Reinforcement Learning. In Proceedings of International Conference on Learning Representations . https://openreview.net/ forum?id=S1CChZ-CZ

work page 2018

[5] [5]

Yuanying Cai, Chuheng Zhang, Wei Shen, Xiaonan He, Xuyun Zhang, and Longbo Huang. 2022. Imitation learning to outperform demonstrators by directly extrap- olating demonstrations. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management . 128–137

work page 2022

[6] [6]

Yuanying Cai, Chuheng Zhang, Wei Shen, Xuyun Zhang, Wenjie Ruan, and Longbo Huang. 2023. RePreM: representation pre-training with masked model for reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 6879–6887

work page 2023

[7] [7]

Yuanying Cai, Chuheng Zhang, Li Zhao, Wei Shen, Xuyun Zhang, Lei Song, Jiang Bian, Tao Qin, and Tieyan Liu. 2022. TD3 with reverse KL regularizer for offline reinforcement learning from mixed datasets. In 2022 IEEE International Conference on Data Mining (ICDM) . IEEE, 21–30

work page 2022

[8] [8]

Giuseppe Castellucci, Valentina Bellomaria, Andrea Favalli, and Raniero Romag- noli. 2019. Multi-lingual intent detection and slot filling in a joint bert-based model. arXiv preprint arXiv:1907.02884 (2019)

arXiv 2019