Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

DeepDialogue: A Multi-Turn Emotionally-Rich Spoken Dialogue Dataset

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A 40,150-dialogue text-and-speech corpus tracks 20 emotions across 41 domains with coherent emotional arcs.

desk verdict Real dataset, real value, but the speech-emotion transfer result is speaker-confounded and the headline findings lack statistical grounding. read the letter →

arxiv 2505.19978 v1 pith:MUDFV2AL submitted 2025-05-26 cs.CL cs.SDeess.AS

Alkis Koudounas , Moreno La Quatra , Elena Baralis This is my paper · ORCID
classification cs.CLcs.SDeess.AS
keywords multi-turndialogueemotionrecognitionspokendatasettext-to-speechsynthesismultimodalLLM-basedgenerationqualityspeech
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces DeepDialogue, a text-and-speech dataset built by pairing nine language models to generate multi-turn conversations, filtering them with a human-calibrated LLM judge, and synthesizing audio for every dialogue. The authors claim the result is the first large-scale open multimodal dialogue resource that keeps emotional progressions intact across turns: 40,150 conversations, 41 domains, 20 emotions, and over 480 hours of speech per synthesis variant. They also argue the generation process exposes general lessons about LLM dialogue quality: smaller models lose coherence past six turns, concrete topics ground better than abstract ones, and conversations between different models beat same-model self-play. A sympathetic reader would care because the dataset gives conversational AI a place to learn tone and continuity that text-only corpora lack.

What carries the argument

The load-bearing mechanism is the four-stage pipeline: stratified sampling of a domain and an initial emotion with a probabilistic directed emotion-transition graph that guides each next turn; prompt-based generation by two LLM agents given full history, domain, target emotion, and a 25-word cap; hybrid filtering in which 984 human-annotated dialogues calibrate an open-source LLM judge ensemble; and two parallel text-to-speech strategies, one conditioned on reference audio from the RAVDESS emotional speech corpus and one using only linguistic cues for prosody. The emotion-transition graph, grounded in circumplex and wheel models of affect, plus the emotion-to-domain mapping, is what the paper uses to guarantee the claimed coherent emotional progressions rather than random affective jumps.

What would settle it

Train the same HuBERT-based classifier on DeepDialogue XTTS-v2 audio and evaluate zero-shot on emotional speech recorded by voices never used as synthesis references: if accuracy falls near chance while a train-on-target baseline stays high, the claimed transfer is an artifact of speaker overlap, while stable accuracy near 56.6% would confirm generalization.

Watch

Extended reading notes

Core claim

DEEP DIALOGUE's central claim is that a large, emotionally coherent multimodal dialogue corpus can be produced entirely by orchestrated LLM generation, hybrid human-machine filtering, and text-to-speech synthesis, and that this corpus is large enough and clean enough to support research on emotionally aware conversational agents. Concretely, the authors generate 65,600 raw conversations by pairing 9 instruction-tuned LLMs (4B–72B parameters) across 41 domains and 20 emotions, keep 40,150 after scoring with an open-source LLM ensemble that reproduces human judgments (Cohen's $\kappa = 0.79$), and synthesize two speech versions: one conditioned on emotional reference audio and one relying on linguistic cues alone. They further claim that the resulting audio passes a transfer check, with a model trained on DeepDialogue speech reaching 56.6% zero-shot accuracy on RAVDESS, close to the 65.3% linear-probe baseline trained directly on that corpus. The paper's secondary findings — longer dialogues induce more failures, concrete domains outperform abstract ones, and cross-model pairs are more coherent than same-model pairs — are presented as intrinsic properties of current LLM dialogue generation.

Load-bearing premise

The load-bearing premise is that the 56.6% zero-shot accuracy on RAVDESS reflects genuine emotional transfer, yet the synthesized training audio was conditioned on reference recordings from those same RAVDESS actors and emotions, so the test may be recognizing familiar voices rather than unseen emotional expression.

Editorial extensions

If this is right

  • The observed quality drop beyond six turns for sub-27B models implies that long-conversational agents need either larger models or explicit memory-and-coherence mechanisms, giving the dataset a concrete target for testing them.
  • Because concrete domains reliably produce more grounded and more evaluable dialogues, synthetic-data pipelines should weigh domain concreteness, not just domain coverage, when planning generation budgets.
  • Pairing different model families yields higher acceptance rates than same-model self-play, so conversational generation can exploit model diversity as a free quality boost.
  • The two speech variants enable a controlled ablation: comparing emotion conveyed by explicit acoustic conditioning against emotion inferred from text isolates the role of prosody in speech emotion recognition.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A cleaner transfer test would use held-out speakers: because the XTTS-v2 audio was conditioned on RAVDESS reference recordings from the same actors used in the zero-shot evaluation, the 56.6% figure probably conflates voice familiarity with emotional generalization, and re-running the probe on non-RAVDESS voices would settle it.
  • The same generation recipe — domain sampling, emotion-transition graph, and LLM pairing — could be repurposed to build emotion-labeled multimodal corpora for other languages or for task-oriented dialogues.
  • The concreteness effect suggests a cheap automatic quality proxy: average word-concreteness scores could filter for grounded dialogue without an expensive LLM judge.
  • The emotion-transition graph encodes one cultural model of affective dynamics, so the dataset's 'coherent' arcs may not fit other cultures' conversational norms; testing the graph across cultures would show how portable the emotional structure is.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper presents DeepDialogue, a synthetically generated multimodal dialogue dataset containing 40,150 filtered conversations across 41 domains, with turn-level labels for 20 emotions, generated by nine LLMs used in 16 model pairings and then synthesized into speech with two TTS systems (XTTS-v2 with RAVDESS-conditioned emotion and Orpheus with implicit prosody). The construction pipeline is described in detail: 984 dialogues were human-annotated with substantial inter-annotator agreement, an LLM ensemble calibrated on that subset filtered 65,600 raw dialogues down to 40,150, and the final release includes text and about 480 hours of audio per TTS variant. The paper also reports secondary findings on model pairing, turn length, and domain concreteness, and uses a speech emotion recognition experiment on a balanced subset of the XTTS-v2 audio to argue that the speech component preserves and transfers emotional content.

Significance. The resource is potentially valuable: DeepDialogue is far larger in dialogue and domain coverage than existing emotion-labeled spoken dialogue datasets, it is openly released, and the appendix documentation (domains, emotion transition graph, prompts, annotation protocol, computational costs, bias analysis) is unusually transparent and would support reproducibility. The human-annotation calibration of LLM judges is a useful methodological contribution. However, the paper's strongest advertised validation—that the speech component faithfully preserves and transfers emotion—rests on an experiment with speaker leakage between training and test, and the headline behavioral findings about cross-model pairings and model size are reported without appropriate controls. If the leakage is not fixable, the transferability claim should be withdrawn; nonetheless, the dataset itself and the text-level quality pipeline are likely to be useful to the community after revision.

major comments (3)
  1. [Section 4.2, Table 3; Section 3.4] The zero-shot transfer conclusion is invalidated by speaker overlap between training audio and test set: the XTTS-v2 training audio was synthesized using reference voices sampled from RAVDESS actors for each actor-emotion combination, and the zero-shot test set is RAVDESS itself, with no actor-disjoint split reported. A HuBERT classifier can therefore reach 56.6% accuracy by recognizing the same voices or speaker-specific expressive styles rather than transferring an abstract emotion representation to unseen speakers. The HuBERT-LP baseline (65.3%) does not remedy this, since it was trained directly on RAVDESS, and the 8.7-point gap is not evidence of near-transfer. The claims that the speech component 'faithfully preserves emotional context' and that the emotional content is 'transferable' should be removed or re-established with strict actor-disjoint evaluation.
  2. [Section 4.1, Figure 2] The cross-model effect is confounded with model size and lacks statistical support. Figure 2a compares aggregated acceptance rates (0.65 vs. 0.54) but the different-model group includes the large LLaMA3-70B-Qwen2.5-72B pair while the same-model group contains many small self-pairs; Figure 2b itself shows acceptance rate increasing with parameter scale. No confidence intervals, hypothesis tests, or model-size-controlled analysis are provided, and the analysis is based on a subset of 984 human-annotated dialogues spread over 16 pairs. The abstract and conclusion repeat the cross-model claim as a finding, so this needs either a controlled analysis or a substantially weakened statement.
  3. [Section 4.1, Figure 3a; abstract/conclusion] The claim that smaller models fail to maintain coherence beyond six turns is not supported by the reported experiments. Figure 3a plots aggregate accepted/rejected counts over all human-annotated dialogues by turn number and shows overall degradation with length, but it does not condition on model size or model pair; Figure 13 in the appendix does break down invalid dialogues by pair, but not by turn number, and no test for a model-size interaction is reported. The paper should either present the missing per-size analysis or revise the claim to refer to the aggregate annotated subset.
minor comments (5)
  1. [Section 1 and Appendix A.4] The number of model pairs is inconsistent: the main text says 14 different model pairs, while Section A.4 says 16 unique pairings and Table 8 lists 16 pairs. Please reconcile.
  2. [Table 7] The emotion mapping includes 'Neutral' as a DeepDialogue emotion, but Section 3.1 and Table 5 define a 20-emotion taxonomy that does not include Neutral; the mapping should be corrected or the taxonomy amended.
  3. [Appendix D] The SER dataset description says 'approximately 8,000 dialogues' and 1,000 utterances per emotion, but the reported split totals 7,956 utterances; please clarify whether the final balanced subset has 1,000 instances for most classes and 619 for Disgust, and say 'utterances' rather than 'dialogues' where appropriate.
  4. [Figure 15] The two panels in Figure 15 appear to show identical conversations; the Books and Coding examples should be distinct.
  5. [Section 4.2] The text states the zero-shot evaluation shares the same emotion label distribution as RAVDESS, but the DeepDialogue training set uses the 8-class mapping; this sentence should specify that this holds after mapping the 20 emotion labels.

Circularity Check

1 steps flagged · score 6.0 of 10

Speech-emotion transfer claim is confounded: DeepDialogue-XTTS training audio was cloned from RAVDESS reference speakers, and the 'zero-shot' test is RAVDESS itself.

  1. fitted input called prediction [Section 3.4 (Speech Synthesis) and Section 4.2 (Speech Emotion Recognition); see also Appendix A.5 and Table 7 for the RAVDESS mapping.]
    "we leverage the RA VDESS dataset [33] ... For each actor-emotion combination, we concatenate the two standard RA VDESS sentences to create reference audio samples ... voices randomly selected from RA VDESS actors forXTTS-v2 (Sec. 3.4) ... We evaluate the best-performing model in a zero-shot setting on the RA VDESS dataset [33], which shares the same emotion label distribution (Sec. 4.2) ... Our zero-shot HuBERT-DD achieves an accuracy of 56.6% ... This suggests that the emotional content in DeepDialogue is not only consistent but also transferable."

    The DeepDialogue-XTTS training audio used in the SER experiment was produced by conditioning XTTS-v2 on RAVDESS reference clips: speakers for each dialogue were selected from RAVDESS actors, and for each actor-emotion pair the two standard RAVDESS sentences served as the reference clips. The 'zero-shot' target is RAVDESS itself, with the same 24 actors and the same 8 emotion labels. Consequently, the test voices and emotion-specific expressive styles are not unseen; a HuBERT classifier can reach 56.6% accuracy by recognizing RAVDESS voice identities or speaker-specific emotion styles that were present, in cloned form, in its training data. The claimed conclusion that the emotional content is transferable is therefore not supported as a true zero-shot generalization result.

full rationale

DeepDialogue's text pipeline is largely self-contained: human annotations (984 dialogues, Fleiss kappa = 0.80) are used to select an LLM judge, the judge filters the generated 65,600 dialogues, and the findings about turn length, domain concreteness, and cross-model pairings are measured on the human-annotated subset or with external lexicons such as Brysbaert concreteness ratings. Those derivations are not circular. The one genuine circularity is in the speech-validation experiment: the XTTS-v2 variant was synthesized using RAVDESS actors as reference voices (Section 3.4), and the same RAVDESS corpus is then used as the zero-shot SER test set (Section 4.2). Because the training audio contains cloned versions of the test speakers and emotion styles, the 56.6% accuracy cannot establish transferability to unseen voices; it is partially in-sample by construction. The HuBERT-LP comparison (reference [26], a same-group citation) is only a baseline number and is not load-bearing beyond this confounded comparison. The limitations section acknowledges synthetic-speech imperfections but does not mention the RAVDESS overlap, which is worth flagging. Overall, the dataset's text claims remain independent, so the circularity is partial rather than total, supporting a score of 6.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claims rest on several design choices (thresholds, temperatures, transition weights) and on domain assumptions about annotation validity, LLM-judge generalization, concreteness measurement, emotion mapping, and the emotion transition graph. No new physical or mathematical entities are introduced.

free parameters (4)
  • acceptance_threshold = score >= 3 on all dimensions and positive binary judgment
    This threshold determines which of the 65,600 dialogues enter the final 40,150 dataset and shapes all downstream statistics about model pairs, turn counts, and domain quality. It is set by the authors without sensitivity analysis.
  • temperature settings = 0.6 for models <=10B, 0.3 for models >=70B
    Chosen to balance consistency and diversity during dialogue generation; affects the style and quality of generated turns and thus the acceptance rates.
  • maximum response length = 25 words per turn
    Imposed in the prompt to keep conversations natural; influences utterance statistics and TTS rendering.
  • emotion transition probabilities = not specified numerically
    The paper states transitions are 'weighted by transition probabilities' and 'sampled from weighted options' but does not report the weights. These probabilities determine the emotional arcs and thus the dataset's emotional coherence.
assumptions (5)
  • domain assumption The 984 human-annotated dialogues provide a valid ground-truth measure of dialogue quality for calibrating automated filters and for drawing conclusions about model capabilities.
    Used throughout Section 3.3 and 4.1 as the reference for both filter selection and the reported findings. The annotator pool is small, volunteer, and mostly under 30, so its representativeness is assumed.
  • domain assumption The LLM ensemble's judgments generalize to the full 65,600-dialogue pool with the same reliability as on the 984 annotated examples.
    Invoked in Section 3.3 when the ensemble is used to filter the entire dataset; no per-pair or per-domain validation is reported beyond the aggregate correlation.
  • domain assumption The Brysbaert concreteness lexicon is applicable to LLM-generated dialogue turns and that GPT-4o's labeling of domains as concrete or abstract is correct.
    Used in Section 4.1 and Figure 4 to compare concrete vs abstract domains. No check that the lexicon covers the LLM vocabulary or that the domain categorization is stable.
  • ad hoc to paper Mapping DeepDialogue's 20 emotions onto RAVDESS's 8 categories preserves the intended emotional content (e.g., 'disappointed' to 'disgust', 'surprised' to 'hopeful').
    This mapping (Table 7) is needed to make the SER training and the zero-shot RAVDESS evaluation possible. The mappings are not validated perceptually or with behavioral data, and some pairings are semantically strained.
  • domain assumption The emotion transition graph derived from Russell's circumplex and Plutchik's wheel is a valid model of realistic emotional progression.
    Used in Section 3.2 and Appendix A.3 to generate the emotional arcs. The specific chains and the stochastic weights are author-chosen and not independently validated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DeepDialogue: A Multi-Turn Emotionally-Rich Spoken Dialogue Dataset." pith.science (2026). https://pith.science/paper/MUDFV2AL

@misc{pith2026250519978,
  author       = {Pith},
  title        = {Pith review of: DeepDialogue: A Multi-Turn Emotionally-Rich Spoken Dialogue Dataset},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MUDFV2AL}},
  note         = {Machine review of arXiv:2505.19978}
}
read the original abstract

Recent advances in conversational AI have demonstrated impressive capabilities in single-turn responses, yet multi-turn dialogues remain challenging for even the most sophisticated language models. Current dialogue datasets are limited in their emotional range, domain diversity, turn depth, and are predominantly text-only, hindering progress in developing more human-like conversational systems across modalities. To address these limitations, we present DeepDialogue, a large-scale multimodal dataset containing 40,150 high-quality multi-turn dialogues spanning 41 domains and incorporating 20 distinct emotions with coherent emotional progressions. Our approach pairs 9 different language models (4B-72B parameters) to generate 65,600 initial conversations, which we then evaluate through a combination of human annotation and LLM-based quality filtering. The resulting dataset reveals fundamental insights: smaller models fail to maintain coherence beyond 6 dialogue turns; concrete domains (e.g., "cars," "travel") yield more meaningful conversations than abstract ones (e.g., "philosophy"); and cross-model interactions produce more coherent dialogues than same-model conversations. A key contribution of DeepDialogue is its speech component, where we synthesize emotion-consistent voices for all 40,150 dialogues, creating the first large-scale open-source multimodal dialogue dataset that faithfully preserves emotional context across multi-turn conversations.

Figures

Figures reproduced from arXiv: 2505.19978 by the authors.

Figure 1
Figure 1. DEEPDIALOGUE dataset generation framework. The pipeline includes (1) text-based dialogue generation with emotion and domain conditioning, (2) human annotation, (3) automated LLM-based filtering, and (4) dual text-to-speech synthesis strategies. Preprint. arXiv:2505.19978v1 [cs.CL] 26 May 2025 [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Acceptance rate (AR) for pairs considering same vs. different models. [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Valid and invalid dialogues (a) and negative reasons (b), as assessed by human annotators. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Comparison of concreteness score of concrete vs. abstract accepted dialogues. To quantitatively assess linguistic concreteness in multi-turn dialogues, we leverage the Brys￾baert concreteness ratings [8], a widely-used lexicon of 40,000 English words annotated with con…
Figure 5
Figure 5. Figure 5: Domain distribution in DEEPDIALOGUE [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: Emotion transition graph in DEEPDIALOGUE . • Semantic grounding: Domains like Finance include emotions as Anxious, Worried, Con￾fused, which are relevant to uncertainty and risk. • Affective diversity: Entertainment domains (Movies, Music) span both positive (Amused, E…
Figure 7
Figure 7. Figure 7: Heatmap of emotion-domain mapping in DEEPDIALOGUE. Presence of an emotion for a specific domain highlighted in forest green, absence in light gray. imposes a maximum response length of 25 words, encouraging concise and natural utterances. We also include example inputs…
Figure 8
Figure 8. Figure 8: Example of model prompts for dialogue generation. The placeholders in curly brackets for [PITH_FULL_IMAGE:figures/full_fig_p020_8.png]
Figure 9
Figure 9. Figure 9: Turn number distribution in DEEP￾DIALOGUE. Section §B.5 reports invalid reasons for dialogue se￾lection as assessed by human annotators, divided by model pairs. Finally, Section §B.6 outlines an evalu￾ation of gender and age bias in our dataset, revealing disparities a…
Figure 10
Figure 10. Figure 10: Model pairs distribution in DEEPDIALOGUE . Gemma3-4B, to ensure architectural diversity. The presence of both symmetric (same model) and asymmetric (cross-model) configurations allows for robust comparative analyses. A pie chart summa￾rizing the dialogue distribution …
Figure 11
Figure 11. Figure 11: Emotion distribution in [PITH_FULL_IMAGE:figures/full_fig_p022_11.png]
Figure 12
Figure 12. Figure 12: DEEPDIALOGUE Average utterance length per models (12a) and domain (12b). difficult, resulting in a larger proportion of dialogues failing to meet quality thresholds. This aligns with recent findings by [27], which highlight the degradation of LLM performance in multi-…
Figure 14
Figure 14. Figure 14: Screenshot of our dialogue annotation interface. The left panel displays the conversation [PITH_FULL_IMAGE:figures/full_fig_p027_14.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Conversational Human Audio-visual Talking Dialogue Generation

    cs.CV 2026-07 conditional novelty 6.0 of 10

    CHAT generates mutually responsive dyadic audio-visual dialogue clips from a single text prompt and yields a 50k synthetic pre-training set that improves facial reaction models on REACT 2024.

Reference graph

Works this paper leans on

53 extracted references · 35 canonical work pages · cited by 1 Pith paper

  1. [1]

    Phi-4 technical report

    Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J Hewett, Mojan Javaheripi, Piero Kauffmann, et al. Phi-4 technical report. arXiv preprint arXiv:2412.08905, 2024

    arXiv 2024

  2. [2]

    Phi-4-mini technical report: Compact yet powerful multimodal language models via mixture-of-loras

    Abdelrahman Abouelenin, Atabak Ashfaq, Adam Atkinson, Hany Awadalla, Nguyen Bach, Jianmin Bao, Alon Benhaim, Martin Cai, Vishrav Chaudhary, Congcong Chen, et al. Phi-4-mini technical report: Compact yet powerful multimodal language models via mixture-of-loras. arXiv preprint arXiv:2503.01743, 2025

    arXiv 2025

  3. [3]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  4. [4]

    Sd-eval: A benchmark dataset for spoken dialogue understanding beyond words

    Junyi Ao, Yuancheng Wang, Xiaohai Tian, Dekun Chen, Jun Zhang, Lu Lu, Yuxuan Wang, Haizhou Li, and Zhizheng Wu. Sd-eval: A benchmark dataset for spoken dialogue understanding beyond words. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024

    work page 2024

  5. [5]

    wav2vec 2.0: A framework for self-supervised learning of speech representations

    Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. wav2vec 2.0: A framework for self-supervised learning of speech representations. Advances in neural information processing systems, 33:12449–12460, 2020

    2020

  6. [6]

    Qwen technical report

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.16609, 2023

    arXiv 2023

  7. [7]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020

    1901

  8. [8]

    Concreteness ratings for 40 thousand generally known english word lemmas

    Marc Brysbaert, Amy Beth Warriner, and Victor Kuperman. Concreteness ratings for 40 thousand generally known english word lemmas. Behavior research methods, 46:904–911, 2014

    work page 2014

Show all 53 references
  1. [9]

    MultiWOZ - a large-scale multi-domain Wizard-of-Oz 10 dataset for task-oriented dialogue modelling

    Paweł Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Iñigo Casanueva, Stefan Ultes, Osman Ramadan, and Milica Gaši´c. MultiWOZ - a large-scale multi-domain Wizard-of-Oz 10 dataset for task-oriented dialogue modelling. In Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun’...

    2018

  2. [10]

    Iemocap: Interactive emotional dyadic motion capture database

    Carlos Busso, Murtaza Bulut, Chi-Chun Lee, Abe Kazemzadeh, Emily Mower, Samuel Kim, Jeannette N Chang, Sungbok Lee, and Shrikanth S Narayanan. Iemocap: Interactive emotional dyadic motion capture database. Language resources and evaluation, 42:335–359, 2008

    2008

  3. [11]

    Xtts: a massively multilingual zero-shot text-to-speech model

    Edresson Casanova, Kelly Davis, Eren Gölge, Görkem Göknar, Iulian Gulea, Logan Hart, Aya Aljafari, Joshua Meyer, Reuben Morais, Samuel Olayemi, and Julian Weber. Xtts: a massively multilingual zero-shot text-to-speech model. In Interspeech 2024, pages 4978–4982, 2024

    2024

  4. [12]

    Action-based conversations dataset: A corpus for building more in-depth task-oriented dialogue systems

    Derek Chen, Howard Chen, Yi Yang, Alexander Lin, and Zhou Yu. Action-based conversations dataset: A corpus for building more in-depth task-oriented dialogue systems. In Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tur, Iz Beltagy, Steven Bethard, Ryan Co...

    2021

  5. [13]

    Wavlm: Large-scale self-supervised pre- training for full stack speech processing

    Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, et al. Wavlm: Large-scale self-supervised pre- training for full stack speech processing. IEEE Journal of Selected Topics in Signal Processing, 16(6)...

    2022

  6. [14]

    Chatbot arena: An open platform for evaluating llms by human preference

    Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Banghua Zhu, Hao Zhang, Michael Jordan, Joseph E Gonzalez, et al. Chatbot arena: An open platform for evaluating llms by human preference. In Forty-first International Conference...

    2024

  7. [15]

    Towards multimodal emotional support conversation systems

    Yuqi Chu, Lizi Liao, Zhiyuan Zhou, Chong-Wah Ngo, and Richang Hong. Towards multimodal emotional support conversation systems. arXiv preprint arXiv:2408.03650, 2024

    2024 arXiv

  8. [16]

    Command a: An enterprise-ready large language model

    Team Cohere, Arash Ahmadian, Marwan Ahmed, Jay Alammar, Yazeed Alnumay, Sophia Althammer, Arkady Arkhangorodsky, Viraat Aryabumi, Dennis Aumiller, Raphaël Avalos, et al. Command a: An enterprise-ready large language model. arXiv preprint arXiv:2504.00698, 2025

    2025 arXiv

  9. [17]

    Key-value retrieval networks for task-oriented dialogue

    Mihail Eric, Lakshmi Krishnan, Francois Charette, and Christopher D Manning. Key-value retrieval networks for task-oriented dialogue. In Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue. Association for Computational Linguistics, 2017

    2017

  10. [18]

    The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability

    Joseph L Fleiss and Jacob Cohen. The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educational and psychological measurement, 33(3):613–619, 1973

    1973

  11. [19]

    Gemini 2.0 flash, 2025

    Google. Gemini 2.0 flash, 2025

    2025

  12. [20]

    Gemini 2.5 flash preview, 2025

    Google. Gemini 2.5 flash preview, 2025

    2025

  13. [21]

    Gemini 2.5 pro preview, 2025

    Google. Gemini 2.5 pro preview, 2025

    2025

  14. [22]

    The llama 3 herd of models

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  15. [23]

    Williams

    Matthew Henderson, Blaise Thomson, and Jason D. Williams. The second dialog state tracking challenge. In Kallirroi Georgila, Matthew Stone, Helen Hastie, and Ani Nenkova, editors, Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (S...

    2014

  16. [24]

    Hubert: Self-supervised speech representation learning by masked prediction of hidden units

    Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdelrahman Mohamed. Hubert: Self-supervised speech representation learning by masked prediction of hidden units. IEEE/ACM transactions on audio, speech, and language processing, 29:...

    2021

  17. [25]

    Gpt-4o system card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024

    2024 arXiv

  18. [26]

    Benchmarking representations for speech, music, and acoustic events

    Moreno La Quatra, Alkis Koudounas, Lorenzo Vaiani, Elena Baralis, Luca Cagliero, Paolo Garza, and Sabato Marco Siniscalchi. Benchmarking representations for speech, music, and acoustic events. In 2024 IEEE International Conference on Acoustics, Speech, and Signal Processing Wo...

    2024

  19. [27]

    Llms get lost in multi-turn conversation, 2025

    Philippe Laban, Hiroaki Hayashi, Yingbo Zhou, and Jennifer Neville. Llms get lost in multi-turn conversation, 2025

    2025

  20. [28]

    Orpheus-3b-0.1-ft: A multilingual text-to-speech model

    Canopy Labs. Orpheus-3b-0.1-ft: A multilingual text-to-speech model. https:// huggingface.co/canopylabs/orpheus-3b-0.1-ft , 2025. Fine-tuned version of Orpheus for expressive TTS

    2025

  21. [29]

    Dailytalk: Spoken dialogue dataset for conver- sational text-to-speech

    Keon Lee, Kyumin Park, and Daeyoung Kim. Dailytalk: Spoken dialogue dataset for conver- sational text-to-speech. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE, 2023

    2023

  22. [30]

    DailyDialog: A manually labelled multi-turn dialogue dataset

    Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu. DailyDialog: A manually labelled multi-turn dialogue dataset. In Greg Kondrak and Taro Watanabe, editors, Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: L...

    2017

  23. [31]

    Concept understanding in large language models: An empirical study

    Jiayi Liao, Xu Chen, and Lun Du. Concept understanding in large language models: An empirical study. In Tiny Papers@ ICLR, 2023

    2023

  24. [32]

    Advancing large language models to capture varied speaking styles and respond properly in spoken conversations

    Guan-Ting Lin, Cheng-Han Chiang, and Hung-Yi Lee. Advancing large language models to capture varied speaking styles and respond properly in spoken conversations. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),...

    2024

  25. [33]

    The ryerson audio-visual database of emotional speech and song (ravdess): A dynamic, multimodal set of facial and vocal expressions in north american english

    Steven R Livingstone and Frank A Russo. The ryerson audio-visual database of emotional speech and song (ravdess): A dynamic, multimodal set of facial and vocal expressions in north american english. PloS one, 13(5):e0196391, 2018

    2018

  26. [34]

    Expresso: A benchmark and analysis of discrete expressive speech resynthesis

    Tu Anh Nguyen, Wei-Ning Hsu, Antony d’Avirro, Bowen Shi, Itai Gat, Maryam Fazel-Zarani, Tal Remez, Jade Copet, Gabriel Synnaeve, Michael Hassid, et al. Expresso: A benchmark and analysis of discrete expressive speech resynthesis. In INTERSPEECH 2023-24th Annual Conference of t...

    2023

  27. [35]

    Gpt-4o-mini: Advancing cost-efficient intelligence, 2024

    OpenAI. Gpt-4o-mini: Advancing cost-efficient intelligence, 2024

    2024

  28. [36]

    Let’s go real talk: Spoken dialogue model for face-to-face conversation

    Se Jin Park, Chae Won Kim, Hyeongseop Rha, Minsu Kim, Joanna Hong, Jeong Hun Yeo, and Yong Man Ro. Let’s go real talk: Spoken dialogue model for face-to-face conversation. In The 62nd Annual Meeting of the Association for Computational Linguistics. Association for Computationa...

    2024

  29. [37]

    The nature of emotions: Human emotions have deep evolutionary roots, a fact that may explain their complexity and provide tools for clinical practice

    Robert Plutchik. The nature of emotions: Human emotions have deep evolutionary roots, a fact that may explain their complexity and provide tools for clinical practice. American scientist, 89(4):344–350, 2001

    2001

  30. [38]

    Meld: A multimodal multi-party dataset for emotion recognition in conver- sations

    Soujanya Poria, Devamanyu Hazarika, Navonil Majumder, Gautam Naik, Erik Cambria, and Rada Mihalcea. Meld: A multimodal multi-party dataset for emotion recognition in conver- sations. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. As...

    2019

  31. [39]

    Emotion recognition in conversation: Research challenges, datasets, and recent advances

    Soujanya Poria, Navonil Majumder, Rada Mihalcea, and Eduard Hovy. Emotion recognition in conversation: Research challenges, datasets, and recent advances. IEEE access, 7:100943– 100953, 2019

    2019

  32. [40]

    Recipes for building an open- domain chatbot

    Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Eric Michael Smith, Y-Lan Boureau, and Jason Weston. Recipes for building an open- domain chatbot. In Paola Merlo, Jorg Tiedemann, and Reut Tsarfaty, editors, Proceedings of the 16...

    2021

  33. [41]

    A circumplex model of affect

    James A Russell. A circumplex model of affect. Journal of personality and social psychology, 39(6):1161, 1980

    1980

  34. [42]

    Building a conversational agent overnight with dialogue self-play

    Pararth Shah, Dilek Hakkani-Tür, Gokhan Tür, Abhinav Rastogi, Ankur Bapna, Neha Nayak, and Larry Heck. Building a conversational agent overnight with dialogue self-play. arXiv preprint arXiv:1801.04871, 2018

    2018 arXiv

  35. [43]

    Spokenwoz: A large-scale speech-text benchmark for spoken task-oriented dialogue agents

    Shuzheng Si, Wentao Ma, Haoyu Gao, Yuchuan Wu, Ting-En Lin, Yinpei Dai, Hangyu Li, Rui Yan, Fei Huang, and Yongbin Li. Spokenwoz: A large-scale speech-text benchmark for spoken task-oriented dialogue agents. Advances in Neural Information Processing Systems, 36:39088–39118, 2023

    2023

  36. [44]

    Gemma 3 technical report

    Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, et al. Gemma 3 technical report. arXiv preprint arXiv:2503.19786, 2025

    2025 arXiv

  37. [45]

    Lamda: Language models for dialog applications

    Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng- Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, et al. Lamda: Language models for dialog applications. arXiv preprint arXiv:2201.08239, 2022

    2022 arXiv

  38. [46]

    Dawn of the transformer era in speech emotion recognition: closing the valence gap

    Johannes Wagner, Andreas Triantafyllopoulos, Hagen Wierstorf, Maximilian Schmitt, Felix Burkhardt, Florian Eyben, and Björn W Schuller. Dawn of the transformer era in speech emotion recognition: closing the valence gap. IEEE Transactions on Pattern Analysis and Machine Intelli...

    2023

  39. [47]

    E-chat: Emotion-sensitive spoken dialogue system with large language models

    Hongfei Xue, Yuhao Liang, Bingshen Mu, Shiliang Zhang, Mengzhe Chen, Qian Chen, and Lei Xie. E-chat: Emotion-sensitive spoken dialogue system with large language models. In 2024 IEEE 14th International Symposium on Chinese Spoken Language Processing (ISCSLP), pages 586–590. IEEE, 2024

    2024

  40. [48]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115, 2024

    2024 arXiv

  41. [49]

    Overview of the tenth dialog system technology challenge: Dstc10

    Koichiro Yoshino, Yun-Nung Chen, Paul Crook, Satwik Kottur, Jinchao Li, Behnam Hedayatnia, Seungwhan Moon, Zhengcong Fei, Zekang Li, Jinchao Zhang, et al. Overview of the tenth dialog system technology challenge: Dstc10. IEEE/ACM Transactions on Audio, Speech, and Language Pro...

  42. [50]

    Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. Personalizing dialogue agents: I have a dog, do you have pets too? In Iryna Gurevych and Yusuke Miyao, editors, Proceedings of the 56th Annual Meeting of the Association for Computational L...

    2018

  43. [51]

    Have you heard about the new Tesla model?

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Tianle Li, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zhuohan Li, Zi Lin, Eric Xing, et al. Lmsys-chat-1m: A large-scale real-world llm conversation dataset. In The Twelfth International Conference on Learning Representations, 2024. ...

    2024

  44. [52]

    A binary validity score (0/1) indicating whether the conversation met minimum quality thresholds

  45. [53]

    Angry”, “Surprised

    A quality rating on a Likert scale (1-5) assessing the overall conversational coherence and naturalness. The annotation instructions emphasized three key dimensions: dialogue coherence, emotional consis- tency, and domain adherence. Annotators were instructed to assess whether...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.