OSUM-EChat: Enhancing End-to-End Empathetic Spoken Chatbot via Understanding-Driven Spoken Dialogue

Chengyou Wang; Dake Guo; Dehui Gao; Guobin Ma; Guojian Li; Hanke Xie; Hongfei Xue; Kangxiang Xia; Lei Xie; Longhao Li

REVIEW 4 major objections 5 minor 2 cited by

OSUM-EChat: Enhancing End-to-End Empathetic Spoken Chatbot via Understanding-Driven Spoken Dialogue

T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read An end-to-end spoken dialogue system can become empathetic by reusing speech understanding: OSUM-EChat trains a 3B model in three stages and adds a linguistic–paralinguistic chain-of-thought, scoring 72.0 on multi-label EChat-eval and beati

desk verdict Useful system and dataset, but the headline empathy claim is confounded by in-distribution evaluation; the right verdict is conditional acceptance after the eval is redone. read the letter →

arxiv 2508.09600 v2 pith:PFZ6LOOQ submitted 2025-08-13 cs.SD

Xuelong Geng , Qijie Shao , Hongfei Xue , Shuiyuan Wang , Hanke Xie , Zhao Guo , Yi Zhao , Guojian Li

show 15 more authors

Wenjie Tian Chengyou Wang Zhixian Zhao Kangxiang Xia Ziyu Zhang Zhennan Lin Tianlun Zuo Mingchen Shao Yuang Cao Guobin Ma Longhao Li Yuhang Dai Dehui Gao Dake Guo Lei Xie

This is my paper · ORCID

classification cs.SD

keywords empatheticspokendialogueend-to-endspeechlanguagemodelparalinguisticcueschain-of-thoughtspeech-to-speechEChat-200KEChat-evalresource-limitedtraining

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to show that an end-to-end spoken dialogue system can produce empathetic replies without the huge proprietary dialogue datasets that current speech-language models depend on. The proposed system, OSUM-EChat, is trained in three stages—understanding, generation, empathy—so that paralinguistic perception learned by a speech understanding model is carried over into speech generation. Before each reply, the model runs a short chain-of-thought that names the user's words and their paralinguistic cues (age, gender, emotion, sound event), then speaks a response tailored to those cues. On the paper's new EChat-eval benchmark, OSUM-EChat scores 72.0 in multi-label empathetic scenarios, ahead of the end-to-end spoken models it is compared with, and ablations show both the training strategy and the chain-of-thought mechanism are needed for the gain. If these results hold, empathetic spoken assistants become feasible with limited data and open components.

What carries the argument

Two coupled mechanisms carry the argument. The first is an understanding-driven spoken dialogue training strategy: three stages (understanding, generation, empathy) that first trains ASR plus paralinguistic multi-task recognition, then adds text-to-speech and speech-to-speech generation, then fine-tunes on the EChat-200K corpus. The second is a linguistic–paralinguistic dual think mechanism: a chain-of-thought enclosed in <think>...</think end> tokens in which the model restates the user's words and predicts age, gender, emotion, and sound-event labels before producing text and speech tokens. Together they let the model use speech-understanding knowledge as a scaffold for generation, so only

What would settle it

Replace ChatGPT-4o in EChat-eval with blind human raters scoring the same query speech, labels, and response audio, and also test on real unsynthesized emotional conversations. If the ranking between OSUM-EChat and the best compared end-to-end model flips on emotion or multi-label tasks—or if the human advantage disappears outside EChat-style synthesized data—then the 72.0 result is an artifact of the automatic scoring pipeline, not a stable empathetic ability.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that paralinguistic understanding—recognizing who is speaking and how—can be deliberately transferred into spoken dialogue generation instead of being re-learned from millions of hours of speech conversation. OSUM-EChat starts with a pretrained speech understanding backbone, adds speech generation through a speech-token codec, and during an 'empathy' stage inserts a think block in which the model first restates the linguistic content and then predicts the paralinguistic labels before generating text and speech tokens. The trained 3B model reaches EChat-eval scores of 58.0 (emotion), 63.1 (age), 62.7 (gender), 87.1 (sound event), and 72.0 (mu

Load-bearing premise

The load-bearing premise is that EChat-eval's automated scores—ChatGPT-4o judging emotion2vec-extracted labels—track what humans mean by empathy, and the paper's own limitation text concedes those automatic scores can diverge from human judgments; if they do not track true empathy, the headline advantage over other models may reflect fitting the synthetic benchmark rather than real empathetic ability.

Editorial extensions

If this is right

If the paper is right, empathy in spoken assistants no longer requires closed, web-scale dialogue corpora; a 3B model plus the EChat-200K corpus reaches the top of the compared end-to-end systems on EChat-eval.
The think block provides a readable intermediate for paralinguistic reasoning, so a deployed assistant could in principle show or audit what cues it detected before responding.
The benchmark gives the field a common multi-dimensional yardstick (emotion, age, gender, sound events, multi-label) that can be reused to compare future end-to-end empathetic systems.
Ablation results imply that simply fine-tuning a speech-language model on empathetic data is not enough: both understanding-stage transfer and the dual think mechanism contribute to the outcome.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper leaves untested whether the dual think mechanism helps on natural human speech outside the EChat pipeline; a likely consequence of the design is that gains are largest when input speech is cleanly labeled, and the benchmark should be cross-checked on real call-center or podcast-style conversations.
Because paralinguistic labels are explicit tokens in the chain-of-thought, the architecture offers a natural control knob: a developer could override a predicted age, emotion, or event label at inference to force a different tone without retraining.
The same understanding-to-generation transfer may extend to other paralinguistic dimensions not in EChat-200K—such as dialect, health state, or social relationship—by adding label slots rather than redesigning the model.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper is worth a serious referee, but not because the headline comparison is convincing. The actual contribution is the understanding-driven training recipe and the EChat-200K corpus. Three-stage training that starts from an ASR+P understanding model and then adds generation and empathy is a sensible way to reduce data hunger for spoken dialogue, and the ablations in Table 2 are internally consistent: removing either the understanding-driven strategy or the dual-think mechanism drops scores across the board. The dataset, with single-label and multi-label paralinguistic annotations and a mix of synthetic and real audio, should be useful to the community. The authors also deserve credit for shipping code, data, and a stated open-source ecosystem, and they are honest in the Limitation section about the automatic scoring being flawed.

The soft spot is the evaluation, and it is a load-bearing one. EChat-eval is constructed with the same DeepSeek/CosyVoice2 pipeline as EChat-200K, and OSUM-EChat is trained on EChat-200K. So the headline results in Table 2 measure how well the model mimics the synthetic query-to-response mapping, while all baselines are zero-shot on that distribution. That is a genuine confound. The Event metric makes it worse: scoring is keyword matching, and the training prompts explicitly require the caption label to be embedded in the response (e.g., "Don't keep coughing—want some warm water?"). A baseline that conveys the same awareness with different wording gets scored zero, while OSUM-EChat is rewarded for doing exactly what it was trained to do. The stress-test note is correct on this. The human evaluation in Table 3 covers only three models, with no inter-annotator agreement or confidence intervals, so it does not rescue the full ranking. The paper's own claim that automated scores "differ substantially from human evaluations" undercuts the main table further.

I would still send this to peer review. The training strategy and dataset are real contributions, the system is openly released, and the evaluation problem is fixable—rerun on a held-out set with out-of-distribution baselines, add human ratings for the full model set, and report variance. The authors seem aware of the limitations, which is a good sign. But the abstract's claim that OSUM-EChat "outperforms other end-to-end spoken dialogue models" is not supported by the current evidence. That needs to be tempered or re-proven.

Referee Report

4 major / 5 minor

Summary. The paper proposes OSUM-EChat, an end-to-end empathetic spoken dialogue system built from a Whisper encoder, a Qwen2.5-3B LLM, and CosyVoice-based speech-token decoding. Training is organized into three stages—speech understanding (ASR+P), generation (TTS and S2S), and empathy—with a linguistic–paralinguistic dual think chain-of-thought inserted before response generation. The authors construct EChat-200K, a synthetic speech-to-speech empathetic dialogue corpus, and EChat-eval, a multidimensional benchmark scored by ChatGPT-4o with emotion2vec-extracted labels, plus a small human evaluation. The reported experiments (Tables 2–6) show consistent internal ablations and competitive linguistic/speech-understanding performance, and the headline table reports large advantages over open end-to-end baselines on EChat-eval.

Significance. If the empirical claims survive closer evaluation, the paper would make a useful contribution to resource-limited end-to-end empathetic spoken dialogue: the three-stage recipe is sensible, the dual-think mechanism is a concrete way to inject paralinguistic understanding into generation, and the open-source release (model, code, dataset, benchmark) is valuable for reproducibility. The ablations are consistent, and the public-test-set measurements (Table 6) are reassuring. However, the central comparative claim is not yet established because the main benchmark is generated by the same pipeline as the training set and scored by an LLM whose limitations the authors themselves acknowledge. The evidence is promising but requires an independent evaluation before the headline conclusion can be accepted.

major comments (4)

[EChat-eval Bench / Appendix A.1; Table 2] The EChat-eval benchmark is 'constructed using the aforementioned data generation pipeline' (DeepSeek question/response generation plus CosyVoice2 synthesis), the same pipeline that produced the EChat-200K corpus used in Stage-3 empathy training (Table 1, 'Empathetic dialogue S2S'). OSUM-EChat is therefore evaluated in-distribution, while every baseline in Table 2 is zero-shot on this synthetic distribution. The reported gains (e.g., emotion 58.0 vs 44.8, multi-label 72.0 vs 63.7) may largely reflect matching the synthetic query→response distribution rather than general empathetic ability. The human validation in Table 3 covers only three models and does not include the main Table 2 baselines. Please add an out-of-distribution evaluation (e.g., human-scored natural dialogues, or public empathetic dialogue tests) covering all compared systems, or otherwise demonstrate that the benchmark i
[Appendix A.2, sound-event scoring; Table 2] The sound-event dimension is scored by exact/surface keyword matching between the response and a fixed list (sigh: [sigh, ai, sighing, ...]; cough: [cough, coughing, ...]). The EChat-200K multi-label construction prompt explicitly instructs the generator to embed the caption label in the response (e.g., 'Don't keep coughing—want some warm water?'). Since OSUM-EChat is trained on responses produced under this instruction, it is specifically optimized to produce the lexical cues used by the scorer. A baseline that conveys the same awareness with different wording (e.g., 'Are you still under the weather?' after a cough) would receive a low/zero Event score. The 87.1 Event score is therefore not a credible quantification of empathetic event awareness. Use semantic or human scoring, and report the matching dictionary and failure cases.
[Limitation; Tables 2-3] The paper concedes that emotion2vec label errors and ChatGPT-4o hallucinations cause automated scores to 'differ substantially from human evaluations,' yet the headline claim rests on the automated Table 2. The only human evaluation (Table 3) compares three systems, has dash entries for Event, and reports no number of annotators, per-item variance, confidence intervals, or inter-annotator agreement. Moreover, on the Emotion dimension the human score of OSUM-EChat (72.0) is lower than Doubao (78.0), so the 'outperforms' claim is already qualified in human evaluation. Statistical testing and a human study covering the main baselines and all dimensions are needed before the central claim can be accepted.
[Main Results / Ablations; Table 2 lower half] The ablations U-Driven and Dual Think also rely on EChat-eval. Since the benchmark shares the training pipeline, the ablation gaps may be inflated by benchmark-specific behavior. The public-test-set results in Table 6 are more convincing; please complement the EChat-eval ablations with human ratings or an independent OOD test set to show that the two proposed components contribute to generalizable empathetic dialogue ability rather than only to synthetic-distribution matching.

minor comments (5)

[Contrastive Models; Table 2] OpenS2S appears in Table 2 but is not named in the contrastive models list. Also, the dash entries in the Event column should be explicitly defined (not evaluated vs. zero score) to avoid ambiguity.
[Appendix A.2 / Figures 1, 3] The emotion-label taxonomy is inconsistent: 'cheerful' appears in response labels in Figures 1 and 3, while the construction prompts use 'happy'; 'angry/fear/surprised' and 'anger/fear/surprise' also vary across prompts. This can affect emotion2vec mapping and the stability of automatic scoring.
[Table 1 / EChat-200K] The entry '0.2 Kh' is ambiguous: the text says approximately 200k conversations, but the column header defines Kh as thousands of hours. State total hours and number of conversations separately.
[Table 5] The caption says 'WER%(↓) and CER%(↓) results' but the table only has test-zh and test-en columns; specify which metric is shown for each column and language.
[Eq. (4)] The reuse of LS2-S2S inside LS3 should be explicitly defined; as written it appears the Stage-2 loss term is added unweighted inside Stage 3, and the reader must infer the supervision format for the CoT tokens.

Circularity Check

1 steps flagged · score 6.0 of 10

EChat-eval's Event metric is a keyword match against the same caption labels that the EChat-200K data-construction prompt forces responses to embed; OSUM-EChat's 87.1 Event score is forced by construction.

self definitional [Appendix A.2 (multi-label data construction prompt; sound event test set evaluation prompt) and Table 2 Event row]
"The caption label should be naturally embedded into the response, e.g., 'Don't keep coughing—want some warm water?' ... The evaluation of the sound event test set is based on matching, using the following mapping relationships: ... cough : [cough, coughing, coughed once, ill]"

Stage 3 trains OSUM-EChat on EChat-200K, whose generation prompt requires the response to embed the caption label verbatim. EChat-eval scores sound-event responses by keyword matching against the same caption vocabulary. OSUM-EChat's 87.1 Event score therefore measures reproduction of the exact lexical template it was trained to emit; baselines that express the same empathy with different wording are not credited. The 'prediction' of empathetic response to an acoustic event is thus defined by the same token-matching rule that generated the training target, making the result partially forced by construction.

full rationale

This is an empirical systems paper, not a formal derivation, so most circularity failure modes do not apply. The three-stage training scheme is audited by ablations on public test sets; the OSUM citations are self-citations but not load-bearing in a circular way, since OSUM is an open-source model with independent public evaluations and is used as initialization/data rather than as a proof of the central claim. The specific construction-level circularity is in the Event dimension of EChat-eval: the data-construction prompt (Appendix A.2) instructs the response generator to embed the caption label verbatim (e.g., 'Don't keep coughing—want some warm water?'), and the evaluation prompt scores by keyword matching the same caption list (e.g., 'cough : [cough, coughing, coughed once, ill]'). Since OSUM-EChat's Stage-3 training corpus is generated from that pipeline, its 87.1 Event score is a measure of how faithfully it reproduces the training template, not an independent measure of acoustic-event empathy. This is partial circularity: one of the headline comparison numbers reduces by construction. The broader benchmark is also generated by the same DeepSeek/CosyVoice2 pipeline as the training corpus ('EChat-eval ... aligns with the task taxonomy of the EChat-200K training corpus and is constructed using the aforementioned data generation pipeline'), which puts the proposed model in-distribution and the baselines out-of-distribution; this is a validity concern rather than an equation-level tautology. The paper's own Limitation section concedes that 'EChat-eval's automatic scoring system faces several challenges, including inaccurate emotional label extraction and difficulty in scoring LLM hallucinations, which can differ substantially from human evaluations.' The human evaluation in Table 3 covers only three models without inter-annotator agreement or confidence intervals, so it does not fully support the full multi-model ranking claimed in the abstract. Apart from the Event keyword-match issue, the remaining results are self-contained and externally benchmarked against public test sets.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

No physically invented entities or mathematical objects are introduced; the contributions are architectural, data, and benchmark choices. The main burden is empirical: evaluation quality depends on the assumptions listed above.

free parameters (2)

Text-to-speech token interleaving ratio (6:18) = 6:18
Hand-chosen ratio controls when text and speech tokens alternate in streaming generation; no sensitivity analysis is reported.
Multi-task data weights (0.1 understanding, 0.2 empathy) = 0.1 and 0.2
Empirical weights balance understanding and empathy data in Stages 2 and 3; no ablation on these weights is shown.

assumptions (4)

domain assumption ASR+P multitask speech understanding transfers to spoken dialogue generation
The whole three-stage strategy assumes knowledge gained from understanding tasks (ASR plus paralinguistic labels) carries into dialogue generation; supported by ablations but not proven outside this setup.
domain assumption ChatGPT-4o plus emotion2vec scores are valid measures of empathetic response quality
All main benchmark scores depend on this; the paper itself notes emotion2vec label errors and ChatGPT hallucinations in the Limitation section, so the assumption is explicitly leaky.
domain assumption CosyVoice2-synthesized speech preserves realistic paralinguistic cues
Most of EChat-200K and parts of EChat-eval are synthesized; if synthetic emotion, age, and gender cues are weaker or different from real human speech, training and evaluation may not transfer.
domain assumption DeepSeek-generated paralinguistic labels and responses are accurate and diverse
The data pipeline relies on LLM-generated labels and responses without reported human verification or inter-annotator agreement.

how reviews work

0 comments

Cite this review

Pith. "Pith review of OSUM-EChat: Enhancing End-to-End Empathetic Spoken Chatbot via Understanding-Driven Spoken Dialogue." pith.science (2026). https://pith.science/paper/PFZ6LOOQ

@misc{pith2026250809600,
  author       = {Pith},
  title        = {Pith review of: OSUM-EChat: Enhancing End-to-End Empathetic Spoken Chatbot via Understanding-Driven Spoken Dialogue},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PFZ6LOOQ}},
  note         = {Machine review of arXiv:2508.09600}
}

read the original abstract

Empathy is crucial in enabling natural interactions within spoken dialogue systems, allowing machines to recognize and respond appropriately to paralinguistic cues such as age, gender, and emotion. Recent advancements in end-to-end speech language models, which unify speech understanding and generation, provide promising solutions. However, several challenges persist, including an over-reliance on large-scale dialogue datasets, insufficient extraction of paralinguistic cues vital for conveying empathy, and the lack of empathy-specific datasets and evaluation frameworks. To address these issues, we introduce OSUM-EChat, an open-source, end-to-end spoken dialogue system designed to enhance empathetic interactions, particularly in resource-limited settings. OSUM-EChat introduces two key innovations: (1) a three-stage understanding-driven spoken dialogue training strategy that extends the capabilities of a large speech understanding model to spoken dialogue tasks, and (2) a linguistic-paralinguistic dual thinking mechanism that integrates paralinguistic understanding through a chain of thought with dialogue generation, enabling the system to produce more empathetic responses. This approach reduces reliance on large-scale dialogue datasets while maintaining high-quality empathetic interactions. Additionally, we introduce the EChat-200K dataset, a rich corpus of empathetic speech-to-speech dialogues, and the EChat-eval benchmark, a comprehensive framework for evaluating the empathetic capabilities of dialogue systems. Experimental results demonstrate that OSUM-EChat outperforms end-to-end spoken dialogue models regarding empathetic responsiveness, validating its effectiveness.

Figures

Figures reproduced from arXiv: 2508.09600 by the authors.

**Figure 2.** The architecture of the OSUM-EChat model. This [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

**Figure 3.** Understanding-driven spoken dialogue training [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: The distribution of paralinguistic labels in the [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]

**Figure 5.** Figure 5: Data construction process for EChat-200K • Analyze the emotion and tone of the response, and prepend an emotion tag at the beginning of the response. Emotion options: angry, scared, happy, surprised, sad, disgusted, confused, sarcastic, embarrassed, curious, worried, …

**Figure 6.** Figure 6: Samples for EChat-eval • The user will provide a piece of text along with the age group of the speaker (child, adult, old). • You need to generate a gentle and understanding response based on the text content and the user’s age group. Responses should differ for child…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Sharp spectral estimates for free boundary problems arising in plasma physics
math.AP 2026-04 unverdicted novelty 5.0 of 10

For a constrained superlinear free-boundary plasma model, the non-local first eigenvalue σ₁ is always positive on balls in every dimension N≥2, despite lacking a general Faber–Krahn property.
Towards High-Level Semantic Intelligence
cs.AI 2026-07 conditional novelty 4.0 of 10

A survey proposing that AI's next stage should be understood as High-Level Semantic Intelligence: mastering humor, sarcasm, metaphor, empathy, persuasion, and narrative across modalities.

Reference graph

Works this paper leans on

23 extracted references · 3 linked inside Pith · cited by 2 Pith papers

[1]

Emotion is completely wrong, or the response is emo- tionally conflicting or meaningless
[2]

Emotion does not match, or the content is off-topic
[3]

Slight emotional mismatch, but the content is relevant
[4]

Emotion is generally appropriate, and the response is rea- sonable
[5]

Empathetic dialogue system: A human-machine dialogue system with emotion perception and adaptation capabilities

Emotion is highly accurate, and the response is semanti- cally relevant. Empathetic dialogue system: A human-machine dialogue system with emotion perception and adaptation capabilities. It can detect the user’s emotional state based on input (espe- cially emotional features in speech), and generate responses with corresponding emotional tones. The goal is...
[9]

The tone, vocabulary, advice, or style are all generic and could apply to any age group

Completely ignores age: The response does not reflect the user’s age in any way. The tone, vocabulary, advice, or style are all generic and could apply to any age group
[10]

Mentions age without meaningful adaptation: The sys- tem might briefly mention age, but the content remains largely unchanged and lacks personalization
[11]

Partially considers age: Some elements reflect age aware- ness, such as softer tone, simpler vocabulary, or slightly age-aligned suggestions, but overall, it is insufficient

Show all 23 references

[12]

Age-adapted: The system tailors the response across multiple dimensions (tone, form of address, advice, style), showing strong alignment with the user’s age
[13]

The style is consistent and highly tailored to the user’s age

Highly age-appropriate and coherent: The system deeply understands and fully integrates age factors across con- tent, structure, tone, and advice. The style is consistent and highly tailored to the user’s age. Input format: age, question, response. Output format: score: intege...
[14]

The re- sponse completely ignores gender-related information, is irrelevant to gender, or even contradicts the gender label

No reference to gender: The system shows no indication that it understands or considers the user’s gender. The re- sponse completely ignores gender-related information, is irrelevant to gender, or even contradicts the gender label
[15]

There is no meaningful personalization related to gender

Slight mention of gender without effective reference: The system briefly mentions gender, but this does not affect the core content of the response. There is no meaningful personalization related to gender
[16]

The overall tone remains neutral

Superficial reference to gender: The system reflects the user’s gender through minor address or expression de- tails, but it does not significantly influence the content. The overall tone remains neutral
[17]

Partial gender adaptation: The response has been par- tially adapted based on gender, possibly showing person- alized suggestions, topic emphasis, or tonal differences, though there is room for improvement
[18]

The gender information shapes the overall response and shows high personalization

Full gender adaptation with content impact: The system thoroughly considers the user’s gender across multiple aspects, including form of address, tone, advice, and con- text handling. The gender information shapes the overall response and shows high personalization. Input form...
[19]

The response is irrelevant, emotionally inappropriate, or potentially harmful
[20]

The response neglects or misinterprets the user’s paralin- guistic features, showing a lack of empathy
[21]

There are noticeable issues in either content or emotional expression
[22]

The response is generally appropriate with minor omis- sions or imperfections
[23]

automatic labeling

The response is natural, strongly aligned with paralin- guistic features, and emotionally appropriate. Only output a single Arabic numeral (1–5). Do not in- clude explanations or any additional text. Task Total Hours Language Open-sourced Dataset ASR 24k EN, CN Wenetspeech (Zh...

2017
[2008]

Language Resources and Evaluation (LREC), 42: 335–359

IEMOCAP: Interactive emotional dyadic motion cap- ture database. Language Resources and Evaluation (LREC), 42: 335–359. Chen, G.; Chai, S.; Wang, G.; Du, J.; Zhang, W.; Weng, C.; Su, D.; Povey, D.; Trmal, J.; Zhang, J.; Jin, M.; Khudanpur, S.; Watanabe, S.; Zhao, S.; Zou, W.; ...

2021 arXiv
[2017]

What do you think I should eat?

Audio Set: An ontology and human-labeled dataset for audio events. In Proceedings of the IEEE Interna- tional Conference on Acoustics, Speech and Signal Process- ing (ICASSP), 776–780. Geng, X.; Wei, K.; Shao, Q.; Liu, S.; Lin, Z.; Zhao, Z.; Li, G.; Tian, W.; Chen, P.; Li, Y ....

2025 arXiv
[2024]

CoRR, abs/2410.17196

V oiceBench: Benchmarking LLM-Based V oice As- sistants. CoRR, abs/2410.17196. Chu, Y .; Xu, J.; Yang, Q.; Wei, H.; Wei, X.; Guo, Z.; Leng, Y .; Lv, Y .; He, J.; Lin, J.; Zhou, C.; and Zhou, J. 2024. Qwen2-Audio Technical Report. CoRR, abs/2407.10759. Datatang Tech Co Ltd. 202...

2024 arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [1]

Emotion is completely wrong, or the response is emo- tionally conflicting or meaningless

[2] [2]

Emotion does not match, or the content is off-topic

[3] [3]

Slight emotional mismatch, but the content is relevant

[4] [4]

Emotion is generally appropriate, and the response is rea- sonable

[5] [5]

Empathetic dialogue system: A human-machine dialogue system with emotion perception and adaptation capabilities

Emotion is highly accurate, and the response is semanti- cally relevant. Empathetic dialogue system: A human-machine dialogue system with emotion perception and adaptation capabilities. It can detect the user’s emotional state based on input (espe- cially emotional features in speech), and generate responses with corresponding emotional tones. The goal is...

[6] [9]

The tone, vocabulary, advice, or style are all generic and could apply to any age group

Completely ignores age: The response does not reflect the user’s age in any way. The tone, vocabulary, advice, or style are all generic and could apply to any age group

[7] [10]

Mentions age without meaningful adaptation: The sys- tem might briefly mention age, but the content remains largely unchanged and lacks personalization

[8] [11]

Partially considers age: Some elements reflect age aware- ness, such as softer tone, simpler vocabulary, or slightly age-aligned suggestions, but overall, it is insufficient