Synthetic Audio Helps for Cognitive State Tasks

Adil Soubki; John Murzaku; Owen Rambow; Peter Zeng

REVIEW 3 major objections 4 minor 40 references

Synthetic Audio Helps for Cognitive State Tasks

T0 review · 3 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Zero-shot TTS audio improves seven cognitive-state NLP tasks when fused with text.

desk verdict Plausible empirical result — synthetic TTS audio helps text-only models on seven cognitive-state tasks — but the causal story about TTS-encoded prosody is under-supported by the controls and missing baselines. read the letter →

arxiv 2502.06922 v1 pith:3OVQ27SC submitted 2025-02-10 cs.SD cs.AIcs.CLcs.LG

Adil Soubki , John Murzaku , Peter Zeng , Owen Rambow This is my paper · ORCID

classification cs.SDcs.AIcs.CLcs.LG

keywords syntheticaudiotext-to-speechcognitivestatemultimodalfusionprosodybeliefpredictionemotionrecognitionsentimentanalysis

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that text-to-speech audio generated directly from a sentence can stand in for human audio in NLP tasks that model a speaker's or writer's cognitive state. It introduces a framework that pairs a BERT text encoder with a Whisper audio encoder trained on zero-shot TTS renderings, and reports that this multimodal pairing beats text-only BERT on all seven non-control tasks it tests, covering sentiment, belief, and emotion. On two corpora with gold human audio, the synthetic audio nearly matches human audio; on one emotion corpus it lags. If the result holds, any text-only cognitive-state dataset can be augmented with synthetic audio rather than expensive human recordings.

What carries the argument

The machinery is the SAD (Synthetic Audio Data) pipeline: take the input text, synthesize audio with an off-the-shelf TTS model (OpenAI tts-1-hd with the Alloy voice, or MatchaTTS) without any adaptation, encode the audio with Whisper-base, encode the text with BERT-base, and fuse the pooled representations either early or late into a shared classifier or regressor. The paper argues that the synthetic audio carries prosodic information about cognitive state that is orthogonal to the lexical information in the text.

What would settle it

Run the SAD pipeline on the same seven tasks with the audio replaced by (a) a flat-pitch TTS rendering of the same words, (b) random noise matched in duration, or (c) a second text encoder trained on truncated inputs; if any of these controls reproduces the reported gains, the central claim that TTS prosody supplies the signal is false, and if the same truncation is applied to the text-only baseline, the BoolQ result already tests whether truncation alone is responsible.

Watch

Extended reading notes

Core claim

The paper claims that TTS models internalize prosodic cues tied to cognitive state and that those cues are orthogonal to the lexical information language models exploit. The evidence is a systematic comparison: adding zero-shot TTS audio to BERT improves or matches text-only across seven tasks, with OpenAI TTS giving e.g. GoEmotions F1 from 51.4 to 53.1 and CB-Prosody MAE from 0.693 to 0.668; on SWBD-S sentiment both gold and OpenAI synthetic audio reach MAE 0.334. This supports the conclusion that synthetic audio is a viable substitute for gold audio when recordings are unavailable.

Load-bearing premise

The load-bearing premise is that the performance gain comes from prosodic information about cognitive state that the TTS system baked into the audio, not from the mere fact of adding a second model, extra parameters, or accidental input truncation; the paper's evidence for this is only the pattern of control tasks, and one control task (BoolQ) did improve.

Editorial extensions

If this is right

On the seven non-control tasks, fusing BERT text with zero-shot TTS audio improves over text-only BERT, with OpenAI TTS best in every one of those tasks.
For SWBD-S and CB-Prosody, synthetic OpenAI audio nearly matches gold human audio (0.334 vs 0.334 MAE on SWBD-S; 0.668 vs 0.665 on CB-Prosody), so SAD can substitute for missing recordings there.
On IEMOCAP emotion, the gap is larger: gold audio multimodal F1 is 63.4 versus 59.3 for the best OpenAI synthetic audio, showing emotion tasks are more sensitive to audio quality.
Control tasks WiC and WSC show no improvement, consistent with the claim that TTS carries task-relevant cognitive-state signal; BoolQ is the exception, which the paper attributes to input truncation.
Because no new architecture is needed, SAD is a drop-in data-augmentation method that can benefit from better TTS and multimodal models as they appear.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A direct test of the orthogonality story would compare TTS audio against a same-parameter control that cannot carry prosody, such as flat intonation or shuffled audio chunks; if that control matches SAD, the 'orthogonal prosodic signal' claim would need revision.
The framework suggests a cheap recipe for any text-only cognitive-state dataset: zero-shot TTS plus a fixed audio encoder, potentially extendable to dialogue act, deception, sarcasm, or common-ground prediction, with the caveat that effect sizes may be smaller for tasks that are less prosody-dependent.
As TTS systems become more expressive, the gap between gold and synthetic audio should narrow; tracking this across TTS generations would test the view that the latent cognitive-state signal grows with naturalness.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: the paper does something useful — it tests, across seven cognitive-state tasks, whether zero-shot synthetic TTS audio can substitute for missing human audio in multimodal training — and the top-line result (OpenAI TTS audio plus BERT text beats text-only on all seven) is plausible and worth knowing about. The weaker claim, that this happens because TTS models encode cognitive-state prosody orthogonal to text, is not supported by the experiments as run.

The work is honest and incremental by design. It uses off-the-shelf components, reports no new architecture, and the authors say so. The control tasks are a good idea, and the comparison with gold audio on SWBD-S, CB-Prosody, and IEMOCAP gives a useful reference point. Releasing code and data-generation scripts is a real plus.

The problems are in the evidence and the interpretation. The table reports only averaged metrics. No per-seed or per-fold variances are given, so differences like CB-Prosody MAE 0.693 vs 0.668 could be run-to-run noise. The t-test and binomial test claims are not auditable from the paper. The one concrete control failure — BoolQ improves by about 7 points with audio — is explained away as a truncation artifact. That might be true, but it means the control logic already fails once, and no random-audio or second-text baseline is run to rule out generic ensembling or regularization effects. Those mechanisms could produce the non-control gains without any TTS-specific signal. The Matcha results, which are weaker, make the story more one-TTS-model than a general claim.

The Section 4.4 statement that 'the absence of (unexplained) improvements in the control tasks suggests that TTS models contain latent signals for cognitive states' is too strong given BoolQ. The empirical result itself is fine as a reported finding; the explanation needs more work.

Who should read it: people working on multimodal NLP for emotion, belief, or sentiment, and anyone who wants a cheap way to add an audio modality to text-only corpora. The paper is worth a serious referee — the question is timely and the breadth of tasks is valuable — but a reviewer should ask for variance estimates, control baselines, and a softened interpretation. With those, this could be a solid incremental contribution.

Referee Report

3 major / 4 minor

Summary. The paper proposes SAD (Synthetic Audio Data fine-tuning), a framework that fine-tunes a BERT text encoder and a Whisper audio encoder on text plus zero-shot synthetic speech generated by an off-the-shelf TTS system (OpenAI TTS or Matcha-TTS) and fuses the two modalities. The authors report that the multimodal system improves over text-only BERT on seven cognitive-state tasks spanning sentiment, belief, and emotion, and that on datasets with gold audio the synthetic-audio fusion is competitive with gold-audio fusion. They also report three SuperGLUE control tasks for which they predict no improvement, and they use a binomial test across tasks to claim significance. The paper interprets the results as evidence that TTS models implicitly encode cognitive-state information that is orthogonal to text.

Significance. If the empirical results are robust, SAD is a practical and low-cost method for injecting audio-like signal into text-only cognitive-state datasets, and the RQ1 comparison between synthetic and gold audio is a useful benchmark for the community. The paper ships code, evaluates two TTS systems, and includes a control-task design, all of which are strengths. However, the explanatory claim that TTS models encode latent cognitive-state signal is not isolated by the current experimental design; the practical claim may hold even if the mechanism remains uncertain.

major comments (3)

[Section 3 / Table 1] The experimental design compares text-only BERT with BERT+Whisper fusion, but there is no condition controlling for the presence of a second encoder or for the audio content. A fusion with a second text encoder (e.g., BERT+RoBERTa) or with random, shuffled, or non-cognitive audio would show whether the improvements in Table 1 (e.g., CB MAE 0.785 to 0.741; FactBank F1 74.9 to 76.0) are caused by cognitive-state information in synthetic audio rather than by generic ensembling or regularization. This is load-bearing for the inference in Section 4.4 that TTS models contain latent signals for cognitive states.
[Section 4.4 / Table 1 BoolQ row] BoolQ is a control task on which the authors predict no improvement, yet audio-only Matcha/OpenAI reach 69.7/69.4 accuracy versus 60.0 for text-only, and the multimodal OpenAI variant reaches 67.1. The explanation that Whisper's 30-second truncation 'seems to have helped for some reason' is post hoc and untested. If truncation or extra-modality fusion can produce gains on a control task, the same mechanisms can account for the non-control gains, so the control-task logic does not establish that TTS audio encodes cognitive state. Please run a control condition that holds input length and architecture constant (e.g., truncate the text to the same effective length or add a second-text-encoder fusion) and revise the final sentence of Section 4.4 accordingly.
[Appendix A / Table 1] No per-seed or per-fold variance is reported; Table 1 gives only averages over three seeds or five folds. Several gains are small (SWBD-S MAE 0.339 to 0.334; IMDB accuracy 89.5 to 89.7), and the t-tests mentioned in Section 4.4 are not described with sample sizes, degrees of freedom, or effect sizes. Without this information, the reader cannot determine whether the improvements are outside run-to-run noise, and the binomial test across tasks does not substitute for per-task variance. Please report standard deviations or confidence intervals, per-fold and per-seed results, and paired tests where appropriate.

minor comments (4)

[Section 4.1] 'IMBD' should be 'IMDB'.
[Section 4.4] The sentence 'OpenAI and Matcha seeing a 1.0 and 1.7 point improvement' is reversed relative to Table 1: OpenAI IEMOCAP improves by 2.7 points (56.6 to 59.3), while Matcha improves by 1.0 point (56.6 to 57.6).
[Table 1] The table is very hard to parse because the column header is mangled and the multi-row structure is not visually separated into text-only, audio-only, and multimodal groups; a clean layout would help readers verify the comparisons.
[Section 4.3 / 4.4] The text refers to 'independent two-sample t-tests' for comparisons that appear to be paired across seeds or folds; paired tests would be more appropriate and should be described.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is an empirical benchmark evaluation whose reported gains are measured on held-out splits, not derived from fitted inputs or from a self-citation chain.

full rationale

The paper's central claim is empirical: Table 1 compares a fixed text-only BERT baseline against text-plus-zero-shot-TTS-audio fusion on standard or internally consistent splits, and the reported numbers are measured outcomes, not quantities computed from fitted parameters. No constant or model component is fitted to the test results, so no 'prediction' reduces to an input by construction. The interpretive claim in Section 4.4 that 'TTS models contain latent signals for cognitive states' is a post-hoc explanation of the observed pattern, not a derivation, and therefore not circular in the experimental logic. The paper's self-citations (e.g., Murzaku et al. 2024 for the Whisper encoder choice, multimodal setup, and prior belief-prediction results) are used to justify experimental design and baselines, but they are not load-bearing for the new synthetic-audio comparisons; the SAD framework would stand or fall on the independently measured Table 1 numbers even if those citations were removed. The BoolQ control anomaly and the absence of a noise-audio or second-text-encoder baseline are threats to the causal interpretation and to internal validity, but they are correctness concerns, not circularity. Overall, the paper is self-contained as an empirical evaluation, so the circularity score is 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new entities. Its main hidden costs are modeling and data assumptions: that the observed multimodal gains come from cognitive-state-aware prosody rather than from generic ensembling or noise, and that the downsampled corpora still represent the original tasks.

free parameters (2)

TTS API cost cap per corpus = $10 USD
Determines how many samples survive random downsampling; for example, IMDB is reduced to 372 samples, which directly affects the reliability of the reported improvements.
Fixed training schedule = 10 epochs, lr 2e-5, batch size 1
Chosen once and used for all tasks without tuning; can produce under- or over-trained models on very small corpora, adding noise to the comparisons.

assumptions (4)

domain assumption OpenAI TTS (tts-1-hd, Alloy voice) produces synthetic audio whose prosody reflects the cognitive state of the text
This is the paper's central hypothesis, stated in Section 3, and it is not verified independently of the downstream task results.
domain assumption Whisper-base fine-tuned on synthetic audio retains enough relevant paralinguistic signal for fusion to help
The audio encoder choice is inherited from prior work and is not ablated in this paper.
domain assumption Randomly downsampled corpora remain representative of the full task distribution
Appendix B describes downsampling to a fixed API budget; no evidence is given that the small subsets preserve the original task difficulty.
domain assumption Control tasks (BoolQ, WiC, WSC) do not involve the speaker's cognitive state
Used in Section 4.4 to argue that TTS signal is cognitive-state-specific; the BoolQ result partly contradicts this assumption.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Synthetic Audio Helps for Cognitive State Tasks." pith.science (2026). https://pith.science/paper/3OVQ27SC

@misc{pith2026250206922,
  author       = {Pith},
  title        = {Pith review of: Synthetic Audio Helps for Cognitive State Tasks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3OVQ27SC}},
  note         = {Machine review of arXiv:2502.06922}
}

read the original abstract

The NLP community has broadly focused on text-only approaches of cognitive state tasks, but audio can provide vital missing cues through prosody. We posit that text-to-speech models learn to track aspects of cognitive state in order to produce naturalistic audio, and that the signal audio models implicitly identify is orthogonal to the information that language models exploit. We present Synthetic Audio Data fine-tuning (SAD), a framework where we show that 7 tasks related to cognitive state modeling benefit from multimodal training on both text and zero-shot synthetic audio data from an off-the-shelf TTS system. We show an improvement over the text-only modality when adding synthetic audio data to text-only corpora. Furthermore, on tasks and corpora that do contain gold audio, we show our SAD framework achieves competitive performance with text and synthetic audio compared to text and gold audio.

Figures

Figures reproduced from arXiv: 2502.06922 by the authors.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

40 extracted references · 23 canonical work pages

[1]

Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. 2020. wav2vec 2.0: A framework for self-supervised learning of speech representations. Advances in neural information processing systems, 33:12449--12460

2020
[2]

Carlos Busso, Murtaza Bulut, Chi-Chun Lee, Abe Kazemzadeh, Emily Mower, Samuel Kim, Jeannette N Chang, Sungbok Lee, and Shrikanth S Narayanan. 2008. Iemocap: Interactive emotional dyadic motion capture database. Language resources and evaluation, 42:335--359

2008
[3]

Lea Canales and Patricio Mart \' nez-Barco. 2014. https://doi.org/10.3115/v1/W14-6905 Emotion detection from text: A survey . In Proceedings of the Workshop on Natural Language Processing in the 5th Information Systems Research Working Days ( JISIC ) , pages 37--43, Quito, Ecuador. Association for Computational Linguistics

work page doi:10.3115/v1/w14-6905 2014
[4]

Eric Chen, Zhiyun Lu, Hao Xu, Liangliang Cao, Yu Zhang, and James Fan. 2020 a . https://aclanthology.org/2020.lrec-1.806 A large scale speech sentiment corpus . In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 6549--6555, Marseille, France. European Language Resources Association

work page 2020
[5]

Xi Chen, Sarah Ita Levitan, Michelle Levine, Marko Mandic, and Julia Hirschberg. 2020 b . Acoustic-prosodic and lexical cues to deception and trust: deciphering how people detect lies. Transactions of the Association for Computational Linguistics, 8:199--214

work page 2020
[6]

Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. 2024. Scaling instruction-finetuned language models. Journal of Machine Learning Research, 25(70):1--53

2024
[7]

Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. 2019. https://doi.org/10.18653/v1/N19-1300 B ool Q : Exploring the surprising difficulty of natural yes/no questions . In Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language...

doi:10.18653/v1/n19-1300 2019
[8]

Pierre Colombo, Emile Chapuis, Matteo Manica, Emmanuel Vignon, Giovanna Varni, and Chloe Clavel. 2020. Guiding attention in sequence-to-sequence models for dialogue act prediction. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 7594--7601

work page 2020

Show all 40 references

[9]

Marie-Catherine De Marneffe, Mandy Simons, and Judith Tonhauser. 2019. The commitmentbank: Investigating projection in naturally occurring discourse. In proceedings of Sinn und Bedeutung, volume 23, pages 107--124

2019
[10]

Dorottya Demszky, Dana Movshovitz-Attias, Jeongwoo Ko, Alan Cowen, Gaurav Nemade, and Sujith Ravi. 2020. Goemotions: A dataset of fine-grained emotions. arXiv preprint arXiv:2005.00547

2020 arXiv
[11]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://doi.org/10.18653/v1/N19-1423 BERT : Pre-training of deep bidirectional transformers for language understanding . In Proceedings of the 2019 Conference of the North A merican Chapter of the Associat...

2019 doi
[12]

ElevenLabs. 2023. https://elevenlabs.io/ Elevenlabs

2023
[13]

John J Godfrey, Edward C Holliman, and Jane McDaniel. 1992. Switchboard: Telephone speech corpus for research and development. In Acoustics, speech, and signal processing, ieee international conference on, volume 1, pages 517--520. IEEE Computer Society

1992
[14]

Matthew Honnibal and Ines Montani. 2017. spaCy 2 : Natural language understanding with B loom embeddings, convolutional neural networks and incremental parsing. To appear

2017
[15]

Jehyun Kyung, Serin Heo, and Joon-Hyuk Chang. 2024. https://doi.org/10.21437/Interspeech.2024-2364 Enhancing multimodal emotion recognition through asr error compensation and llm fine-tuning . In Interspeech 2024, pages 4683--4687

2024 doi
[16]

Levesque, Ernest Davis, and Leora Morgenstern

Hector J. Levesque, Ernest Davis, and Leora Morgenstern. 2012. The winograd schema challenge. In Proceedings of the Thirteenth International Conference on Principles of Knowledge Representation and Reasoning, KR'12, page 552–561. AAAI Press

2012
[17]

Sarah I Levitan, Guzhen An, Mandi Wang, Gideon Mendels, Julia Hirschberg, Michelle Levine, and Andrew Rosenberg. 2015. Cross-cultural production and detection of deception from speech. In Proceedings of the 2015 ACM on workshop on multimodal deception detection, pages 1--8

2015
[18]

Sarah Ita Levitan and Julia Hirschberg. 2022. Believe it or not: Acoustic-prosodic cues to trust and mistrust in spoken dialogue. In Speech Prosody, volume 2022, pages 610--614

2022
[19]

Sarah Ita Levitan, Angel Maredia, and Julia Hirschberg. 2018. https://doi.org/10.21437/Interspeech.2018-2443 Acoustic-Prosodic Indicators of Deception and Trust in Interview Dialogues . In Proc. Interspeech 2018, pages 416--420

2018 doi
[20]

Yinhan Liu. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692

2019 arXiv
[21]

Maas, Raymond E

Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011. https://aclanthology.org/P11-1015 Learning word vectors for sentiment analysis . In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Hu...

2011
[22]

Taylor Mahler, Marie-Catherine de Marneffe, and Catherine Lai. 2020. The prosody of presupposition projection in naturally-occurring utterances. In Proceedings of Sinn und Bedeutung, volume 24, pages 20--37

2020
[23]

Magdalena Markowska, Mohammad Taghizadeh, Adil Soubki, Seyed Mirroshandel, and Owen Rambow. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.551 Finding common ground: Annotating and predicting common ground in spoken conversations . In Findings of the Association for Com...

2023 doi
[24]

Shivam Mehta, Ruibo Tu, Jonas Beskow, \'E va Sz \'e kely, and Gustav Eje Henter. 2024. Matcha- TTS : A fast TTS architecture with conditional flow matching. In Proc. ICASSP

2024
[25]

Gideon Mendels, Sarah Ita Levitan, Kai-Zhan Lee, and Julia Hirschberg. 2017. Hybrid acoustic-lexical deep learning approach for deception detection. In Interspeech

2017
[26]

John Murzaku, Adil Soubki, and Owen Rambow. 2024. https://doi.org/10.21437/Interspeech.2024-2103 Multimodal belief prediction . In Interspeech 2024, pages 1075--1079

2024 doi
[27]

John Murzaku, Peter Zeng, Magdalena Markowska, and Owen Rambow. 2022. https://aclanthology.org/2022.coling-1.66 Re-examining F act B ank: Predicting the author ' s presentation of factuality . In Proceedings of the 29th International Conference on Computational Linguistics, pa...

2022
[28]

Behnaz Nojavanasghari, Deepak Gopinath, Jayanth Koushik, Tadas Baltru s aitis, and Louis-Philippe Morency. 2016. Deep multimodal fusion for persuasiveness prediction. In Proceedings of the 18th ACM International Conference on Multimodal Interaction, pages 284--288

2016
[29]

OpenAI. 2023. https://platform.openai.com/docs/models/tts Text-to-speech

2023
[30]

Leonardo Pepino, Pablo Riera, and Luciana Ferrer. 2021. Emotion recognition from speech using wav2vec 2.0 embeddings. arXiv preprint arXiv:2104.03502

2021 arXiv
[31]

Mohammad Taher Pilehvar and Jose Camacho-Collados. 2019. https://doi.org/10.18653/v1/N19-1128 W i C : the word-in-context dataset for evaluating context-sensitive meaning representations . In Proceedings of the 2019 Conference of the North A merican Chapter of the Association ...

2019 doi
[32]

Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. Robust speech recognition via large-scale weak supervision. In International Conference on Machine Learning, pages 28492--28518. PMLR

2023
[33]

Roser Saur \' and James Pustejovsky. 2009. Factbank: a corpus annotated with event factuality. Language resources and evaluation, 43:227--268

2009
[34]

Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2019. Superglue: a stickier benchmark for general-purpose language understanding systems. In Proceedings of the 33rd International Conference on Neural I...

2019
[35]

Janyce M. Wiebe. 1990. https://aclanthology.org/C90-2069 Identifying subjective characters in narrative . In COLING 1990 Volume 2: Papers presented to the 13th International Conference on Computational Linguistics

1990
[36]

Justine Zhang, Jonathan Chang, Cristian Danescu-Niculescu-Mizil, Lucas Dixon, Yiqing Hua, Dario Taraborelli, and Nithum Thain. 2018. https://doi.org/10.18653/v1/P18-1125 Conversations gone awry: Detecting early signs of conversational failure . In Proceedings of the 56th Annua...

2018 doi
[37]

Zihan Zhao, Yanfeng Wang, and Yu Wang. 2022. https://doi.org/10.21437/Interspeech.2022-10230 Multi-level fusion of wav2vec 2.0 and bert for multimodal emotion recognition . In Interspeech 2022, pages 4725--4729

2022 doi
[38]

Ziping Zhao, Tian Gao, Haishuai Wang, and Björn Schuller. 2024. https://doi.org/10.21437/Interspeech.2024-1735 Mfdr: Multiple-stage fusion and dynamically refined network for multimodal emotion recognition . In Interspeech 2024, pages 3719--3723

2024 doi
[39]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...
[40]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.

[1] [1]

Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. 2020. wav2vec 2.0: A framework for self-supervised learning of speech representations. Advances in neural information processing systems, 33:12449--12460

2020

[2] [2]

Carlos Busso, Murtaza Bulut, Chi-Chun Lee, Abe Kazemzadeh, Emily Mower, Samuel Kim, Jeannette N Chang, Sungbok Lee, and Shrikanth S Narayanan. 2008. Iemocap: Interactive emotional dyadic motion capture database. Language resources and evaluation, 42:335--359

2008

[3] [3]

Lea Canales and Patricio Mart \' nez-Barco. 2014. https://doi.org/10.3115/v1/W14-6905 Emotion detection from text: A survey . In Proceedings of the Workshop on Natural Language Processing in the 5th Information Systems Research Working Days ( JISIC ) , pages 37--43, Quito, Ecuador. Association for Computational Linguistics

work page doi:10.3115/v1/w14-6905 2014

[4] [4]

Eric Chen, Zhiyun Lu, Hao Xu, Liangliang Cao, Yu Zhang, and James Fan. 2020 a . https://aclanthology.org/2020.lrec-1.806 A large scale speech sentiment corpus . In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 6549--6555, Marseille, France. European Language Resources Association

work page 2020

[5] [5]

Xi Chen, Sarah Ita Levitan, Michelle Levine, Marko Mandic, and Julia Hirschberg. 2020 b . Acoustic-prosodic and lexical cues to deception and trust: deciphering how people detect lies. Transactions of the Association for Computational Linguistics, 8:199--214

work page 2020

[6] [6]

Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. 2024. Scaling instruction-finetuned language models. Journal of Machine Learning Research, 25(70):1--53

2024

[7] [7]

Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. 2019. https://doi.org/10.18653/v1/N19-1300 B ool Q : Exploring the surprising difficulty of natural yes/no questions . In Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language...

doi:10.18653/v1/n19-1300 2019

[8] [8]

Pierre Colombo, Emile Chapuis, Matteo Manica, Emmanuel Vignon, Giovanna Varni, and Chloe Clavel. 2020. Guiding attention in sequence-to-sequence models for dialogue act prediction. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 7594--7601

work page 2020