Pith. sign in

REVIEW 4 major objections 5 minor 184 references

Emotion Recognition and Generation: A Comprehensive Review of Face, Speech, and Text Modalities

T0 review · 4 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper argues that emotion recognition and emotion generation belong in one holistic survey across face, speech, and text, and delivers that survey with datasets, methods, evaluation metrics, and future directions.

desk verdict Useful survey concept, but the comparative tables are too unreliable to cite as-is; fixable with a careful re-verification pass. read the letter →

arxiv 2502.06803 v1 pith:DJ4E4OCW submitted 2025-02-02 cs.LG cs.AIcs.CV

Rebecca Mobbs , Dimitrios Makris , Vasileios Argyriou This is my paper · ORCID
classification cs.LGcs.AIcs.CV
keywords emotionrecognitiongenerationfacialexpressionspeechtextsentimenttalking-headvoiceconversionaffectivecomputing
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Emotion recognition and emotion generation have usually been reviewed as separate topics, each limited to one technical approach or one modality. This paper contends that the field needs a single holistic survey and then provides one, covering facial, vocal, and textual modalities for both analysis and synthesis. It walks a newcomer from preprocessing and datasets through state-of-the-art methods, evaluation metrics, comparative tables, challenges, and future directions. If the paper is right, a researcher beginning in affective AI can start from one place and see how recognition and generation share structure across modalities.

What carries the argument

The organising device is a two-axis taxonomy: modality (face, speech, text) crossed with task (recognition versus generation), with a separate section for emotion-control methods within generation. Within each cell, the survey groups methods by technical family, such as attention-based, transformer-based, GAN-based, and diffusion-based, plus LLMs for text, and anchors everything to the eight basic emotions derived from Ekman's model. The comparative tables then translate heterogeneous papers into common metrics so that different approaches can be positioned against one another.

What would settle it

Locating a published survey that already covers both emotion recognition and emotion generation across face, speech, and text with comparable scope would disprove the paper's gap claim; alternatively, spot-checking every entry in the comparative tables against its source paper and finding systematic misreporting would undermine the reliability claim.

Watch

Extended reading notes

Core claim

The central claim is that no existing review integrates emotion recognition with emotion generation, and that a review which does so is a useful map for newcomers. On the paper's own terms, it provides that map: it categorises recent state-of-the-art research by technical approach, explains the theoretical foundations of each approach, and compares methods on common metrics such as accuracy, F1 score, FID, WER, and perplexity. It also identifies shared limitations, including scarce and biased datasets, inconsistent evaluation, difficulty of real-time and subtle emotion generation, and ethical risks, and it proposes future directions such as multimodal integration, standardised benchmarks, and responsible deployment.

Load-bearing premise

The survey's value as a starting point assumes that the numbers in its comparative tables and the attribution of methods to cited papers faithfully reproduce what the cited papers actually report.

Editorial extensions

If this is right

  • A newcomer can use the survey as a single entry point covering both recognition and generation across all three modalities.
  • The comparative tables give baseline expectations for performance on widely used datasets such as AffectNet, RAF-DB, IEMOCAP, and LibriSpeech.
  • The taxonomy shows which technical families dominate each task and where gaps remain, notably text-driven facial expression generation.
  • The survey's stated challenges define a concrete research agenda: larger diverse in-the-wild datasets, standardised metrics, real-time emotion control, and ethical safeguards.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • In the editor's reading, the comparative tables are best treated as a guide to typical operating points rather than a strict leaderboard, since the collected results come from different evaluation protocols.
  • A testable consequence of the survey's structure is that multimodal systems sharing representations across recognition and generation will outperform single-modality pipelines, which the paper names as a future direction but does not itself prove.
  • One extension a reader could pursue is to build a unified benchmark that scores recognition and generation jointly on the same emotional episodes, directly addressing the standardisation gap the paper identifies.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper is a survey of emotion recognition and generation across face, speech, and text modalities. It covers preprocessing techniques, datasets, state-of-the-art methods for recognition and generation, evaluation metrics, comparative analyses, and future research directions. The stated goal is to provide a holistic, integrated review that helps researchers beginning in the field, addressing what the authors describe as a gap in the literature covering these two domains together.

Significance. If the survey were accurate, it would fill a useful niche: most prior reviews treat emotion recognition and generation separately, and a single structured overview spanning modalities would be valuable to newcomers. The paper has broad coverage of datasets, methods, and metrics, and it explicitly discusses applications and open challenges. However, the survey's usefulness depends critically on the correctness of its comparative tables and their interpretation; the reported numbers and citations currently contain several internal inconsistencies that undermine the central claim of providing reliable comparative guidance. The paper does not include code or machine-checked proofs, but that is not expected for a survey; the burden instead lies on accurate reporting of cited work.

major comments (4)
  1. [§6.2.2, Table 3] The prose states that SadTalker achieves a lower ACC (10.31) than Wav2Lip's 9.38, which is numerically incorrect since 10.31 > 9.38. More generally, the ACC column mixes values from different evaluation protocols (0.8, 9.38, 58.8, 75.43) without any explanation of what is being measured, so the table cannot support the comparative conclusions drawn from it. This directly affects the review's stated goal of offering comparative analyses.
  2. [§6.2.5, Table 6] The text credits Emotion BERT [176] with the highest F1 score of 0.88 on EmotionLines, but Table 6 reports only ACC=0.71 for that model and no F1 score. The same paragraph attributes an F1 score of 0.47 and accuracy of 0.5 to AutoVC on the ESD dataset, but AutoVC and ESD appear in Table 5 (speech generation), not in Table 6 (text sentiment recognition). These mismatches make the claimed SOTA results unverifiable from the review itself.
  3. [§5.4.2] The text-driven talking-head framework attributed to reference [114] is described in detail (components Gmou, Gupp, Ghed, Gldmk), but reference [114] is StarGANv2-VC, a voice-conversion paper. The described method is therefore left without a correct citation, undermining the reliability of the method categorization for readers trying to locate the original work.
  4. [§6.2.4, Table 5] The prose states that FreeVC achieves the lowest WER (5.4%) and EER (11.28%) on LibriSpeech, but Table 5 lists FreeVC with EER 35.63 and Phoneme Hallucinator with a lower WER (5.1). This is another instance where the textual interpretation contradicts the table data, making the comparative discussion internally inconsistent.
minor comments (5)
  1. [§1] The phrase 'emotion control methods accross modalities' contains a typo: 'accross' should be 'across'.
  2. [§6.1.3] The metric list refers to 'GPQU' but the correct abbreviation is 'GPQA'; also, the statement that 'All of these metrics are obtained from user studies' is inaccurate for MMLU, MATH, HumanEval, MGSM, and DROP, which are benchmark evaluations rather than subjective user studies.
  3. [§6.2.6, Table 7] The prose says 'The GPT-4 model [28] achieves the highest MMLU score of 88.7%', but Table 7 attributes 88.7 to GPT-4o, not GPT-4; the model name should be corrected for consistency.
  4. [References] Reference [150] is titled 'Makelttalk: speaker-aware talking-head animation' in the bibliography; the correct title is 'MakeItTalk: Speaker-Aware Talking-Head Animation'.
  5. [Table 1] The FERPlus dataset size is listed as 'Unlimited', which is undefined and not informative; the actual number of images in FER2013/FERPlus should be reported instead.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the survey summarizes and compares external results; it derives no predictions from fitted inputs.

full rationale

This paper is a literature review and survey. It does not fit parameters to data and then predict related quantities, nor does it invoke a self-citation chain as a load-bearing derivation. The central claim—that the review offers a holistic, integrated overview of emotion recognition and generation across face, speech, and text modalities—rests on the selection and transcription of external published results, not on any circular derivation. The comparative tables and prose report metrics from cited papers (e.g., Table 3, Table 4, Table 6), and while some of those transcriptions appear internally inconsistent or incorrect (e.g., the SadTalker 'lower ACC' statement in Section 6.2.2 or the Emotion BERT F1 claim in Section 6.2.5), such accuracy issues concern factual correctness and quality of the survey, not circular reasoning. No equation or claim in the paper reduces by construction to its own inputs, and no fitted input is renamed as a prediction. The paper also does not appeal to a 'uniqueness theorem' or to the authors' prior work to force a conclusion. Therefore, the appropriate finding is no significant circularity.

Assumptions & free parameters 0 free parameters · 0 assumptions · 0 invented entities

This is a literature review. It introduces no original equations, fitted parameters, or postulated entities. The only 'inputs' are the cited papers themselves.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Emotion Recognition and Generation: A Comprehensive Review of Face, Speech, and Text Modalities." pith.science (2026). https://pith.science/paper/DJ4E4OCW

@misc{pith2026250206803,
  author       = {Pith},
  title        = {Pith review of: Emotion Recognition and Generation: A Comprehensive Review of Face, Speech, and Text Modalities},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DJ4E4OCW}},
  note         = {Machine review of arXiv:2502.06803}
}
read the original abstract

Emotion recognition and generation have emerged as crucial topics in Artificial Intelligence research, playing a significant role in enhancing human-computer interaction within healthcare, customer service, and other fields. Although several reviews have been conducted on emotion recognition and generation as separate entities, many of these works are either fragmented or limited to specific methodologies, lacking a comprehensive overview of recent developments and trends across different modalities. In this survey, we provide a holistic review aimed at researchers beginning their exploration in emotion recognition and generation. We introduce the fundamental principles underlying emotion recognition and generation across facial, vocal, and textual modalities. This work categorises recent state-of-the-art research into distinct technical approaches and explains the theoretical foundations and motivations behind these methodologies, offering a clearer understanding of their application. Moreover, we discuss evaluation metrics, comparative analyses, and current limitations, shedding light on the challenges faced by researchers in the field. Finally, we propose future research directions to address these challenges and encourage further exploration into developing robust, effective, and ethically responsible emotion recognition and generation systems.

Figures

Figures reproduced from arXiv: 2502.06803 by the authors.

Figure 1
Figure 1. The EmoFAN pipeline integrates facial landmark detection, discrete emotion classification, and continuous [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. The SER model processes frame-level speech features as input, using a 2-layer LSTM to generate outputs [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. The TER system by[86] uses a BERT-based dual-channel pipeline for text emotion recognition. First, input sentences are converted into contextual embeddings with a pre-trained BERT model. These embeddings are then processed through two parallel channels: one uses CNN for feature extraction followed by BiLSTM for capturing sequence information, while the other uses BiLSTM first, followed by CNN. The outputs from both … view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: In the place of 3D modelling, EMO utilising Stable Diffusion for generating new frames. The pipeline [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: The PromptVC pipeline uses a latent diffusion model for voice style conversion using natural language [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: The LLaMA pipeline involves pre-training transformer-based models on large textual datasets, followed [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

184 extracted references · 54 canonical work pages

  1. [114]

    Starganv2-vc: A diverse, unsupervised, non-parallel framework for natural-sounding voice conversion

    Yinghao Aaron Li, Ali Zare, and Nima Mesgarani. Starganv2-vc: A diverse, unsupervised, non-parallel framework for natural-sounding voice conversion. Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, 6:4770–4774, 2021

    2021

  2. [176]

    EmotionX-IDEA: Emotion BERT -- an Affectional Model for Conversation

    Yen-Hao Huang, Ssu-Rui Lee, Mau-Yun Ma, Yi-Hsin Chen, Ya-Wen Yu, and Yi-Shin Chen. Emotionx-idea: Emotion bert–an affectional model for conversation. arXiv preprint arXiv:1908.06264, 2019

    work page Pith review arXiv 1908

  3. [1]

    Recognizing action units for facial expression analysis

    Y-I Tian, Takeo Kanade, and Jeffrey F Cohn. Recognizing action units for facial expression analysis. IEEE Transactions on pattern analysis and machine intelligence, 23(2):97–115, 2001

    2001

  4. [2]

    Face validity

    Ellen Johnson. Face validity. In Encyclopedia of autism spectrum disorders, pages 1957–1957. Springer, 2021

    1957

  5. [3]

    The expression of the emotions in man and animals

    C Darwin and P Prodger. The expression of the emotions in man and animals. oxford university press, usa. 1998

    1998

  6. [4]

    Constants across cultures in the face and emotion

    Paul Ekman and Wallace V Friesen. Constants across cultures in the face and emotion. Journal of personality and social psychology, 17(2):124, 1971

    1971

  7. [5]

    Strong evidence for universals in facial expressions: A reply to russell’s mistaken critique

    Paul Ekman. Strong evidence for universals in facial expressions: A reply to russell’s mistaken critique. 1994

    1994

  8. [6]

    Einfochips

    Einfochips. Einfochips; . https://www.einfochips.com/, 2024. Accessed: February 26, 2024

    2024

Show all 184 references
  1. [7]

    Elevate AI

    Elevate Ai. Elevate AI. https://www.elevateai.com, 2024. Accessed: February 26, 2024

    2024

  2. [8]

    Facial expression recognition of emotional situations in mild and moderate alzheimer’s disease

    Michelle Brandt, Felipe de Oliveira Silva, José Pedro Simões Neto, Maria Alice Tourinho Baptista, Tatiana Belfort, Isabel Barbeito Lacerda, and Marcia Cristina Nascimento Dourado. Facial expression recognition of emotional situations in mild and moderate alzheimer’s disease. J...

    2024

  3. [9]

    Mental states and personality based on real-time physical activity and facial expression recognition

    Yating Huang, Dengyue Zhai, Jingze Song, Xuanheng Rao, Xiao Sun, and Jin Tang. Mental states and personality based on real-time physical activity and facial expression recognition. Frontiers in Psychiatry, 13:1019043, 2023

    2023

  4. [10]

    Using a social robot to evaluate facial expressions in the wild

    Silvia Ramis, Jose Maria Buades, and Francisco J Perales. Using a social robot to evaluate facial expressions in the wild. Sensors, 20(23):6716, 2020

    2020

  5. [11]

    Classroom teaching evaluation based on facial expression recognition

    Xiao-Yu Tang, Wang-Yue Peng, Si-Rui Liu, and Jian-Wen Xiong. Classroom teaching evaluation based on facial expression recognition. In Proceedings of the 2020 9th International Conference on Educational and Information Technology, pages 62–67, 2020

    2020

  6. [12]

    Driver fatigue detection based on deeply-learned facial expression representation

    Zhongmin Liu, Yuxi Peng, and Wenjin Hu. Driver fatigue detection based on deeply-learned facial expression representation. Journal of Visual Communication and Image Representation, 71:102723, 2020

    2020

  7. [13]

    An accurate facial expression detector using multi- landmarks selection and local transform features

    Syeda Amna Rizwan, Ahmad Jalal, and Kibum Kim. An accurate facial expression detector using multi- landmarks selection and local transform features. In 2020 3rd International conference on advancements in computational sciences (ICACS), pages 1–6. IEEE, 2020

    2020

  8. [14]

    Societies becoming the same: Visual representation of the individual via the faceapp: Application

    Hilal Sansar. Societies becoming the same: Visual representation of the individual via the faceapp: Application. In International Symposium on Intelligent Manufacturing and Service Systems, pages 10–14. Springer, 2023

    2023

  9. [15]

    Is there an app for that? a review of popular apps for depression, anxiety, and well-being

    Akash R Wasil, Emma H Palermo, Lorenzo Lorenzo-Luaces, and Robert J DeRubeis. Is there an app for that? a review of popular apps for depression, anxiety, and well-being. Cognitive and Behavioral Practice, 29(4):883–901, 2022

    2022

  10. [16]

    Evaluating the therapeutic alliance with a free-text cbt conversational agent (wysa): a mixed-methods study

    Clare Beatty, Tanya Malik, Saha Meheli, and Chaitali Sinha. Evaluating the therapeutic alliance with a free-text cbt conversational agent (wysa): a mixed-methods study. Frontiers in Digital Health, 4:847991, 2022

    2022

  11. [17]

    Rashmi Adyapady and B

    R. Rashmi Adyapady and B. Annappa. A comprehensive review of facial expression recognition techniques. Multimedia Systems, 29:73–103, 2 2023

    2023

  12. [18]

    A survey of textual emotion recognition and its challenges

    Jiawen Deng and Fuji Ren. A survey of textual emotion recognition and its challenges. IEEE Transactions on Affective Computing, 14:49–67, 1 2023

    2023

  13. [19]

    Speech emotion recognition: a comprehensive survey

    Mohammed Jawad Al-Dujaili and Abbas Ebrahimi-Moghadam. Speech emotion recognition: a comprehensive survey. Wireless Personal Communications, 129(4):2525–2561, 2023. 20 Emotion Recognition and Generation: A Comprehensive Review of Face, Speech, and Text ModalitiesA PREPRINT

    2023

  14. [20]

    A comprehensive survey of ai-generated content (aigc): A history of generative ai from gan to chatgpt

    Yihan Cao, Siyu Li, Yixin Liu, Zhiling Yan, Yutong Dai, Philip S Yu, and Lichao Sun. A comprehensive survey of ai-generated content (aigc): A history of generative ai from gan to chatgpt. arXiv preprint arXiv:2303.04226, 2023

    2023 arXiv

  15. [21]

    A comprehensive survey and analysis of generative models in machine learning

    GM Harshvardhan, Mahendra Kumar Gourisaria, Manjusha Pandey, and Siddharth Swarup Rautaray. A comprehensive survey and analysis of generative models in machine learning. Computer Science Review , 38:100285, 2020

    2020

  16. [22]

    Deep facial expression recognition: A survey

    Shan Li and Weihong Deng. Deep facial expression recognition: A survey. 4 2018

    2018

  17. [23]

    Estimation of continuous valence and arousal levels from faces in naturalistic conditions

    Antoine Toisoul, Jean Kossaifi, Adrian Bulat, Georgios Tzimiropoulos, and Maja Pantic. Estimation of continuous valence and arousal levels from faces in naturalistic conditions. Nature Machine Intelligence, 3(1):42–50, 2021

    2021

  18. [24]

    Training and profiling a pediatric facial expression classifier for children on mobile devices: machine learning study

    Agnik Banerjee, Onur Cezmi Mutlu, Aaron Kline, Saimourya Surabhi, Peter Washington, and Dennis Paul Wall. Training and profiling a pediatric facial expression classifier for children on mobile devices: machine learning study. JMIR formative research, 7:e39917, 2023

    2023

  19. [25]

    Effects of digital avatar on perceived social presence and co-presence in business meetings between the managers and their co-workers

    Mika Yasuoka, Marko Zivko, Hiroshi Ishiguro, Yuichiro Yoshikawa, and Kazuki Sakai. Effects of digital avatar on perceived social presence and co-presence in business meetings between the managers and their co-workers. In International Conference on Collaboration Technologies a...

    2022

  20. [26]

    i’m here for you

    Marloes MC van Wezel, Emmelyn AJ Croes, and Marjolijn L Antheunis. “i’m here for you”: Can social chatbots truly support their users? a literature review. In Chatbot Research and Design: 4th International Workshop, CONVERSATIONS 2020, Virtual Event, November 23–24, 2020, Revis...

    2020

  21. [27]

    CharacterAI

    CharacterAI. CharacterAI. https://www.character.ai/, 2024. Accessed: July 13, 2024

    2024

  22. [28]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    2023 arXiv

  23. [29]

    Meta. Llama3. https://www.ai.meta.com/blog/meta-llama-3/ , 2024. Accessed: July 13, 2024

    2024

  24. [30]

    Gemini: a family of highly capable multimodal models

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023

    2023 arXiv

  25. [31]

    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems, pages 1097–1105, 2012

    2012

  26. [32]

    Gradient-based learning applied to document recognition

    Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998

    1998

  27. [33]

    Gonzalez and Richard E

    Rafael C. Gonzalez and Richard E. Woods. Digital Image Processing. Prentice Hall, 2002

    2002

  28. [34]

    John W. Tukey. Exploratory Data Analysis. Addison-Wesley, 1977

    1977

  29. [35]

    Pizer, E

    Stephen M. Pizer, E. Philip Amburn, John D. Austin, Robert Cromartie, Alan Geselowitz, Trey Greer, Bartter Ter Haar Romeny, John B. Zimmerman, and Karel Zuiderveld. Adaptive histogram equalization and its variations. Computer Vision, Graphics, and Image Processing, 39(3):355–368, 1987

    1987

  30. [36]

    Khoshgoftaar

    Connor Shorten and Taghi M. Khoshgoftaar. A survey on image data augmentation for deep learning. Journal of Big Data, 6(1):60, 2019

    2019

  31. [37]

    Rapid object detection using a boosted cascade of simple features

    Paul Viola and Michael Jones. Rapid object detection using a boosted cascade of simple features. In Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, volume 1, pages I–I, 2001

    2001

  32. [38]

    Facial landmark detection by deep multi-task learning

    Zhifeng Zhang, Ping Luo, Chen Change Loy, and Xiaoou Tang. Facial landmark detection by deep multi-task learning. In European Conference on Computer Vision, pages 94–108. Springer, Cham, 2014

    2014

  33. [39]

    Very deep convolutional networks for large-scale image recognition

    Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014

    2014 arXiv

  34. [40]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016

    2016

  35. [41]

    Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam

    Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861, 2017

    2017 arXiv

  36. [42]

    Bovik, Hamid R

    Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600–612, 2004. 21 Emotion Recognition and Generation: A Comprehensive Review of Face, Speec...

    2004

  37. [43]

    Steven F. Boll. Suppression of acoustic noise in speech using spectral subtraction. IEEE Transactions on Acoustics, Speech, and Signal Processing, 27(2):113–120, 1979

    1979

  38. [44]

    Lim and Alan V

    Jae S. Lim and Alan V . Oppenheim. Enhancement and bandwidth compression of noisy speech.Proceedings of the IEEE, 67(12):1586–1604, 1979

    1979

  39. [45]

    Bernard Widrow and Samuel D. Stearns. Adaptive Signal Processing. Prentice-Hall, 1985

    1985

  40. [46]

    Fundamentals of Speech Recognition

    Lawrence Rabiner and Biing-Hwang Juang. Fundamentals of Speech Recognition. Prentice-Hall, 1993

    1993

  41. [47]

    Rabiner and Myron R

    Lawrence R. Rabiner and Myron R. Sambur. An algorithm for determining the endpoints of isolated utterances. The Bell System Technical Journal, 54(2):297–315, 1975

    1975

  42. [48]

    Sadjadi and John H

    Shahin O. Sadjadi and John H. Hansen. Unsupervised speech activity detection using voicing measures and perceptual spectral flux. IEEE Signal Processing Letters, 20(3):197–200, 2013

    2013

  43. [49]

    Davis and Paul Mermelstein

    Steven B. Davis and Paul Mermelstein. Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences. IEEE Transactions on Acoustics, Speech, and Signal Processing, 28(4):357–366, 1980

    1980

  44. [50]

    Linear prediction: A tutorial review

    John Makhoul. Linear prediction: A tutorial review. Proceedings of the IEEE, 63(4):561–580, 1975

    1975

  45. [51]

    Accurate short-term analysis of the fundamental frequency and the harmonics-to-noise ratio of a sampled sound

    Paul Boersma. Accurate short-term analysis of the fundamental frequency and the harmonics-to-noise ratio of a sampled sound. Proceedings of the Institute of Phonetic Sciences, 17(1193):97–110, 1993

    1993

  46. [52]

    Crochiere and Lawrence R

    Ronald E. Crochiere and Lawrence R. Rabiner. Multirate Digital Signal Processing. Prentice-Hall, 1983

    1983

  47. [53]

    Speech dereverberation

    Patrick A Naylor and Nikolay D Gaubitch. Speech dereverberation. Springer Science & Business Media, 2010

    2010

  48. [54]

    Speech enhancement using vector quantization and a formant distance measure

    Douglas O’Shaughnessy. Speech enhancement using vector quantization and a formant distance measure. In ICASSP-88., International Conference on Acoustics, Speech, and Signal Processing , pages 549–550. IEEE Computer Society, 1988

    1988

  49. [55]

    Frame blocking and windowing speech signal

    Oday Kamil Hamid. Frame blocking and windowing speech signal. Journal of Information, Communication, and Intelligence Systems (JICIS), 4(5):87–94, 2018

    2018

  50. [56]

    A recursive feature vector normalization approach for robust speech recognition in noise

    Olli Viikki, David Bye, and Kari Laurila. A recursive feature vector normalization approach for robust speech recognition in noise. In Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP’98 (Cat. No. 98CH36181), volume 2, pa...

    1998

  51. [57]

    Efficient estimation of word representations in vector space

    Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781, 2013

    2013 arXiv

  52. [58]

    Manning, Prabhakar Raghavan, and Hinrich Schütze

    Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schütze. Introduction to Information Retrieval. Cambridge University Press, 2008

    2008

  53. [59]

    Martin F. Porter. An algorithm for suffix stripping. Program, 14(3):130–137, 1980

    1980

  54. [60]

    Natural Language Processing with Python

    Steven Bird, Ewan Klein, and Edward Loper. Natural Language Processing with Python. O’Reilly Media, 2009

    2009

  55. [61]

    Ghosh and D

    S. Ghosh and D. L. Reilly. Credit card fraud detection with a neural-network. In Proceedings of the 27th Annual Hawaii International Conference on System Sciences, volume 3, pages 621–630, 1994

    1994

  56. [62]

    Jeffrey Pennington, Richard Socher, and Christopher D. Manning. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543, 2014

    2014

  57. [63]

    Neural Network Methods for Natural Language Processing

    Yoav Goldberg. Neural Network Methods for Natural Language Processing. Morgan & Claypool Publishers, 2017

    2017

  58. [64]

    Corrado, and Jeffrey Dean

    Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S. Corrado, and Jeffrey Dean. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems, pages 3111–3119, 2013

    2013

  59. [65]

    Deep pyramid convolutional neural networks for text categorization

    Rie Johnson and Tong Zhang. Deep pyramid convolutional neural networks for text categorization. InProceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 562–570, 2017

    2017

  60. [66]

    Eda: Easy data augmentation techniques for boosting performance on text classification tasks

    Jason Wei and Kai Zou. Eda: Easy data augmentation techniques for boosting performance on text classification tasks. arXiv preprint arXiv:1901.11196, 2019

    1901 arXiv

  61. [67]

    The interspeech 2010 paralinguistic challenge

    Björn Schuller, Stefan Steidl, Andreas Batliner, Felix Burkhardt, Laurence Devillers, Christian Müller, and Shrikanth Narayanan. The interspeech 2010 paralinguistic challenge. In Proceedings of INTERSPEECH 2010, 2010. 22 Emotion Recognition and Generation: A Comprehensive Revi...

    2010

  62. [68]

    Deep learning

    Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. Nature, 521(7553):436–444, 2015

    2015

  63. [69]

    Littlewort, J

    G. Littlewort, J. Whitehill, T. Wu, I. Fasel, M. Frank, J. Movellan, and M. Bartlett. The computer expression recognition toolbox (cert). In Proceedings of the IEEE International Conference on Automatic Face & Gesture Recognition and Workshops (FG), pages 298–305, 2011

    2011

  64. [70]

    Ziheng Zhang and Michael J. Lyons. Multi-modal face and audio-visual emotion recognition in development. In Proceedings of the IEEE International Conference on Automatic Face & Gesture Recognition (FG), pages 806–811, 2011

    2011

  65. [71]

    M. S. Bartlett, G. Littlewort, M. G. Frank, C. Lainscsek, I. Fasel, and J. R. Movellan. Recognizing facial expression: Machine learning and application to spontaneous behavior. Neural Networks, 18(5-6):547–557, 2006

    2006

  66. [72]

    Challenges in representation learning: A report on three machine learning contests

    Ian J Goodfellow, Dumitru Erhan, Pierre Luc Carrier, Aaron Courville, Mehdi Mirza, Ben Hamner, Will Cukierski, Yichuan Tang, David Thaler, Dong-Hyun Lee, et al. Challenges in representation learning: A report on three machine learning contests. In International Conference on N...

    2013

  67. [73]

    Emoca: Emotion driven monocular face capture and animation

    Radek Danecek, Michael Black, and Timo Bolkart. Emoca: Emotion driven monocular face capture and animation. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2022-June:20279–20290, 2022

    2022

  68. [74]

    Abaw: Valence-arousal estimation, expression recognition, action unit detection & emotional reaction intensity estimation challenges, 2023

    Dimitrios Kollias, Panagiotis Tzirakis, Hume Ai, Alice Baird Hume, Usa Alice@hume Ai, Alan Cowen, Usa Alan@hume Ai, and Stefanos Zafeiriou. Abaw: Valence-arousal estimation, expression recognition, action unit detection & emotional reaction intensity estimation challenges, 2023

    2023

  69. [75]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. arXiv preprint arXiv...

    2021 arXiv

  70. [76]

    Prompting visual-language models for dynamic facial expression recognition

    Zengqun Zhao and Ioannis Patras. Prompting visual-language models for dynamic facial expression recognition. arXiv preprint arXiv:2308.13382, 2023

    2023 arXiv

  71. [77]

    Patch attention convolutional vision transformer for facial expression recognition with occlusion

    Chang Liu, Kaoru Hirota, and Yaping Dai. Patch attention convolutional vision transformer for facial expression recognition with occlusion. Information Sciences, 619:781–794, 1 2023

    2023

  72. [78]

    Poster++: A simpler and stronger facial expression recognition network

    Jiawei Mao, Rui Xu, Xuesong Yin, Yuanqi Chang, Binling Nie, and Aibin Huang. Poster++: A simpler and stronger facial expression recognition network. 1 2023

    2023

  73. [79]

    Enhanced spatial-temporal learning network for dynamic facial expression recognition

    Weijun Gong, Yurong Qian, Weihang Zhou, and Hongyong Leng. Enhanced spatial-temporal learning network for dynamic facial expression recognition. Biomedical Signal Processing and Control, 88:105316, 2024

    2024

  74. [80]

    Towards accurate marker-less 3D facial performance capture

    Ayush Tewari, Michael Zollhöfer, Justus Thies, Pablo Garrido, Florian Bernard, Derek Bradley, Thabo Beeler, Patrick Perez, and Christian Theobalt. Towards accurate marker-less 3D facial performance capture. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patte...

    2020

  75. [81]

    Speech emotion classification using attention-based lstm, 2023

    Yue Xie, Ruiyu Liang, Zhenlin Liang, Chengwei Huang, Cairong Zou, and Björn Schuller. Speech emotion classification using attention-based lstm, 2023

    2023

  76. [82]

    Hidden markov model-based speech emotion recognition

    Björn Schuller, Gerhard Rigoll, and Manfred Lang. Hidden markov model-based speech emotion recognition. In Proceedings 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), volume 2, pages II–1, 2003

    2003

  77. [83]

    Klaus R. Scherer. V ocal communication of emotion: A review of research paradigms.Speech Communication, 40(1-2):227–256, 2003

    2003

  78. [84]

    Exploiting emotion-semantic correlations for empathetic response generation

    Zhou Yang, Zhaochun Ren, Yufeng Wang, Xiaofei Zhu, Zhihao Chen, Tiecheng Cai, Yunbing Wu, Yisong Su, Sibo Ju, and Xiangwen Liao. Exploiting emotion-semantic correlations for empathetic response generation. Findings of the Association for Computational Linguistics: EMNLP 2023, ...

    2023

  79. [85]

    Speech emotion recognition using deep convolutional neural networks improved by the fast continuous wavelet transform

    Björn E Van Zwol, Mathijs A Langezaal, Lukas Arts, Albert Gatt, and Egon L Van Den Broek. Speech emotion recognition using deep convolutional neural networks improved by the fast continuous wavelet transform. In Workshop Proceedings of the 19th International Conference on Inte...

    2023

  80. [86]

    A bert based dual-channel explainable text emotion recognition system

    Puneet Kumar and Balasubramanian Raman. A bert based dual-channel explainable text emotion recognition system. Neural Networks, 150:392–407, 6 2022

    2022

  81. [87]

    Daniel Jurafsky and James H. Martin. Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition. Prentice Hall, 2009. 23 Emotion Recognition and Generation: A Comprehensive Review of Face, Speech, and Text ...

    2009

  82. [88]

    BERT: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. CoRR, abs/1810.04805, 2019

    2019 arXiv

  83. [89]

    Attention is all you need

    A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017

    2017

  84. [90]

    Srl4e-semantic role labeling for emotions: A unified evaluation framework, 2022

    Cesare Campagnano, Simone Conia, and Roberto Navigli. Srl4e-semantic role labeling for emotions: A unified evaluation framework, 2022

    2022

  85. [91]

    Clarin-emo: Training emotion recognition models using human annotation and chatgpt

    Bartłomiej Koptyra, Anh Ngo, Łukasz Radli´nski, and Jan Koco´n. Clarin-emo: Training emotion recognition models using human annotation and chatgpt. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformati...

    2023

  86. [92]

    Xlm-emo: Multilingual emotion prediction in social media text

    Federico Bianchi, Debora Nozza, and Dirk Hovy. Xlm-emo: Multilingual emotion prediction in social media text. pages 195–203, 2022

    2022

  87. [93]

    Dual attention networks for multimodal reasoning and matching

    Zhong-Yuan Li, Jing-Yi Duan, Ming Zhou, and Yu-Gang Zhao. Dual attention networks for multimodal reasoning and matching. IEEE Transactions on Image Processing, 29:7387–7396, 2020

    2020

  88. [94]

    Acoustic emotion recognition: A benchmark comparison of performances

    Björn Schuller, Bogdan Vlasenko, Florian Eyben, Martin Wöllmer, and Gerhard Rigoll. Acoustic emotion recognition: A benchmark comparison of performances. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pages 485–488, 2004

    2004

  89. [95]

    Zhihua Zeng, Maja Pantic, and Glenn I. Roisman. A survey of affect recognition methods: Audio, visual, and spontaneous expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(4):849–865, 2020

    2020

  90. [96]

    A review of deepfakes and an analysis of detection methods

    Michael Matton, Marin Ferecatu, and Nozha Boujemaa. A review of deepfakes and an analysis of detection methods. Journal of Imaging, 5(5):52, 2019

    2019

  91. [97]

    The deepfake detection challenge dataset

    Balazs Dolhansky, Alexander Howie, Hui Zheng, Ser-Nam Lim, and Charles Nicholas. The deepfake detection challenge dataset. arXiv preprint arXiv:2006.07397, 2020

    2006 arXiv

  92. [98]

    Deep Learning, volume 1

    Courville Aaron Goodfellow Ian, Bengio Yoshua and Bengio Samy. Deep Learning, volume 1. 2016

    2016

  93. [99]

    Emo: Emote portrait alive – generating expressive portrait videos with audio2video diffusion model under weak conditions

    Linrui Tian, Qi Wang, Bang Zhang, and Liefeng Bo. Emo: Emote portrait alive – generating expressive portrait videos with audio2video diffusion model under weak conditions. 2 2024

    2024

  94. [100]

    Instagram

    Meta. Instagram. https://www.instagram.com/, 2024. Accessed: July 04, 2024

    2024

  95. [101]

    Snapchat

    Snap Inc. Snapchat. https://www.snapchat.com/, 2024. Accessed: July 04, 2024

    2024

  96. [102]

    FaceApp. Faceapp. https://www.faceapp.com/, 2017. Accessed: 2024-07-04

    2017

  97. [103]

    Vasa-1: Lifelike audio-driven talking faces generated in real time.arXiv preprint arXiv:2404.10667, 2024

    Sicheng Xu, Guojun Chen, Yu-Xiao Guo, Jiaolong Yang, Chong Li, Zhenyu Zang, Yizhong Zhang, Xin Tong, and Baining Guo. Vasa-1: Lifelike audio-driven talking faces generated in real time.arXiv preprint arXiv:2404.10667, 2024

    2024 arXiv

  98. [104]

    in-the-wild

    Foivos Paraperas Papantoniou, Panagiotis P Filntisis, Petros Maragos, and Anastasios Roussos. Neural emotion director: Speech-preserving semantic control of facial expressions in "in-the-wild" videos, 2021

    2021

  99. [105]

    Efficient emotional adaptation for audio- driven talking-head generation, 2023

    Yuan Gan, Zongxin Yang, Xihang Yue, Lingyun Sun, and Yi Yang. Efficient emotional adaptation for audio- driven talking-head generation, 2023

    2023

  100. [106]

    Emotion guided speech-driven facial animation

    Sewhan Chun, Daegeun Choe, Shindong Kang, Shounan An, Youngbak Jo, and Insoo Oh. Emotion guided speech-driven facial animation. Association for Computing Machinery, Inc, 12 2021

    2021

  101. [107]

    Expressive speech-driven facial animation with controllable emotions

    Yutong Chen, Junhong Zhao, and Wei-Qiang Zhang. Expressive speech-driven facial animation with controllable emotions. arXiv preprint arXiv:2301.02008, 2023

    2023 arXiv

  102. [108]

    Pose-controllable talking face generation by implicitly modularized audio-visual representation

    Hang Zhou, Yasheng Sun, Wayne Wu, Chen Change Loy, Xiaogang Wang, and Ziwei Liu. Pose-controllable talking face generation by implicitly modularized audio-visual representation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4176–4186, 2021

    2021

  103. [109]

    A lip sync expert is all you need for speech to lip generation in the wild

    KR Prajwal, Rudrabha Mukhopadhyay, Vinay P Namboodiri, and CV Jawahar. A lip sync expert is all you need for speech to lip generation in the wild. In Proceedings of the 28th ACM international conference on multimedia, pages 484–492, 2020

    2020

  104. [110]

    Seeing what you said: Talking face generation guided by a lip reading expert

    Jiadong Wang, Xinyuan Qian, Malu Zhang, Robby T Tan, and Haizhou Li. Seeing what you said: Talking face generation guided by a lip reading expert. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14653–14662, 2023

    2023

  105. [111]

    Se Jin Park, Minsu Kim, Joanna Hong, Jeongsoo Choi, and Yong Man Ro. Synctalkface: Talking face generation with precise lip-syncing via audio-lip memory.Proceedings of the 36th AAAI Conference on Artificial Intelligence, AAAI 2022, 36:2062–2070, 2022. 24 Emotion Recognition an...

    2022

  106. [112]

    Sadtalker: Learning realistic 3d motion coefficients for stylized audio-driven single image talking face animation

    Wenxuan Zhang, Xiaodong Cun, Xuan Wang, Yong Zhang, Xi Shen, Yu Guo, Ying Shan, and Fei Wang. Sadtalker: Learning realistic 3d motion coefficients for stylized audio-driven single image talking face animation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pa...

    2023

  107. [113]

    Cohn, and László A

    Koichiro Niinuma, Itir Onal Ertugrul, Jeffrey F. Cohn, and László A. Jeni. Facial expression manipulation for personalized facial action estimation. Frontiers in Signal Processing, 2, 4 2022

    2022

  108. [115]

    Realistic speech-driven facial animation with gans

    Konstantinos V ougioukas, Stavros Petridis, and Maja Pantic. Realistic speech-driven facial animation with gans. International Journal of Computer Vision, 128:1398–1413, 2020

    2020

  109. [116]

    Dualpathgan: Facial reenacted emotion synthesis

    Jiahui Kong, Haibin Shen, and Kejie Huang. Dualpathgan: Facial reenacted emotion synthesis. IET Computer Vision, 15:501–513, 10 2021

    2021

  110. [117]

    Toward fine-grained facial expression manipulation

    Jun Ling, Han Xue, Li Song, Shuhui Yang, Rong Xie, and Xiao Gu. Toward fine-grained facial expression manipulation. 4 2020

    2020

  111. [118]

    Combining gan with reverse correlation to construct personalized facial expressions

    Sen Yan, Catherine Soladié, Jean Julien Aucouturier, and Renaud Seguier. Combining gan with reverse correlation to construct personalized facial expressions. PLoS ONE, 18, 8 2023

    2023

  112. [119]

    Attention based facial expression manipulation

    Feng Wang, Suncheng Xiang, Ting Liu, and Yuzhuo Fu. Attention based facial expression manipulation. Institute of Electrical and Electronics Engineers Inc., 2021

    2021

  113. [120]

    Filntisis, and Petros Maragos

    Ioannis Pikoulis, Panagiotis P. Filntisis, and Petros Maragos. Photorealistic and identity-preserving image-based emotion manipulation with latent diffusion models. 8 2023

    2023

  114. [121]

    Dreamtalk: When expressive talking head generation meets diffusion probabilistic models

    Yifeng Ma, Shiwei Zhang, Jiayu Wang, Xiang Wang, Yingya Zhang, and Zhidong Deng. Dreamtalk: When expressive talking head generation meets diffusion probabilistic models. arXiv preprint arXiv:2312.09767, 2023

    2023 arXiv

  115. [122]

    Promptvc: Flexible stylistic voice conversion in latent space driven by natural language prompts

    Jixun Yao, Yuguang Yang, Yi Lei, Ziqian Ning, Yanni Hu, Yu Pan, Jingjing Yin, Hongbin Zhou, Heng Lu, and Lei Xie. Promptvc: Flexible stylistic voice conversion in latent space driven by natural language prompts. In ICASSP 2024-2024 IEEE International Conference on Acoustics, S...

    2024

  116. [123]

    Triaan-vc: Triple adaptive attention normalization for any-to-any voice conversion

    Hyun Joon Park, Seok Woo Yang, Jin Sob Kim, Wooseok Shin, and Sung Won Han. Triaan-vc: Triple adaptive attention normalization for any-to-any voice conversion. ICASSP , IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, 2023

    2023

  117. [124]

    ElevenLabs

    ElevenLabs. ElevenLabs. https://www.elevenlabs.io/, 2024. Accessed: July 04, 2024

    2024

  118. [125]

    Controllable residual speaker representation for voice conversion

    Le Xu, Jiangyan Yi, Jianhua Tao, Tao Wang, Yong Ren, and Rongxiu Zhong. Controllable residual speaker representation for voice conversion. 2023

    2023

  119. [126]

    Stylebert: Text-audio sentiment analysis with bi-directional style enhancement

    Fei Lin, Shengqiang Liu, Cong Zhang, Jin Fan, and Zizhao Wu. Stylebert: Text-audio sentiment analysis with bi-directional style enhancement. Information Systems, 114, 3 2023

    2023

  120. [127]

    Speaking style conversion in the waveform domain using discrete self-supervised units

    Gallil Maimon and Yossi Adi. Speaking style conversion in the waveform domain using discrete self-supervised units. 12 2022

    2022

  121. [128]

    Paul Taylor and Alan W. Black. The state of the art in text-to-speech synthesis. Speech Communication, 51(9):850–863, 2009

    2009

  122. [129]

    Ganspeech: Adversarial training for high-fidelity multi-speaker speech synthesis

    Jinhyeok Yang, Jae-Sung Bae, Taejun Bak, Youngik Kim, and Hoon-Young Cho. Ganspeech: Adversarial training for high-fidelity multi-speaker speech synthesis. arXiv preprint arXiv:2106.15153, 2021

    2021 arXiv

  123. [130]

    Dddm-vc: Decoupled denoising diffusion models with disentangled representation and prior mixup for verified robust voice conversion

    Ha-Yeong Choi, Sang-Hoon Lee, and Seong-Whan Lee. Dddm-vc: Decoupled denoising diffusion models with disentangled representation and prior mixup for verified robust voice conversion. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 17862–17870, 2024

    2024

  124. [131]

    Sequence-to-sequence voice conversion using f0 and time conditioning and adversarial learning

    Frederik Bous, Laurent Benaroya, Nicolas Obin, and Axel Roebel. Sequence-to-sequence voice conversion using f0 and time conditioning and adversarial learning. 2021

    2021

  125. [132]

    Lin, Chung Ming Chien, and Hung Yi Lee

    Jheng Hao Lin, Yist Y . Lin, Chung Ming Chien, and Hung Yi Lee. S2vc: A framework for any-to-any voice conversion with self-supervised pretrained representations. Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, 6:4785–4789, 2021

    2021

  126. [133]

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context

    Machel Reid, Nikolay Savinov, Denis Teplyashin, Dmitry Lepikhin, Timothy Lillicrap, Jean-baptiste Alayrac, Radu Soricut, Angeliki Lazaridou, Orhan Firat, Julian Schrittwieser, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv pre...

    2024 arXiv

  127. [134]

    Learning representations by back-propagating errors

    David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. Learning representations by back-propagating errors. Nature, 323(6088):533–536, 1986

    1986

  128. [135]

    Long short-term memory

    Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Computation, 9(8):1735–1780, 1997

    1997

  129. [136]

    Mike Schuster and Kuldip K. Paliwal. Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing, 45(11):2673–2681, 1997

    1997

  130. [137]

    Ilya Sutskever, Oriol Vinyals, and Quoc V . Le. Sequence to sequence learning with neural networks.Advances in Neural Information Processing Systems, 27:3104–3112, 2014

    2014

  131. [138]

    Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C

    Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C. Courville, and Yoshua Bengio. Generative adversarial nets. Advances in Neural Information Processing Systems, 27:2672–2680, 2014

    2014

  132. [139]

    Neural machine translation by jointly learning to align and translate

    Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014

    2014 arXiv

  133. [140]

    Adapting a language model for controlled affective text generation

    Tushar Goswamy, Ishika Singh, Ahsan Barkati, and Ashutosh Modi. Adapting a language model for controlled affective text generation. 2020

    2020

  134. [141]

    Wav2vec 2.0: A framework for self-supervised learning of speech representations.Advances in neural information processing systems, 33:12449– 12460, 2020

    Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. Wav2vec 2.0: A framework for self-supervised learning of speech representations.Advances in neural information processing systems, 33:12449– 12460, 2020

    2020

  135. [142]

    Language models are unsupervised multitask learners

    Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. OpenAI Blog, 2019

    2019

  136. [143]

    Libreface: An open-source toolkit for deep facial expression analysis, 2024

    Di Chang, Yufeng Yin, Zongjian Li, Minh Tran, and Mohammad Soleymani. Libreface: An open-source toolkit for deep facial expression analysis, 2024

    2024

  137. [144]

    End-to-end modeling and transfer learning for audiovisual emotion recognition in-the-wild

    Denis Dresvyanskiy, Elena Ryumina, Heysem Kaya, Maxim Markitantov, Alexey Karpov, and Wolfgang Minker. End-to-end modeling and transfer learning for audiovisual emotion recognition in-the-wild. Multimodal Technologies and Interaction, 6, 2 2022

    2022

  138. [145]

    A discriminatively deep fusion approach with improved conditional gan (im-cgan) for facial expression recognition

    Zhe Sun, Hehao Zhang, Jiatong Bai, Mingyang Liu, and Zhengping Hu. A discriminatively deep fusion approach with improved conditional gan (im-cgan) for facial expression recognition. Pattern Recognition, 135, 3 2023

    2023

  139. [146]

    Sanoar Hossain, Saiyed Umer, Ranjeet Kr Rout, and M. Tanveer. Fine-grained image analysis for facial expression recognition using deep convolutional neural networks with bilinear pooling. Applied Soft Computing, 134, 2 2023

    2023

  140. [147]

    Talkclip: Talking head generation with text-guided expressive speaking styles

    Yifeng Ma, Suzhen Wang, Yu Ding, Bowen Ma, Tangjie Lv, Changjie Fan, Zhipeng Hu, Zhidong Deng, and Xin Yu. Talkclip: Talking head generation with text-guided expressive speaking styles. arXiv preprint arXiv:2304.00334, 2023

    2023 arXiv

  141. [148]

    Styletalk: One-shot talking head generation with controllable speaking styles

    Yifeng Ma, Suzhen Wang, Zhipeng Hu, Changjie Fan, Tangjie Lv, Yu Ding, Zhidong Deng, and Xin Yu. Styletalk: One-shot talking head generation with controllable speaking styles. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 1896–1904, 2023

    1904

  142. [149]

    One-shot talking face generation from single-speaker audio-visual correlation learning

    Suzhen Wang, Lincheng Li, Yu Ding, and Xin Yu. One-shot talking face generation from single-speaker audio-visual correlation learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 2531–2539, 2022

    2022

  143. [150]

    Makelttalk: speaker-aware talking-head animation

    Yang Zhou, Xintong Han, Eli Shechtman, Jose Echevarria, Evangelos Kalogerakis, and Dingzeyu Li. Makelttalk: speaker-aware talking-head animation. ACM Transactions On Graphics (TOG), 39(6):1–15, 2020

    2020

  144. [151]

    Eamm: One-shot emotional talking face via audio-based emotion-aware motion model

    Xinya Ji, Hang Zhou, Kaisiyuan Wang, Qianyi Wu, Wayne Wu, Feng Xu, and Xun Cao. Eamm: One-shot emotional talking face via audio-based emotion-aware motion model. In ACM SIGGRAPH 2022 Conference Proceedings, pages 1–10, 2022

    2022

  145. [152]

    Expressive talking head generation with granular audio-visual control

    Borong Liang, Yan Pan, Zhizhi Guo, Hang Zhou, Zhibin Hong, Xiaoguang Han, Junyu Han, Jingtuo Liu, Errui Ding, and Jingdong Wang. Expressive talking head generation with granular audio-visual control. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog...

    2022

  146. [153]

    Audio2head: Audio-driven one-shot talking- head generation with natural head motion

    Suzhen Wang, Lincheng Li, Yu Ding, Changjie Fan, and Xin Yu. Audio2head: Audio-driven one-shot talking- head generation with natural head motion. arXiv preprint arXiv:2107.09293, 2021

    2021 arXiv

  147. [154]

    Hierarchical cross-modal talking face generation with dynamic pixel-wise loss

    Lele Chen, Ross K Maddox, Zhiyao Duan, and Chenliang Xu. Hierarchical cross-modal talking face generation with dynamic pixel-wise loss. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7832–7841, 2019. 26 Emotion Recognition and Gener...

    2019

  148. [155]

    Clstm: Deep feature-based speech emotion recognition using the hierarchical convlstm network

    Soonil Kwon et al. Clstm: Deep feature-based speech emotion recognition using the hierarchical convlstm network. Mathematics (2227-7390), 8(12), 2020

    2020

  149. [156]

    Speech emotion recognition from 3d log-mel spectrograms with deep learning network

    Hao Meng, Tianhao Yan, Fei Yuan, and Hongwei Wei. Speech emotion recognition from 3d log-mel spectrograms with deep learning network. IEEE access, 7:125868–125881, 2019

    2019

  150. [157]

    Decision tree svm model with fisher feature selection for speech emotion recognition

    Linhui Sun, Sheng Fu, and Fu Wang. Decision tree svm model with fisher feature selection for speech emotion recognition. EURASIP Journal on Audio, Speech, and Music Processing, 2019(1):1–14, 2019

    2019

  151. [158]

    Speech emotion recognition with deep convolutional neural networks

    Dias Issa, M Fatih Demirci, and Adnan Yazici. Speech emotion recognition with deep convolutional neural networks. Biomedical Signal Processing and Control, 59:101894, 2020

    2020

  152. [159]

    Optimal feature selection based speech emotion recognition using two-stream deep convolutional neural network

    Mustaqeem and Soonil Kwon. Optimal feature selection based speech emotion recognition using two-stream deep convolutional neural network. International Journal of Intelligent Systems, 36(9):5116–5135, 2021

    2021

  153. [160]

    Speech emotion recognition based on an improved brain emotion learning model

    Zhen-Tao Liu, Qiao Xie, Min Wu, Wei-Hua Cao, Ying Mei, and Jun-Wei Mao. Speech emotion recognition based on an improved brain emotion learning model. Neurocomputing, 309:145–156, 2018

    2018

  154. [161]

    Lanser: Language-model supported speech emotion recognition

    Taesik Gong, Josh Belanich, Krishna Somandepalli, Arsha Nagrani, Brian Eoff, and Brendan Jou. Lanser: Language-model supported speech emotion recognition. volume 2023-August, pages 2408–2412. International Speech Communication Association, 2023

    2023

  155. [162]

    An iterative emotion interaction network for emotion recognition in conversations

    Xin Lu, Yanyan Zhao, Yang Wu, Yijian Tian, Huipeng Chen, and Bing Qin. An iterative emotion interaction network for emotion recognition in conversations. In Proceedings of the 28th international conference on computational linguistics, pages 4078–4088, 2020

    2020

  156. [163]

    Speech emotion classification from affective dimensions: Limitation and advantage

    Meysam Shamsi. Speech emotion classification from affective dimensions: Limitation and advantage. 2023 11th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos, ACIIW 2023, 2023

    2023

  157. [164]

    Emotion recognition from speech using wav2vec 2.0 embeddings

    Leonardo Pepino, Pablo Riera, and Luciana Ferrer. Emotion recognition from speech using wav2vec 2.0 embeddings. arXiv preprint arXiv:2104.03502, 2021

    2021 arXiv

  158. [165]

    Emotion recognition from speech using artificial neural networks and recurrent neural networks

    Shambhavi Sharma. Emotion recognition from speech using artificial neural networks and recurrent neural networks. In 2021 11th International Conference on Cloud Computing, Data Science & Engineering (Confluence), pages 153–158. IEEE, 2021

    2021

  159. [166]

    Any-to-many voice conversion with location-relative sequence-to-sequence modeling

    Songxiang Liu, Yuewen Cao, Disong Wang, Xixin Wu, Xunying Liu, and Helen Meng. Any-to-many voice conversion with location-relative sequence-to-sequence modeling. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 29:1717–1728, 2021

    2021

  160. [167]

    Autovc: Zero-shot voice style transfer with only autoencoder loss

    Kaizhi Qian, Yang Zhang, Shiyu Chang, Xuesong Yang, and Mark Hasegawa-Johnson. Autovc: Zero-shot voice style transfer with only autoencoder loss. In International Conference on Machine Learning, pages 5210–5219. PMLR, 2019

    2019

  161. [168]

    Global prosody style transfer without text transcriptions

    Kaizhi Qian, Yang Zhang, Shiyu Chang, Jinjun Xiong, Chuang Gan, David Cox, and Mark Hasegawa-Johnson. Global prosody style transfer without text transcriptions. In International Conference on Machine Learning, pages 8650–8660. PMLR, 2021

    2021

  162. [169]

    Vqmivc: Vector quantization and mutual information-based unsupervised speech representation disentanglement for one-shot voice conversion

    Disong Wang, Liqun Deng, Yu Ting Yeung, Xiao Chen, Xunying Liu, and Helen Meng. Vqmivc: Vector quantization and mutual information-based unsupervised speech representation disentanglement for one-shot voice conversion. arXiv preprint arXiv:2106.10132, 2021

    2021 arXiv

  163. [170]

    V oice conversion for stuttered speech, instruments, unseen languages and textually described voices

    Matthew Baas and Herman Kamper. V oice conversion for stuttered speech, instruments, unseen languages and textually described voices. In Southern African Conference for Artificial Intelligence Research, pages 136–150. Springer, 2023

    2023

  164. [171]

    Freevc: Towards high-quality text-free one-shot voice conversion

    Jingyi Li, Weiping Tu, and Li Xiao. Freevc: Towards high-quality text-free one-shot voice conversion. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE, 2023

    2023

  165. [172]

    Yourtts: Towards zero-shot multi-speaker tts and zero-shot voice conversion for everyone

    Edresson Casanova, Julian Weber, Christopher D Shulby, Arnaldo Candido Junior, Eren Gölge, and Moacir A Ponti. Yourtts: Towards zero-shot multi-speaker tts and zero-shot voice conversion for everyone. InInternational Conference on Machine Learning, pages 2709–2720. PMLR, 2022

    2022

  166. [173]

    Phoneme hallucinator: One-shot voice conversion via set expansion

    Siyuan Shan, Yang Li, Amartya Banerjee, and Junier B Oliva. Phoneme hallucinator: One-shot voice conversion via set expansion. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 14910– 14918, 2024

    2024

  167. [174]

    V oicemixer: Adversarial voice style mixup

    Sang-Hoon Lee, Ji-Hoon Kim, Hyunseung Chung, and Seong-Whan Lee. V oicemixer: Adversarial voice style mixup. Advances in Neural Information Processing Systems, 34:294–308, 2021. 27 Emotion Recognition and Generation: A Comprehensive Review of Face, Speech, and Text ModalitiesA...

    2021

  168. [175]

    An analysis of annotated corpora for emotion classification in text

    Laura Ana Maria Oberländer and Roman Klinger. An analysis of annotated corpora for emotion classification in text. In Proceedings of the 27th international conference on computational linguistics, pages 2104–2119, 2018

    2018

  169. [177]

    Multimodal approach of speech emotion recognition using multi-level multi-head fusion attention-based recurrent neural network

    Ngoc-Huynh Ho, Hyung-Jeong Yang, Soo-Hyung Kim, and Gueesang Lee. Multimodal approach of speech emotion recognition using multi-level multi-head fusion attention-based recurrent neural network. IEEE Access, 8:61672–61686, 2020

    2020

  170. [178]

    Modeling both context-and speaker-sensitive dependence for emotion detection in multi-speaker conversations

    Dong Zhang, Liangqing Wu, Changlong Sun, Shoushan Li, Qiaoming Zhu, and Guodong Zhou. Modeling both context-and speaker-sensitive dependence for emotion detection in multi-speaker conversations. In IJCAI, pages 5415–5421. Macao, 2019

    2019

  171. [179]

    Model of multi-turn dialogue in emotional chatbot

    Chien-Hao Kao, Chih-Chieh Chen, and Yu-Tza Tsai. Model of multi-turn dialogue in emotional chatbot. In 2019 International Conference on Technologies and Applications of Artificial Intelligence (TAAI), pages 1–5. IEEE, 2019

    2019

  172. [180]

    Two-stage text feature selection method for human emotion recognition

    Lovejit Singh, Sarbjeet Singh, and Naveen Aggarwal. Two-stage text feature selection method for human emotion recognition. In Proceedings of 2nd International Conference on Communication, Computing and Networking: ICCCN 2018, NITTTR Chandigarh, India, pages 531–538. Springer, 2018

    2018

  173. [181]

    Text emotion distribution learning via multi-task convolutional neural network

    Yuxiang Zhang, Jiamei Fu, Dongyu She, Ying Zhang, Senzhang Wang, and Jufeng Yang. Text emotion distribution learning via multi-task convolutional neural network. In IJCAI, pages 4595–4601, 2018

    2018

  174. [182]

    Xlm-t: Multilingual language models in twitter for sentiment analysis and beyond

    Francesco Barbieri, Luis Espinosa Anke, and Jose Camacho-Collados. Xlm-t: Multilingual language models in twitter for sentiment analysis and beyond. 4 2021

    2021

  175. [183]

    Xed: A multilingual dataset for sentiment analysis and emotion detection

    Emily Öhman, Marc Pàmies, Kaisla Kajava, and Jörg Tiedemann. Xed: A multilingual dataset for sentiment analysis and emotion detection. arXiv preprint arXiv:2011.01612, 2020

    2011 arXiv

  176. [184]

    Anthropic. Claude3. https://www.anthropic.com/claude/, 2024. Accessed: July 13, 2024. 28

    2024

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.