Pith. sign in

REVIEW 4 major objections 7 minor 59 references

Think-Before-Draw: Decomposing Emotion Semantics & Fine-Grained Controllable Expressive Talking Head Generation

T0 review · 4 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Think-Before-Draw claims that splitting an emotion label into coarse and muscle-level text prompts, applied in that order during diffusion denoising, yields fine-grained control over emotional talking-head video and state-of-the-art…

desk verdict Useful recombination of CoT and progressive conditioning, but the SOTA claim is unverifiable until the evaluation protocol is documented and baselines are re-run. read the letter →

arxiv 2507.12761 v1 pith:3HXOGTH3 submitted 2025-07-17 cs.CV cs.AI

Hanlei Shi , Leyuan Qu , Yu Liu , Di Gao , Yuhua Zheng , Taihao Li This is my paper
classification cs.CVcs.AI
keywords emotionaltalking-headgenerationchain-of-thoughtreasoningfacialactionunitstext-drivenanimationprogressiveguidancedenoisingdiffusionmodelexpressioncontrolzero-shot
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The authors set out to show that discrete emotion labels are too coarse to drive natural talking-head animation, and that decomposing those labels into muscle-level text before generation yields finer, more controllable expressions. Their framework, Think-Before-Draw, first runs a chain-of-thought module that expands an emotion label into a holistic prompt (identity plus overall emotion) and a fine-grained prompt naming the facial action units and muscles involved. A diffusion-based video generator then applies the holistic prompt during early denoising steps and the fine-grained prompt during later steps, mimicking an artist's global-to-local workflow. The authors report that this scheme outperforms prior audio-, video-, and label-driven methods on the MEAD and HDTF benchmarks, and that it generalizes to previously unseen portrait images. If correct, the framework gives text-driven talking-head systems a practical route from abstract emotion semantics to anatomically grounded facial motion.

What carries the argument

The load-bearing machinery is the combination of a Chain-of-Thought Facial Animation module (CoT-FA) with a progressive guidance denoising schedule. CoT-FA is a four-tier prompt pipeline—facial information extraction, facial action unit analysis, facial muscle analysis, and prompt design—that converts an emotion label and a reference image into two text embeddings: a coarse global condition and a fine muscle-level condition. The progressive strategy feeds the coarse embedding into the denoising U-Net's text-attention layers during the first denoising stage and switches to the fine embedding in the second stage, blending the two with an alpha weight that depends on the timestep's position within its stage rather than using a fixed value. This global-to-local conditioning scheme is the mechanism that carries the argument, because it is what lets a single diffusion backbone produce expressions that are both emotionally coherent and locally detailed.

What would settle it

Re-run MakeItTalk, SadTalker, PC-AVS, EAMM, StyleTalk, and EAT on the exact same curated MEAD split and preprocessing pipeline used for TBD, and recompute the Table 1 metrics with the same evaluation code; if the PSNR gap over StyleTalk collapses or reverses, the state-of-the-art claim for the stated protocol does not hold.

Watch

Extended reading notes

Core claim

The central claim is that the semantic gap between an abstract emotion label and facial motion can be closed by decomposing the label before generation. The paper's CoT-FA module performs this decomposition in four stages: it extracts identity attributes from the reference image, identifies the Facial Action Units associated with the emotion under the Facial Action Coding System, analyzes the underlying facial muscles and their synergies, and finally designs two text prompts—a coarse global description and a fine muscle-level description. The progressive guidance denoising strategy then schedules these conditions so that the global prompt steers the early denoising steps and the local prompt steers the later steps, with a timestep-dependent blending weight that smooths the transition between stages. On the MEAD benchmark the method reports PSNR 28.64, SSIM 0.88, and FID 17.28, and on HDTF PSNR 23.28, SSIM 0.74, and FID 16.17, which the authors present as state-of-the-art results; the lone exception is lip-sync confidence on HDTF, attributed to the uniform neutral text labels used because that dataset has no emotion annotations.

Load-bearing premise

The load-bearing premise is that all methods in Table 1 were evaluated under the same protocol, since the paper trains on an unspecified curated version of MEAD and does not state that the baselines were re-run on that same split with identical preprocessing.

Editorial extensions

If this is right

  • Text-driven talking-head systems can replace flat emotion labels with multi-level prompts, giving users explicit control over expression intensity as well as emotion category.
  • The two-stage guidance schedule can be layered onto any diffusion-based talking-head generator that already accepts text conditioning, since it only changes how conditions are ordered during denoising.
  • Because each generated expression is traceable to the action units and muscles named in the prompt, the framework makes emotional animation more explainable and easier to debug.
  • For a fixed 40-step denoising budget, two-stage partitioning is reported to beat three- and four-stage partitioning, since too many switches destabilize generation.
  • On HDTF, where emotion annotations are absent, the method still improves image quality and FID over baselines using neutral labels, with the only reported weakness being lip-sync confidence.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: If the MEAD numbers in Table 1 were produced under the same protocol for all methods, the unusually large PSNR gap over StyleTalk (28.64 vs. 21.60) would be strong evidence for the method; if baselines were not re-run on the same curated split, that gap needs verification before being read comparatively.
  • Inference: The progressive guidance idea is not specific to faces: any conditional diffusion task with a natural coarse-to-fine prompt ordering, such as scene-level then object-level text-to-video, could adopt the same two-stage scheduling.
  • Inference: Because CoT-FA prompts pass through manual review, the pipeline's ceiling depends on how reliably the language model produces anatomically correct muscle descriptions; an automated check against known action-unit-muscle mappings would be a cheap way to test that dependency.
  • Inference: The three annotated intensity levels suggest a path to continuous intensity control: if the language model can interpolate between mild, moderate, and intense descriptions, users could dial expression strength continuously rather than choosing from discrete levels.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes Think-Before-Draw (TBD), a text-driven emotional talking-head generation framework with two main components: a Chain-of-Thought Facial Animation (CoT-FA) module that converts abstract emotion labels into coarse- and fine-grained text descriptions grounded in FACS/AU terminology, and a progressive guidance denoising strategy that first applies coarse text embeddings in early denoising steps and fine-grained embeddings in later steps, with an alpha-blended transition. Experiments on MEAD and HDTF report large quantitative gains over several audio-, video-, and label-driven baselines across PSNR, SSIM, FID, CPBD, and SyncConf, supplemented by qualitative comparisons, an ablation study, t-SNE visualizations, and a small user study. The central claim is state-of-the-art performance on widely used benchmarks, including MEAD and HDTF.

Significance. If the reported results are reproducible under controlled evaluation protocols, the framework is a meaningful step forward: it replaces discrete emotion labels with anatomically grounded, FACS-aligned text descriptions, provides an intuitive coarse-to-fine conditioning schedule for diffusion-based talking-head generation, and demonstrates fine-grained intensity control. The t-SNE style-code analysis and zero-shot portrait evaluations are useful additions. However, the paper does not release code, exact data splits, or evaluation scripts, and the quantitative evidence currently lacks the protocol transparency needed to support the state-of-the-art claim. The contribution is potentially valuable, but the verification burden is not yet met.

major comments (4)
  1. [§4.1 (Datasets) and Table 1] The central state-of-the-art claim is not verifiable because the evaluation protocol is underspecified. The MEAD split is described only as "a curated version" with an 80/20 ratio; there is no statement of whether the split is by identity or by video clip, no description of the curation, and no confirmation that baselines were re-run on the exact same split with identical preprocessing. For HDTF, 20 subjects with 10 clips each are "selected" for training, but no test split is defined; if the reported HDTF metrics were computed on clips also used for training, those numbers are not valid benchmark results. The 7.04 dB MEAD PSNR gap over StyleTalk (28.64 vs 21.60) is far larger than the improvements shown in the ablation study and is consistent with protocol differences rather than a genuine method advantage. The authors must specify the exact test identities and videos, describe the curation, and re-run all baselines under identical conditions.
  2. [§4.2, Table 1 and §4.4, Table 2] No error bars, standard deviations, or statistical significance tests are reported for any quantitative metric, so it is impossible to assess whether the observed gaps are meaningful. Moreover, the internal numbers are inconsistent: the full model's FID in Table 2 is 16.725, while Table 1 reports TBD's MEAD FID as 17.28 for what appears to be the same configuration. This discrepancy suggests either different evaluation sets or a reporting error. The authors should report mean and standard deviation over multiple runs or seeds, and clarify the relationship between the evaluation sets used in Tables 1 and 2.
  3. [§3.2 (Step-IV)] The statement "Each data entry undergoes manual review to filter out anomalous descriptions" introduces a selection step that can bias the evaluation. The paper does not specify whether this filtering occurred before the train/test split or whether the test set was frozen before manual review. If anomalous or hard test examples were removed from the test set, the reported scores are inflated. The authors should clarify the temporal ordering of filtering and splitting, and ideally evaluate on a fixed test set that is not subject to manual filtering.
  4. [§2.1 and §4.1 (Comparison setting)] The quantitative comparison omits the closest text-driven fine-grained baselines, including FG-EmoTalk [14], ExpCLIP [15], and Expressive Talking Avatars [16], all of which are cited in the related work. Since the paper's contribution is specifically fine-grained text-driven control, comparing only against EAT as a text-driven baseline does not support the claim of state-of-the-art performance over text-driven methods. These baselines should be included in the quantitative comparison, or a clear justification for their exclusion should be provided.
minor comments (7)
  1. [§1] There are typos such as "vitrual" instead of "virtual" and grammatical issues such as "a effective solution"; these should be corrected.
  2. [§3.4] The stage-split notation is ambiguous: s1 = t * 0.4 uses t both as the total number of sampling steps and as the current timestep, and "P si = t" should be a summation. Algorithm 2 would also benefit from clearer naming and a precise definition of the stage boundaries.
  3. [Table 3] The baseline "ETA" should be "EAT" to match the method name used elsewhere in the paper.
  4. [Figure 11] The subfigure captions appear to be swapped relative to the text: the text describes (a) as identity features from one speaker and (b) as expression features, while the caption labels (a) as emotion-related clustering and (b) as identity-related dispersion.
  5. [§4.3] Hallo and Sonic are discussed as qualitative comparisons only, yet they are not listed in the quantitative comparison table; the paper should state explicitly that these methods are evaluated qualitatively and not on the same benchmark.
  6. [§4.6] The user study is based on only 10 evaluators and 10 samples per condition; the authors should report per-item variance, confidence intervals, or a statistical test to support the MOS comparisons.
  7. [References] References [25] and [40] both cite the Hallo paper; one duplicate should be removed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the contribution is an empirical text-conditioned generation pipeline whose benchmark claims depend on external comparisons, not on a self-referential derivation.

full rationale

The paper's derivation chain is empirical rather than formal. The CoT-FA module takes an emotion label and a reference image and produces coarse and fine text prompts via Qwen2-VL guided by FACS knowledge; the progressive guidance denoising strategy then uses those prompts as conditioning in a diffusion U-Net trained with the standard noise-prediction loss in Eq. (1). There is no equation or construction by which an output quantity is defined as, or fitted to, the quantity it is said to predict. The central claims are benchmark comparisons against external baselines (MakeItTalk, SadTalker, PC-AVS, EAMM, StyleTalk, EAT) on MEAD and HDTF, and ablations that vary only the prompt-generation module and the guidance schedule. No load-bearing self-citation appears in the reference list, and no uniqueness theorem or prior-author result is invoked to force the design. The aspects flagged by the skeptical reader concern evaluation protocol ambiguity: the 'curated version of the MEAD dataset' is not precisely specified, the HDTF test split is not defined, and no baseline re-runs on the identical split are documented. Those are threats to comparability and to the strength of the SOTA claim, but they are not circularity under the definition used here, because the reported numbers are not constructed to equal the inputs by design. The manual review of prompt descriptions is a data-quality filter, not a fitted parameter renamed as a prediction. Therefore the appropriate circularity score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The framework rests on pretrained components (Stable Diffusion, ReferenceNet, Qwen2-VL, CLIP, Wav2Vec) and a small set of hand-chosen guidance hyperparameters. The main assumptions are about the validity of the FACS-based CoT decomposition and the ability of CLIP embeddings to carry muscle-level control. No new physical entities are introduced.

free parameters (3)
  • stage split s1 = 0.4*t = s1 = 0.4 * total denoising steps
    Chosen by hand for the two-stage guidance; no search is reported beyond the ablation of k=2 versus k=3 versus k=4.
  • alpha blend bounds = alpha_max=1.0, alpha_min=0.5
    Algorithm 1 sets these bounds for the timestep-dependent interpolation between coarse and fine text embeddings.
  • number of guidance stages k = 2
    Selected by ablation; the paper reports k=2 gives the best generation quality, so it is tuned to the validation metric rather than derived.
assumptions (4)
  • domain assumption FACS action units and facial muscle kinematics provide a valid mapping from emotion labels to physiologically grounded motion descriptions.
    Section 3.2 grounds CoT-FA on FACS and Duchenne's anatomy; if this mapping is not accurate, the generated fine prompts carry no real physiological signal.
  • domain assumption CLIP text embeddings of the fine muscle descriptions are informative enough for the U-Net text-attention layer to steer facial motion at the muscle level.
    Section 3.3 adds a text-attention layer conditioned on CLIP embeddings; the paper provides no direct evidence that CLIP separates fine muscle-level semantics.
  • domain assumption HDTF videos can be treated as neutral emotion for training and evaluation.
    Section 4.1 states "we assigned neutral emotion labels to its textual descriptions"; this makes HDTF results weak evidence for emotional expressiveness.
  • standard math The Stable Diffusion denoising objective and cross-attention formulation in Section 3.3 are assumed as standard background.
    Equation (1) and Equation (2) restate standard diffusion and attention machinery from cited prior work.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Think-Before-Draw: Decomposing Emotion Semantics & Fine-Grained Controllable Expressive Talking Head Generation." pith.science (2026). https://pith.science/paper/3HXOGTH3

@misc{pith2026250712761,
  author       = {Pith},
  title        = {Pith review of: Think-Before-Draw: Decomposing Emotion Semantics & Fine-Grained Controllable Expressive Talking Head Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3HXOGTH3}},
  note         = {Machine review of arXiv:2507.12761}
}
read the original abstract

Emotional talking-head generation has emerged as a pivotal research area at the intersection of computer vision and multimodal artificial intelligence, with its core value lying in enhancing human-computer interaction through immersive and empathetic engagement.With the advancement of multimodal large language models, the driving signals for emotional talking-head generation has shifted from audio and video to more flexible text. However, current text-driven methods rely on predefined discrete emotion label texts, oversimplifying the dynamic complexity of real facial muscle movements and thus failing to achieve natural emotional expressiveness.This study proposes the Think-Before-Draw framework to address two key challenges: (1) In-depth semantic parsing of emotions--by innovatively introducing Chain-of-Thought (CoT), abstract emotion labels are transformed into physiologically grounded facial muscle movement descriptions, enabling the mapping from high-level semantics to actionable motion features; and (2) Fine-grained expressiveness optimization--inspired by artists' portrait painting process, a progressive guidance denoising strategy is proposed, employing a "global emotion localization--local muscle control" mechanism to refine micro-expression dynamics in generated videos.Our experiments demonstrate that our approach achieves state-of-the-art performance on widely-used benchmarks, including MEAD and HDTF. Additionally, we collected a set of portrait images to evaluate our model's zero-shot generation capability.

Figures

Figures reproduced from arXiv: 2507.12761 by the authors.

Figure 1
Figure 1. Illustration of our proposed framework. Our framework primarily consists of two components: Chain-of [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. The overview of Think-Before-Draw. In the CoT-FA & Input Processing stage, the CoT-FA module pro￾cesses the reference image and text, generating multi-level descriptions encoded by text encoder into text embeddings ct1 , ct2 . The image encoder extracts image features xi , while the audio encoder extracts audio features ca. In the pro￾gressive guidance denoising processing stage, we employed a diffusion-based genera… view at source ↗
Figure 3
Figure 3. The overview of Chain-of-Thought Facial Animation. We establish a four-tier progressive processing [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Illustrative example of CoT-FA erate anatomically grounded prompts for emotionally expressive talking head generation. Each data entry undergoes manual review to filter out anomalous descriptions. Step-I: facial information extraction In the initial processing stage, w…
Figure 5
Figure 5. Figure 5: Illustration of either a CrossAttnDownBlock or CrossAttnUpBlock within the denoising U-Net architec [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: Illustration of progressive guidance denoising strategy. In this example, there are only two-stage guidance [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: Qualitative comparison of facial expression editing based on di [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: Visualization of prompts with CoT-FA and without CoT-FA [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: Visualization of different progressive guidance denoising strategy. Results demonstrate that the best is achieved when k=2, where two different text prompts guide the generation at denoising steps [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]
Figure 10
Figure 10. Figure 10: Visualization of different intensity control. The three intensity levels of textual descriptions—“Mild”, “Moderate”, and “Intense”—are used to guide the generation. The results show that distinct anger expressions corre￾sponding to each intensity level from the mouth …
Figure 11
Figure 11. Figure 11: t-SNE visualization of style codes. (a) Emotion-related clustering: Style codes from a single speaker, [PITH_FULL_IMAGE:figures/full_fig_p019_11.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

59 extracted references · 38 canonical work pages

  1. [14]

    Z. Sun, Y . Xuan, F. Liu, Y . Xiang, Fg-emotalk: Talking head video generation with fine-grained controllable facial expressions, in: Proceed- ings of the AAAI Conference on Artificial Intelligence, V ol. 38, 2024, pp. 5043–5051

    work page 2024

  2. [15]

    Zhong, H

    Y . Zhong, H. Wei, P. Yang, Z. Wang, Expclip: Bridging text and facial expressions via semantic alignment, in: Proceedings of the AAAI Conference on Artificial Intelligence, V ol. 38, 2024, pp. 7614–7622

    work page 2024

  3. [16]

    Y . Pan, S. Tan, S. Cheng, Q. Lin, Z. Zeng, K. Mitchell, Expressive talking avatars, IEEE Transactions on Visualization and Computer Graphics (2024)

    work page 2024

  4. [1]

    Mystakidis, Metaverse, Encyclopedia 2 (1) (2022) 486–497

    S. Mystakidis, Metaverse, Encyclopedia 2 (1) (2022) 486–497

    work page 2022

  5. [2]

    H. Wang, H. Ning, Y . Lin, W. Wang, S. Dhelim, F. Farha, J. Ding, M. Daneshmand, A survey on the metaverse: The state-of-the-art, technologies, applications, and challenges, IEEE Internet of Things Journal 10 (16) (2023) 14671–14688

    work page 2023

  6. [3]

    R. Zhen, W. Song, Q. He, J. Cao, L. Shi, J. Luo, Human-computer interaction system: A survey of talking-head generation, Electronics 12 (1) (2023) 218

    work page 2023

  7. [4]

    P. Zhou, L. Wang, Z. Liu, Y . Hao, P. Hui, S. Tarkoma, J. Kangasharju, A survey on generative ai and llm for video generation, understanding, and streaming, arXiv preprint arXiv:2404.16038 (2024)

    arXiv 2024

  8. [5]

    X. Ji, H. Zhou, K. Wang, W. Wu, C. C. Loy, X. Cao, F. Xu, Audio-driven emotional video portraits, in: Proceedings of the IEEE /CVF conference on computer vision and pattern recognition, 2021, pp. 14080–14089

    work page 2021

Show all 59 references
  1. [6]

    X. Ji, H. Zhou, K. Wang, Q. Wu, W. Wu, F. Xu, X. Cao, Eamm: One-shot emotional talking face via audio-based emotion-aware motion model, in: ACM SIGGRAPH 2022 conference proceedings, 2022, pp. 1–10

    2022

  2. [7]

    Y . Ma, S. Wang, Z. Hu, C. Fan, T. Lv, Y . Ding, Z. Deng, X. Yu, Styletalk: One-shot talking head generation with controllable speaking styles, in: Proceedings of the AAAI conference on artificial intelligence, V ol. 37, 2023, pp. 1896–1904

    2023

  3. [8]

    Zhang, X

    W. Zhang, X. Cun, X. Wang, Y . Zhang, X. Shen, Y . Guo, Y . Shan, F. Wang, Sadtalker: Learning realistic 3d motion coefficients for stylized audio-driven single image talking face animation, in: Proceedings of the IEEE /CVF conference on computer vision and pattern recognition...

    2023

  4. [9]

    Z. Peng, H. Wu, Z. Song, H. Xu, X. Zhu, J. He, H. Liu, Z. Fan, Emotalk: Speech-driven emotional disentanglement for 3d face animation, in: Proceedings of the IEEE/CVF international conference on computer vision, 2023, pp. 20687–20697

    2023

  5. [10]

    Y . Gan, Z. Yang, X. Yue, L. Sun, Y . Yang, Efficient emotional adaptation for audio-driven talking-head generation, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 22634–22645

    2023

  6. [11]

    H. Liu, W. Sun, D. Di, S. Sun, J. Yang, C. Zou, H. Bao, Moee: Mixture of emotion experts for audio-driven portrait animation, arXiv preprint arXiv:2501.01808 (2025)

    2025 arXiv

  7. [12]

    Liang, Y

    Z. Liang, Y . Xu, Y . Hong, P. Shang, Q. Wang, Q. Fu, K. Liu, A survey of multimodel large language models, in: Proceedings of the 3rd International Conference on Computer, Artificial Intelligence and Control Engineering, 2024, pp. 405–409

    2024

  8. [13]

    P. Wang, S. Bai, S. Tan, S. Wang, Z. Fan, J. Bai, K. Chen, X. Liu, J. Wang, W. Ge, et al., Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution, arXiv preprint arXiv:2409.12191 (2024)

    2024 arXiv

  9. [17]

    Ekman, W

    P. Ekman, W. V . Friesen, Facial action coding system, Environmental Psychology & Nonverbal Behavior (1978)

    1978

  10. [18]

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V . Le, D. Zhou, et al., Chain-of-thought prompting elicits reasoning in large language models, Advances in neural information processing systems 35 (2022) 24824–24837

    2022

  11. [19]

    Zhang, A

    Z. Zhang, A. Zhang, M. Li, A. Smola, Automatic chain of thought prompting in large language models, arXiv preprint arXiv:2210.03493 (2022)

    2022 arXiv

  12. [20]

    Jarlier, D

    S. Jarlier, D. Grandjean, S. Delplanque, K. N’diaye, I. Cayeux, M. I. Velazco, D. Sander, P. Vuilleumier, K. R. Scherer, Thermal analysis of facial muscles contractions, IEEE transactions on affective computing 2 (1) (2011) 2–9

    2011

  13. [21]

    S. Z. Li, A. K. Jain, Y .-L. Tian, T. Kanade, J. F. Cohn, Facial expression analysis, Handbook of face recognition (2005) 247–275

    2005

  14. [22]

    Duchenne, The mechanism of human facial expression, Cambridge university press, 1990

    G.-B. Duchenne, The mechanism of human facial expression, Cambridge university press, 1990

    1990

  15. [23]

    Faigin, The artist’s complete guide to facial expression, Watson-Guptill, 2012

    G. Faigin, The artist’s complete guide to facial expression, Watson-Guptill, 2012. 23

    2012

  16. [24]

    Loomis, Drawing the head & hands, Clube de Autores, 2021

    A. Loomis, Drawing the head & hands, Clube de Autores, 2021

    2021

  17. [26]

    Z. Chen, J. Cao, Z. Chen, Y . Li, C. Ma, Echomimic: Lifelike audio-driven portrait animations through editable landmark conditions, in: Proceedings of the AAAI Conference on Artificial Intelligence, V ol. 39, 2025, pp. 2403–2410

    2025

  18. [27]

    X. Ji, X. Hu, Z. Xu, J. Zhu, C. Lin, Q. He, J. Zhang, D. Luo, Y . Chen, Q. Lin, et al., Sonic: Shifting focus to global audio perception in portrait animation, arXiv preprint arXiv:2411.16331 (2024)

    2024 arXiv

  19. [28]

    S. Tan, B. Ji, M. Bi, Y . Pan, Edtalk: E fficient disentanglement for emotional talking head synthesis, in: European Conference on Computer Vision, Springer, 2024, pp. 398–416

    2024

  20. [29]

    J. Wei, Y . Tay, R. Bommasani, C. Raffel, B. Zoph, S. Borgeaud, D. Yogatama, M. Bosma, D. Zhou, D. Metzler, et al., Emergent abilities of large language models, arXiv preprint arXiv:2206.07682 (2022)

    2022 arXiv

  21. [30]

    Prystawski, M

    B. Prystawski, M. Li, N. Goodman, Why think step by step? reasoning emerges from the locality of experience, Advances in Neural Information Processing Systems 36 (2023) 70926–70947

    2023

  22. [31]

    Zhang, A

    Z. Zhang, A. Zhang, M. Li, H. Zhao, G. Karypis, A. Smola, Multimodal chain-of-thought reasoning in language models, arXiv preprint arXiv:2302.00923 (2023)

    2023 arXiv

  23. [32]

    H. Shao, S. Qian, H. Xiao, G. Song, Z. Zong, L. Wang, Y . Liu, H. Li, Visual cot: Advancing multi-modal language models with a compre- hensive dataset and benchmark for chain-of-thought reasoning, Advances in Neural Information Processing Systems 37 (2024) 8612–8642

    2024

  24. [33]

    J. Xie, W. Mao, Z. Bai, D. J. Zhang, W. Wang, K. Q. Lin, Y . Gu, Z. Chen, Z. Yang, M. Z. Shou, Show-o: One single transformer to unify multimodal understanding and generation, arXiv preprint arXiv:2408.12528 (2024)

    2024 arXiv

  25. [34]

    Z. Guo, R. Zhang, C. Tong, Z. Zhao, P. Gao, H. Li, P.-A. Heng, Can we generate images with cot? let’s verify and reinforce image generation step by step, arXiv preprint arXiv:2501.13926 (2025)

    2025 arXiv

  26. [35]

    H. Fei, S. Wu, W. Ji, H. Zhang, T.-S. Chua, Dysen-vdm: Empowering dynamics-aware text-to-video di ffusion with llms, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 7641–7653

    2024

  27. [36]

    J. Ji, R. Krishna, L. Fei-Fei, J. C. Niebles, Action genome: Actions as compositions of spatio-temporal scene graphs, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 10236–10247

    2020

  28. [37]

    Schneider, A

    S. Schneider, A. Baevski, R. Collobert, M. Auli, wav2vec: Unsupervised pre-training for speech recognition, arXiv preprint arXiv:1904.05862 (2019)

    2019 arXiv

  29. [38]

    D. P. Kingma, M. Welling, et al., Auto-encoding variational bayes (2013)

    2013

  30. [39]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al., Learning transferable visual models from natural language supervision, in: International conference on machine learning, PmLR, 2021, pp. 8748–8763

    2021

  31. [40]

    M. Xu, H. Li, Q. Su, H. Shang, L. Zhang, C. Liu, J. Wang, Y . Yao, S. Zhu, Hallo: Hierarchical audio-driven visual synthesis for portrait image animation, arXiv preprint arXiv:2406.08801 (2024)

    2024 arXiv

  32. [41]

    Blattmann, T

    A. Blattmann, T. Dockhorn, S. Kulal, D. Mendelevitch, M. Kilian, D. Lorenz, Y . Levi, Z. English, V . V oleti, A. Letts, et al., Stable video diffusion: Scaling latent video diffusion models to large datasets, arXiv preprint arXiv:2311.15127 (2023)

    2023 arXiv

  33. [42]

    Y . Guo, C. Yang, A. Rao, Z. Liang, Y . Wang, Y . Qiao, M. Agrawala, D. Lin, B. Dai, Animatediff: Animate your personalized text-to-image diffusion models without specific tuning, arXiv preprint arXiv:2307.04725 (2023)

    2023 arXiv

  34. [43]

    J. Ho, A. Jain, P. Abbeel, Denoising diffusion probabilistic models, Advances in neural information processing systems 33 (2020) 6840–6851

    2020

  35. [44]

    J. Song, C. Meng, S. Ermon, Denoising di ffusion implicit models, arXiv preprint arXiv:2010.02502 (2020)

    2020 arXiv

  36. [45]

    K. Wang, Q. Wu, L. Song, Z. Yang, W. Wu, C. Qian, R. He, Y . Qiao, C. C. Loy, Mead: A large-scale audio-visual dataset for emotional talking-face generation, in: European conference on computer vision, Springer, 2020, pp. 700–717

    2020

  37. [46]

    Zhang, L

    Z. Zhang, L. Li, Y . Ding, C. Fan, Flow-guided one-shot talking face generation with a high-resolution audio-visual dataset, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 3661–3670

    2021

  38. [47]

    Y . Zhou, X. Han, E. Shechtman, J. Echevarria, E. Kalogerakis, D. Li, Makelttalk: speaker-aware talking-head animation, ACM Transactions 24 On Graphics (TOG) 39 (6) (2020) 1–15

    2020

  39. [48]

    H. Zhou, Y . Sun, W. Wu, C. C. Loy, X. Wang, Z. Liu, Pose-controllable talking face generation by implicitly modularized audio-visual representation, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 4176–4186

    2021

  40. [49]

    Huynh-Thu, M

    Q. Huynh-Thu, M. Ghanbari, Scope of validity of psnr in image /video quality assessment, Electronics letters 44 (13) (2008) 800–801

    2008

  41. [50]

    Z. Wang, A. C. Bovik, H. R. Sheikh, E. P. Simoncelli, Image quality assessment: from error visibility to structural similarity, IEEE transactions on image processing 13 (4) (2004) 600–612

    2004

  42. [51]

    Seitzer, pytorch-fid: FID Score for PyTorch, https://github.com/mseitzer/pytorch-fid, version 0.3.0 (August 2020)

    M. Seitzer, pytorch-fid: FID Score for PyTorch, https://github.com/mseitzer/pytorch-fid, version 0.3.0 (August 2020)

    2020

  43. [52]

    N. D. Narvekar, L. J. Karam, A no-reference image blur metric based on the cumulative probability of blur detection (cpbd), IEEE Transac- tions on Image Processing 20 (9) (2011) 2678–2683. doi:10.1109/TIP.2011.2131660

    2011

  44. [53]

    J. S. Chung, A. Zisserman, Out of time: automated lip sync in the wild, in: Computer Vision–ACCV 2016 Workshops: ACCV 2016 International Workshops, Taipei, Taiwan, November 20-24, 2016, Revised Selected Papers, Part II 13, Springer, 2017, pp. 251–263

    2016

  45. [54]

    Afouras, J

    T. Afouras, J. S. Chung, A. Senior, O. Vinyals, A. Zisserman, Deep audio-visual speech recognition, IEEE transactions on pattern analysis and machine intelligence 44 (12) (2018) 8717–8727

    2018

  46. [55]

    Unterthiner, S

    T. Unterthiner, S. Van Steenkiste, K. Kurach, R. Marinier, M. Michalski, S. Gelly, Fvd: A new metric for video generation (2019)

    2019

  47. [56]

    Zhang, P

    R. Zhang, P. Isola, A. A. Efros, E. Shechtman, O. Wang, The unreasonable effectiveness of deep features as a perceptual metric, in: Proceed- ings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 586–595

    2018

  48. [57]

    Van der Maaten, G

    L. Van der Maaten, G. Hinton, Visualizing data using t-sne., Journal of machine learning research 9 (11) (2008)

    2008

  49. [58]

    J. Deng, J. Guo, X. Niannan, S. Zafeiriou, Arcface: Additive angular margin loss for deep face recognition, in: CVPR, 2019

    2019

  50. [59]

    Szegedy, S

    C. Szegedy, S. Io ffe, V . Vanhoucke, A. Alemi, Inception-v4, inception-resnet and the impact of residual connections on learning, in: Proceed- ings of the AAAI conference on artificial intelligence, V ol. 31, 2017

    2017

  51. [60]

    Peebles, S

    W. Peebles, S. Xie, Scalable di ffusion models with transformers, in: Proceedings of the IEEE /CVF international conference on computer vision, 2023, pp. 4195–4205. 25

    2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.