Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

EDTalk++: Full Disentanglement for Controllable Talking Head Synthesis

T0 review · 3 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read EDTalk++ decomposes a talking head into four orthogonal motion spaces — mouth, head pose, eyes, and expression — so each can be controlled separately and driven by either video or audio.

desk verdict Solid incremental extension of EDTalk with strong numbers and a genuinely new eye/diffusion stage, but the 'full disentanglement' claim outruns the evidence. read the letter →

arxiv 2508.13442 v1 pith:KFXX2NW2 submitted 2025-08-19 cs.CV

Shuai Tan , Bin Ji This is my paper
classification cs.CV
keywords talkingheadgenerationfacialdisentanglementorthogonallatentspacesaudio-drivensynthesisone-shotreenactmentemotionalexpressiondiffusionmodelnavigation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that the full space of facial motion in a talking head can be split into four disjoint, independently controllable latent spaces: mouth shape, head pose, eye movement, and emotional expression. If this claim holds, a single one-shot generator can animate an identity image by copying the mouth from one video, the pose from another, the gaze from a third, and the emotion from a fourth — or by inferring all four from audio alone. The authors argue that earlier methods fall short because they rely on external priors, lack constraints between spaces, or retrain heavy networks; EDTalk++ instead learns an orthogonal set of basis directions per component and stores them in banks. The practical payoff is efficient training and fine-grained, composable control over talking-head video.

What carries the argument

The load-bearing object is the orthogonal base bank. Each of the four Component-aware Latent Navigation (CLN) modules maintains a bank of learnable vectors; a per-component MLP predicts weights $W^{*}$, and the component's motion feature is $f^{r\to *}=\sum_i w_i^{*} b_i^{*}$. Imposing $\langle b_i^{*}, b_j^{*}\rangle=0$ within a bank and $\langle b_i^{*}, b_j^{\diamond}\rangle=0$ across banks guarantees that the four components occupy orthogonal subspaces of the latent code, which is what makes additive recombination $f^{i\to d}=f^{i\to r}+f^{r\to p}+f^{r\to m}+f^{r\to ey}+f^{r\to ex}$ preserve independent control. The training strategy then uses cross-reconstruction and self-reconstruction

What would settle it

Take frames from HDTF or VFHQ that the pipeline treats as neutral and run an independent expression classifier on them; if those frames carry systematic expression scores that correlate with the learned mouth or eye bank weights, the neutrality premise fails. More directly, build a paired test set in which only the mouth source changes while pose, eye, and expression sources are frozen, and check whether the generated head pose or expression changes: any measurable movement in the supposedly frozen components refutes complete disentanglement.

Watch

Extended reading notes

Core claim

The central claim is that facial dynamics can be fully separated into four component-aware latent spaces — mouth ($B^{m}$), pose ($B^{p}$), eye ($B^{ey}$), and expression ($B^{ex}$) — each spanned by learnable bases. Enforcing orthogonality both within each bank and across banks makes the spaces disjoint, so the final driving feature is the simple sum $f^{r\to d}=f^{r\to p}+f^{r\to m}+f^{r\to ey}+f^{r\to ex}$. A progressive training scheme assigns responsibility to each space in three stages: first head pose is separated from the face region via cross-reconstruction, then mouth and eye are separated by swapping eye regions, and finally expression is learned by self-reconstruction from what t

Load-bearing premise

The load-bearing premise is that the HDTF and VFHQ training videos are truly emotion-neutral, so the pose, mouth, and eye banks never absorb expression-related information during the first two training stages; if expression leaks into those banks, the later expression stage cannot make the four spaces fully disjoint.

Editorial extensions

If this is right

  • With the four spaces disjoint, a user can combine any subset of facial motions — mouth from the audio, pose from one clip, expression from another — in one synchronized output.
  • The same banks trained from video transfer to audio-only input; only the weight predictors need to be trained, which the paper reports in hours rather than weeks.
  • Expression can be generated from the tone of voice and transcript, removing the need for a separate emotional reference video.
  • Because the bases are semantically meaningful, interpolating two expression weight vectors produces gradual expression morphing between two clips.
  • Freezing trained modules means adding a new component does not require retraining the whole generator, only the new lightweight module.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Extension beyond the paper: one could test perceptual independence directly by sweeping one bank's weights while holding the other three fixed and measuring whether any change appears in the supposedly frozen components; latent orthogonality does not by itself guarantee perceptual independence after the nonlinear generator.
  • The additive bank structure suggests a reusable component library: the mouth, pose, eye, and expression banks could in principle be attached to a different renderer, since they are consumed as additive latent offsets — though the paper does not demonstrate this.
  • A concrete stress test: strip the expression from an emotionally expressive video via the neutral reconstruction and check whether the mouth bank still carries upper-face or brow information; if it does, the claimed disjointness is only approximate.
  • The paper's own limitation note points to a natural extension: conditioning the pose/eye diffusion on emotion labels once datasets with emotion-annotated head motion exist would close the loop between expression and pose.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces EDTalk++, a one-shot talking-head generator that aims to fully disentangle four facial-motion spaces—mouth shape, head pose, eye movement, and emotional expression—and to support both video-driven and audio-driven control. The method represents each motion space as a linear combination of learnable orthogonal bases stored in banks (Sec. III-A), and trains four lightweight navigation modules through a three-stage progressive scheme: Face-Pose Decouple, Mouth-Eye Decouple, and Expression Decouple (Sec. III-B). An Audio-to-Motion module then predicts bank weights from audio, optionally conditioned on transcripts, for audio-driven synthesis (Sec. III-C). Extensive experiments on MEAD, HDTF, VFHQ, Voxceleb2, and LRW report state-of-the-art or competitive scores on image quality, lip-sync, and emotion accuracy, together with user studies and efficiency comparisons.

Significance. If the central claim of complete decoupling is correct, EDTalk++ would be a practically valuable unified framework: it offers independent control of four facial components, additive composition of their features, and shared visual priors across video and audio modalities, while requiring only lightweight training of the disentanglement modules. The paper's strengths include consistently strong quantitative results across multiple datasets (Tables I, II, and IV), a clear and efficient training design, and a user study (Table III). However, the strongest contribution claim—'complete decoupling of these four spaces'—is currently supported mainly by qualitative manipulation examples rather than by quantitative disentanglement metrics. The manuscript also relies on an unverified dataset-neutrality assumption that is load-bearing for the training strategy. These issues need to be addressed before the central claim can be accepted.

major comments (3)
  1. [Sec. I, Sec. IV-C, Figs. 9-12] The central claim of 'complete decoupling' is not quantitatively tested. The evidence consists of selected qualitative manipulations (mouth/pose/eye/expression controls and base-wise edits) and ablation images (Fig. 14), but there is no metric measuring whether changing one component leaves the other three invariant. A concrete protocol should be added: e.g., fix identity and pose/eye/expression sources, drive mouth from target A, and report mouth landmark distance to A alongside pose/eye/expression distances to their fixed sources; repeat for each component. Without such non-interference measurements, the additive decomposition f_r→d = f_r→p + f_r→m + f_r→ey + f_r→ex (Sec. III-A) is not established.
  2. [Sec. IV-B, Sec. III-B] The statement that HDTF and VFHQ 'consistently exhibit a Neutral emotion' is used to justify training PLN, MLN, and EYLN exclusively on these datasets and then freezing them. No quantitative evidence for this neutrality is provided. If expression-correlated variation (smiles, eyebrow raises, squints) is present in these YouTube/interview videos, the cross-reconstruction and eye-swap losses will assign that variation to pose, mouth, or eye banks, because no emotion-neutrality or expression-invariance constraint is imposed in the first two stages. The later Expression Decouple stage (Sec. III-B.c) trains only on residuals and cannot remove expression information already absorbed into the frozen banks. Please verify neutrality quantitatively (e.g., emotion classifier/action-unit distribution on the actual training subsets) and, ideally, directly measure expression leakage by testing whether
  3. [Sec. III-C, Eq. (14), L_exp] The audio-to-motion modules are trained to regress weight vectors produced by the paper's own pretrained modules: W_m = MLN(E(I)) in Eq. (14), and similarly W_ex from pretrained EXLN for L_exp. This is self-distillation: it can teach the audio module to imitate the visual module, but it cannot by itself validate that the target spaces contain the intended semantic content. The external SyncNet and Emotion-Fan metrics partially mitigate this concern for lip-sync and expression accuracy, but the training loss for expression uses W_ex from the same model as ground truth. If EXLN has missed or leaked expression information, the audio expression module will inherit the defect. Please add an independent check, e.g., training with MEAD emotion labels or action units as auxiliary targets, or reporting agreement between audio-predicted weights and annotations on a held-out set.
minor comments (6)
  1. [Tables I and II] Several table entries are visually corrupted or hard to parse; for example, the EDTalk++ row in Table I shows '1.213/1.1960 15.6798.0368.0368.036 68.21'. Please regenerate the tables with unambiguous column separation and verify the numeric formatting.
  2. [Sec. III-B.b] Typo: 'bn training' should be 'by training'.
  3. [Sec. III-B.c] Typo: 'EXN L' should be 'EXLN'.
  4. [Sec. V-c] The limitation paragraph contains a duplicate 'Second, Second' and the cross-reference 'as depicted in Eq. (13)' appears to be incorrect; Eq. (13) is the mouth consistency loss, not the proposed extension with emotion label e.
  5. [Sec. I] The claim of disentanglement 'without relying on external knowledge' is overstated: the pipeline uses a pretrained LIA encoder/generator, VGG19, EMOCA, SyncNet, HuBERT, and EmoBERTa. The novelty lies in not using external labels for the bank decomposition, and the text should say so explicitly.
  6. [Sec. IV-C.c] The user study is based on 10 videos per method and 20 participants. Please report confidence intervals or significance tests for the differences in Table III, since the score gaps for some methods are small.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: EDTalk++'s disentanglement and audio-to-motion pipelines are self-contained and externally validated.

full rationale

EDTalk++ does not exhibit circular reasoning. The four latent spaces are learned from video via cross-reconstruction and self-reconstruction losses (Eqs. 6-13) with the encoder/generator frozen; the expression bank is learned as the residual after pose/mouth/eye banks, which is a model design choice rather than a renaming of the target. The Audio-to-Motion modules (Eqs. 14-16) regress weight vectors produced by the paper's own CLN modules, but this is self-distillation rather than circularity: the training targets are intermediate representations, and the final claims are evaluated with external SyncNet, M-LMD, Emotion-Fan, and human studies. The self-citations ([1], [42], [96], [97]) are conference-version/baseline references, not load-bearing. The main weakness is the unverified 'Neutral' assumption for HDTF/VFHQ (Sec. IV-B: 'videos consistently exhibit a Neutral emotion'), which is a correctness/robustness risk for the full-disentanglement claim, not a circular step.

Assumptions & free parameters 4 free parameters · 4 assumptions · 2 invented entities

The framework rests on an additivity and orthogonality assumption about facial latent spaces, plus a strong dataset-neutrality assumption for the first two training stages. The free parameters (bank sizes, loss weights, orthogonality strength) are hand-chosen or unreported. The banks and canonical feature are internal constructs without external falsifiable handles.

free parameters (4)
  • Bank sizes (B_m=20, B_p=6, B_ey=10, B_ex=10) = 20, 6, 10, 10
    Hand-tuned; Table V explores 5/10/20/40 and picks 20 for mouth and 10 for expression; these capacities directly determine what each disentangled space can express.
  • Loss weight for L_mot = 10
    Set in Sec. IV-B; all other loss weights set to 1; not sensitivity-analyzed.
  • Latent and base dimension = 512
    Chosen in Sec. IV-B without justification; affects the capacity of the disentangled spaces.
  • Orthogonality regularization strength = unreported
    Sec. III-A states orthogonality is enforced but never states the loss term or weight, making the actual fitting unquantifiable.
assumptions (4)
  • domain assumption Facial motion space can be decomposed as a linear sum of four mutually orthogonal bank subspaces, and the generator can combine them additively.
    Sec. III-A states f_r->d = f_r->p + f_r->m + f_r->ey + f_r->ex; no proof that real facial dynamics satisfy linearity or that the generator's latent space is additive.
  • domain assumption HDTF and VFHQ datasets are emotion-neutral, so training on them extracts pose/mouth/eye without expression contamination.
    Used in Sec. IV-B to train Face-Pose and Mouth-Eye decoupling; the entire expression decoupling stage depends on these banks being expression-free.
  • domain assumption Expression features change slowly, so averaging K frames yields a clean expression feature.
    Sec. III-B-c states 'we define a window of size K around I_d and average K extracted expression features'; K is never specified, and this is the core mechanism of the Expression Decouple stage.
  • domain assumption Pretrained EMOCA features (Sec. III-B-c, Eq. 12) provide reliable motion supervision for expression decoupling.
    L_mot uses the 3D face reconstruction and emotion networks of [33]; this external prior is loaded into the training but not questioned.
invented entities (2)
  • Orthogonal basis banks B_m, B_p, B_ey, B_ex
    purpose: Represent each facial motion component as a linear combination of learned orthogonal bases, enabling disjoint control and reuse for audio-driven synthesis.
    These are internal learned parameters with no external validation; the semantic meaning of each basis is demonstrated only through qualitative examples (Fig. 12).
  • Canonical feature f_r
    purpose: An intermediate latent feature such that f_i_to_* = f_i_to_r + f_r_to_* (Sec. III-A).
    Inherited from LIA [34] and assumed; not independently verified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of EDTalk++: Full Disentanglement for Controllable Talking Head Synthesis." pith.science (2026). https://pith.science/paper/KFXX2NW2

@misc{pith2026250813442,
  author       = {Pith},
  title        = {Pith review of: EDTalk++: Full Disentanglement for Controllable Talking Head Synthesis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KFXX2NW2}},
  note         = {Machine review of arXiv:2508.13442}
}
read the original abstract

Achieving disentangled control over multiple facial motions and accommodating diverse input modalities greatly enhances the application and entertainment of the talking head generation. This necessitates a deep exploration of the decoupling space for facial features, ensuring that they a) operate independently without mutual interference and b) can be preserved to share with different modal inputs, both aspects often neglected in existing methods. To address this gap, this paper proposes EDTalk++, a novel full disentanglement framework for controllable talking head generation. Our framework enables individual manipulation of mouth shape, head pose, eye movement, and emotional expression, conditioned on video or audio inputs. Specifically, we employ four lightweight modules to decompose the facial dynamics into four distinct latent spaces representing mouth, pose, eye, and expression, respectively. Each space is characterized by a set of learnable bases whose linear combinations define specific motions. To ensure independence and accelerate training, we enforce orthogonality among bases and devise an efficient training strategy to allocate motion responsibilities to each space without relying on external knowledge. The learned bases are then stored in corresponding banks, enabling shared visual priors with audio input. Furthermore, considering the properties of each space, we propose an Audio-to-Motion module for audio-driven talking head synthesis. Experiments are conducted to demonstrate the effectiveness of EDTalk++.

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. PortraitDirector: A Hierarchical Disentanglement Framework for Controllable and Real-time Facial Reenactment

    cs.CV 2026-04 unverdicted novelty 5.0 of 10

    PortraitDirector uses hierarchical disentanglement of spatial physical motions and semantic emotions to deliver controllable, high-fidelity real-time facial reenactment at 20 FPS.

Reference graph

Works this paper leans on

111 extracted references · 49 canonical work pages · cited by 1 Pith paper

  1. [1]

    Edtalk: Efficient disentanglement for emotional talking head synthesis,

    S. Tan, B. Ji, M. Bi, and Y . Pan, “Edtalk: Efficient disentanglement for emotional talking head synthesis,” inEuropean Conference on Computer Vision. Springer, 2024, pp. 398–416

    2024

  2. [2]

    Style2talker: High-resolution talking head generation with emotion style and art style,

    S. Tan, B. Ji, and Y . Pan, “Style2talker: High-resolution talking head generation with emotion style and art style,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 5, 2024, pp. 5079–5087

    2024

  3. [3]

    Emotional voice puppetry,

    Y . Pan, R. Zhang, S. Cheng, S. Tan, Y . Ding, K. Mitchell, and X. Yang, “Emotional voice puppetry,”IEEE Transactions on Visualization and Computer Graphics, vol. 29, no. 5, pp. 2527–2535, 2023

    2023

  4. [4]

    Expressive talking avatars,

    Y . Pan, S. Tan, S. Cheng, Q. Lin, Z. Zeng, and K. Mitchell, “Expressive talking avatars,”IEEE Transactions on Visualization and Computer Graphics, 2024

    2024

  5. [5]

    Vasa-rig: Audio-driven 3d facial animation with ‘live’mood dynamics in virtual reality,

    Y . Pan, C. Liu, S. Xu, S. Tan, and J. Yang, “Vasa-rig: Audio-driven 3d facial animation with ‘live’mood dynamics in virtual reality,”IEEE Transactions on Visualization and Computer Graphics, 2025

    2025

  6. [6]

    Cnn-based real-time dense face reconstruction with inverse-rendered photo-realistic face images,

    Y . Guo, J. Cai, B. Jiang, J. Zhenget al., “Cnn-based real-time dense face reconstruction with inverse-rendered photo-realistic face images,” IEEE transactions on pattern analysis and machine intelligence, vol. 41, no. 6, pp. 1294–1307, 2018

    2018

  7. [7]

    Styletalk++: A unified framework for controlling the speaking styles of talking heads,

    S. Wang, Y . Ma, Y . Ding, Z. Hu, C. Fan, T. Lv, Z. Deng, and X. Yu, “Styletalk++: A unified framework for controlling the speaking styles of talking heads,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 46, no. 6, pp. 4331–4347, 2024

    2024

  8. [8]

    Dagan++: Depth-aware generative adversarial network for talking head video generation,

    F.-T. Hong, L. Shen, and D. Xu, “Dagan++: Depth-aware generative adversarial network for talking head video generation,”IEEE Transac- tions on Pattern Analysis and Machine Intelligence, vol. 46, no. 5, pp. 2997–3012, 2023

    2023

Show all 111 references
  1. [9]

    Metaearth: A generative foundation model for global-scale remote sensing image generation,

    Z. Yu, C. Liu, L. Liu, Z. Shi, and Z. Zou, “Metaearth: A generative foundation model for global-scale remote sensing image generation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    2024

  2. [10]

    Fixtalk: Taming identity leakage for high-quality talking head generation in extreme cases,

    S. Tan, B. Gong, B. Ji, and Y . Pan, “Fixtalk: Taming identity leakage for high-quality talking head generation in extreme cases,”arXiv preprint arXiv:2507.01390, 2025

    2025 arXiv

  3. [11]

    Pomp: Physics- consistent motion generative model through phase manifolds,

    B. Ji, Y . Pan, Z. Liu, S. Tan, X. Jin, and X. Yang, “Pomp: Physics- consistent motion generative model through phase manifolds,” inPro- ceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 22 690–22 701

    2025

  4. [12]

    Animate-x++: Universal character image animation with dynamic backgrounds,

    S. Tan, B. Gong, Z. Liu, Y . Wang, X. Chen, Y . Feng, and H. Zhao, “Animate-x++: Universal character image animation with dynamic backgrounds,”arXiv preprint arXiv:2508.09454, 2025

    2025 arXiv

  5. [13]

    Synmotion: Semantic-visual adaptation for motion customized video generation,

    S. Tan, B. Gong, Y . Wei, S. Zhang, Z. Liu, D. Zheng, J. Chen, Y . Wang, H. Ouyang, K. Zhenget al., “Synmotion: Semantic-visual adaptation for motion customized video generation,”arXiv preprint arXiv:2506.23690, 2025

    2025 arXiv

  6. [14]

    Dreamrelation: Relation-centric video customization,

    Y . Wei, S. Zhang, H. Yuan, B. Gong, L. Tang, X. Wang, H. Qiu, H. Li, S. Tan, Y . Zhanget al., “Dreamrelation: Relation-centric video customization,”arXiv preprint arXiv:2503.07602, 2025

    2025 arXiv

  7. [15]

    Vqtalker: Towards multilingual talking avatars through facial motion tokenization,

    T. Liu, Z. Ma, Q. Chen, F. Chen, S. Fan, X. Chen, and K. Yu, “Vqtalker: Towards multilingual talking avatars through facial motion tokenization,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 39, no. 6, 2025, pp. 5586–5594

    2025

  8. [16]

    Ai-generated characters for supporting person- alized learning and well-being,

    P. Pataranutaporn, V . Danry, J. Leong, P. Punpongsanon, D. Novy, P. Maes, and M. Sra, “Ai-generated characters for supporting person- alized learning and well-being,”Nature Machine Intelligence, vol. 3, no. 12, pp. 1013–1022, 2021

    2021

  9. [17]

    Learning dynamic facial radiance fields for few-shot talking head synthesis,

    S. Shen, W. Li, Z. Zhu, Y . Duan, J. Zhou, and J. Lu, “Learning dynamic facial radiance fields for few-shot talking head synthesis,” inEuropean Conference on Computer Vision. Springer, 2022, pp. 666–682

    2022

  10. [18]

    Real- istic one-shot mesh-based head avatars,

    T. Khakhulin, V . Sklyarova, V . Lempitsky, and E. Zakharov, “Real- istic one-shot mesh-based head avatars,” inEuropean Conference on Computer Vision. Springer, 2022, pp. 345–362

    2022

  11. [19]

    Face2faceρ: Real-time high-resolution one-shot face reenactment,

    K. Yang, K. Chen, D. Guo, S.-H. Zhang, Y .-C. Guo, and W. Zhang, “Face2faceρ: Real-time high-resolution one-shot face reenactment,” in European conference on computer vision. Springer, 2022, pp. 55–71

    2022

  12. [20]

    Styleheat: One-shot high-resolution editable talking face generation via pre-trained stylegan,

    F. Yin, Y . Zhang, X. Cun, M. Cao, Y . Fan, X. Wang, Q. Bai, B. Wu, J. Wang, and Y . Yang, “Styleheat: One-shot high-resolution editable talking face generation via pre-trained stylegan,” inEuropean conference on computer vision. Springer, 2022, pp. 85–101

    2022

  13. [21]

    Progressive disentangled representation learning for fine-grained controllable talk- ing head synthesis,

    D. Wang, Y . Deng, Z. Yin, H.-Y . Shum, and B. Wang, “Progressive disentangled representation learning for fine-grained controllable talk- ing head synthesis,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 17 979–17 989

    2023

  14. [22]

    Anitalker: Animate vivid and diverse talking faces through identity- decoupled facial motion encoding,

    T. Liu, F. Chen, S. Fan, C. Du, Q. Chen, X. Chen, and K. Yu, “Anitalker: Animate vivid and diverse talking faces through identity- decoupled facial motion encoding,”arXiv preprint arXiv:2405.03121, 2024

    2024 arXiv

  15. [23]

    Neural voice puppetry: Audio-driven facial reenactment,

    J. Thies, M. Elgharib, A. Tewari, C. Theobalt, and M. Nießner, “Neural voice puppetry: Audio-driven facial reenactment,” inComputer Vision– ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVI 16. Springer, 2020, pp. 716–731

    2020

  16. [24]

    First order motion model for image animation,

    A. Siarohin, S. Lathuili `ere, S. Tulyakov, E. Ricci, and N. Sebe, “First order motion model for image animation,”Advances in neural information processing systems, vol. 32, 2019

    2019

  17. [25]

    Depth-aware generative ad- versarial network for talking head video generation,

    F.-T. Hong, L. Zhang, L. Shen, and D. Xu, “Depth-aware generative ad- versarial network for talking head video generation,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 3397–3406

    2022

  18. [26]

    Efficient emotional adaptation for audio-driven talking-head generation,

    Y . Gan, Z. Yang, X. Yue, L. Sun, and Y . Yang, “Efficient emotional adaptation for audio-driven talking-head generation,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 22 634–22 645

    2023

  19. [27]

    Facial action coding system,

    P. Ekman and W. V . Friesen, “Facial action coding system,”Environ- mental Psychology & Nonverbal Behavior, 1978

    1978

  20. [28]

    Talking head generation with probabilistic audio-to-visual diffusion priors,

    Z. Yu, Z. Yin, D. Zhou, D. Wang, F. Wong, and B. Wang, “Talking head generation with probabilistic audio-to-visual diffusion priors,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 7645–7655

    2023

  21. [29]

    Expressive talking head generation with granular audio-visual control,

    B. Liang, Y . Pan, Z. Guo, H. Zhou, Z. Hong, X. Han, J. Han, J. Liu, E. Ding, and J. Wang, “Expressive talking head generation with granular audio-visual control,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 3387–3396

    2022

  22. [30]

    Dpe: Disentanglement of pose and expression for general video portrait editing,

    Y . Pang, Y . Zhang, W. Quan, Y . Fan, X. Cun, Y . Shan, and D.- m. Yan, “Dpe: Disentanglement of pose and expression for general video portrait editing,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 427–436

    2023

  23. [31]

    Pose- controllable talking face generation by implicitly modularized audio- visual representation,

    H. Zhou, Y . Sun, W. Wu, C. C. Loy, X. Wang, and Z. Liu, “Pose- controllable talking face generation by implicitly modularized audio- visual representation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 4176–4186

    2021

  24. [32]

    Supervised contrastive learning,

    P. Khosla, P. Teterwak, C. Wang, A. Sarna, Y . Tian, P. Isola, A. Maschinot, C. Liu, and D. Krishnan, “Supervised contrastive learning,”Advances in neural information processing systems, vol. 33, pp. 18 661–18 673, 2020

    2020

  25. [33]

    Emoca: Emotion driven monocular face capture and animation,

    R. Dan ˇeˇcek, M. J. Black, and T. Bolkart, “Emoca: Emotion driven monocular face capture and animation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 20 311–20 322

    2022

  26. [34]

    Latent image animator: Learning to animate images via latent space navigation,

    Y . Wang, D. Yang, F. Bremond, and A. Dantcheva, “Latent image animator: Learning to animate images via latent space navigation,” in International Conference on Learning Representations, 2021

    2021

  27. [35]

    Sadtalker: Learning realistic 3d motion coefficients for stylized audio-driven single image talking face animation,

    W. Zhang, X. Cun, X. Wang, Y . Zhang, X. Shen, Y . Guo, Y . Shan, and F. Wang, “Sadtalker: Learning realistic 3d motion coefficients for stylized audio-driven single image talking face animation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni...

    2023

  28. [36]

    Denoising diffusion probabilistic models,

    J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,”NeurIPS, vol. 33, pp. 6840–6851, 2020

    2020

  29. [37]

    Denoising diffusion implicit mod- els,

    J. Song, C. Meng, and S. Ermon, “Denoising diffusion implicit mod- els,” inICLR, 2021

    2021

  30. [38]

    Animate-x: Universal character image animation with enhanced motion representation,

    S. Tan, B. Gong, X. Wang, S. Zhang, D. Zheng, R. Zheng, K. Zheng, J. Chen, and M. Yang, “Animate-x: Universal character image animation with enhanced motion representation,”arXiv preprint arXiv:2410.10306, 2024

    2024 arXiv

  31. [39]

    Mimir: Improving video diffusion models for precise text understanding,

    S. Tan, B. Gong, Y . Feng, K. Zheng, D. Zheng, S. Shi, Y . Shen, J. Chen, and M. Yang, “Mimir: Improving video diffusion models for precise text understanding,”arXiv preprint arXiv:2412.03085, 2024

    2024 arXiv

  32. [40]

    Sport: From zero- shot prompts to real-time motion generation,

    B. Ji, Y . Pan, Z. Liu, S. Tan, and X. Yang, “Sport: From zero- shot prompts to real-time motion generation,”IEEE Transactions on Visualization and Computer Graphics, 2025

    2025

  33. [41]

    Audio- driven emotional video portraits,

    X. Ji, H. Zhou, K. Wang, W. Wu, C. C. Loy, X. Cao, and F. Xu, “Audio- driven emotional video portraits,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 14 080–14 089

    2021

  34. [42]

    Emmn: Emotional motion memory network for audio-driven emotional talking face generation,

    S. Tan, B. Ji, and Y . Pan, “Emmn: Emotional motion memory network for audio-driven emotional talking face generation,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 22 146–22 156. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 16

    2023

  35. [43]

    Ae-nerf: Audio enhanced neural radiance field for few shot talking head synthesis,

    D. Li, K. Zhao, W. Wang, B. Peng, Y . Zhang, J. Dong, and T. Tan, “Ae-nerf: Audio enhanced neural radiance field for few shot talking head synthesis,”arXiv preprint arXiv:2312.10921, 2023

    2023 arXiv

  36. [44]

    Momentum contrast for unsupervised visual representation learning,

    K. He, H. Fan, Y . Wu, S. Xie, and R. Girshick, “Momentum contrast for unsupervised visual representation learning,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 9729–9738

    2020

  37. [45]

    Video rewrite: Driving visual speech with audio,

    C. Bregler, M. Covell, and M. Slaney, “Video rewrite: Driving visual speech with audio,” inSeminal Graphics Papers: Pushing the Bound- aries, Volume 2, 2023, pp. 715–722

    2023

  38. [46]

    Semantic-aware implicit neural audio-driven video portrait generation,

    X. Liu, Y . Xu, Q. Wu, H. Zhou, W. Wu, and B. Zhou, “Semantic-aware implicit neural audio-driven video portrait generation,” inEuropean Conference on Computer Vision. Springer, 2022, pp. 106–125

    2022

  39. [47]

    Emo: Emote portrait alive- generating expressive portrait videos with audio2video diffusion model under weak conditions,

    L. Tian, Q. Wang, B. Zhang, and L. Bo, “Emo: Emote portrait alive- generating expressive portrait videos with audio2video diffusion model under weak conditions,”arXiv preprint arXiv:2402.17485, 2024

    2024 arXiv

  40. [48]

    Vasa-1: Lifelike audio-driven talking faces generated in real time,

    S. Xu, G. Chen, Y .-X. Guo, J. Yang, C. Li, Z. Zang, Y . Zhang, X. Tong, and B. Guo, “Vasa-1: Lifelike audio-driven talking faces generated in real time,”arXiv preprint arXiv:2404.10667, 2024

    2024 arXiv

  41. [49]

    Hallo2: Long-duration and high-resolution audio-driven portrait image animation,

    J. Cui, H. Li, Y . Yao, H. Zhu, H. Shang, K. Cheng, H. Zhou, S. Zhu, and J. Wang, “Hallo2: Long-duration and high-resolution audio-driven portrait image animation,”arXiv preprint arXiv:2410.07718, 2024

    2024 arXiv

  42. [50]

    Hallo3: Highly dynamic and realistic portrait image animation with diffusion transformer networks,

    J. Cui, H. Li, Y . Zhan, H. Shang, K. Cheng, Y . Ma, S. Mu, H. Zhou, J. Wang, and S. Zhu, “Hallo3: Highly dynamic and realistic portrait image animation with diffusion transformer networks,”arXiv preprint arXiv:2412.00733, 2024

    2024 arXiv

  43. [51]

    Echomimicv2: Towards striking, simplified, and semi-body human animation,

    R. Meng, X. Zhang, Y . Li, and C. Ma, “Echomimicv2: Towards striking, simplified, and semi-body human animation,”arXiv preprint arXiv:2411.10061, 2024

    2024

  44. [52]

    Cyberhost: Taming audio-driven avatar diffusion model with region codebook attention,

    G. Lin, J. Jiang, C. Liang, T. Zhong, J. Yang, and Y . Zheng, “Cyberhost: Taming audio-driven avatar diffusion model with region codebook attention,”arXiv preprint arXiv:2409.01876, 2024

    2024 arXiv

  45. [53]

    Loopy: Taming audio-driven portrait avatar with long-term motion depen- dency,

    J. Jiang, C. Liang, J. Yang, G. Lin, T. Zhong, and Y . Zheng, “Loopy: Taming audio-driven portrait avatar with long-term motion depen- dency,”arXiv preprint arXiv:2409.02634, 2024

    2024 arXiv

  46. [54]

    Mimaface: Face animation via motion-identity modulated appearance feature learning,

    Y . Han, J. Zhu, Y . Feng, X. Ji, K. He, X. Li, Y . Liuet al., “Mimaface: Face animation via motion-identity modulated appearance feature learning,”arXiv preprint arXiv:2409.15179, 2024

    2024 arXiv

  47. [55]

    Aniportrait: Audio-driven synthesis of photorealistic portrait animation,

    H. Wei, Z. Yang, and Z. Wang, “Aniportrait: Audio-driven synthesis of photorealistic portrait animation,”arXiv preprint arXiv:2403.17694, 2024

    2024 arXiv

  48. [56]

    Makelttalk: speaker-aware talking-head animation,

    Y . Zhou, X. Han, E. Shechtman, J. Echevarria, E. Kalogerakis, and D. Li, “Makelttalk: speaker-aware talking-head animation,”ACM Transactions On Graphics (TOG), vol. 39, no. 6, pp. 1–15, 2020

    2020

  49. [57]

    Hierarchical cross-modal talking face generation with dynamic pixel-wise loss,

    L. Chen, R. K. Maddox, Z. Duan, and C. Xu, “Hierarchical cross-modal talking face generation with dynamic pixel-wise loss,” inProceedings of the IEEE/CVF conference on computer vision and pattern recogni- tion, 2019, pp. 7832–7841

    2019

  50. [58]

    Speech-driven facial animation using cascaded gans for learning of motion and texture,

    D. Das, S. Biswas, S. Sinha, and B. Bhowmick, “Speech-driven facial animation using cascaded gans for learning of motion and texture,” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXX 16. Springer, 2020, pp. 408–424

    2020

  51. [59]

    Few- shot adversarial learning of realistic neural talking head models,

    E. Zakharov, A. Shysheya, E. Burkov, and V . Lempitsky, “Few- shot adversarial learning of realistic neural talking head models,” in Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 9459–9468

    2019

  52. [60]

    Identity-preserving talking face generation with landmark and appear- ance priors,

    W. Zhong, C. Fang, Y . Cai, P. Wei, G. Zhao, L. Lin, and G. Li, “Identity-preserving talking face generation with landmark and appear- ance priors,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 9729–9738

    2023

  53. [61]

    Audio2head: Audio- driven one-shot talking-head generation with natural head motion,

    S. Wang, L. Li, Y . Ding, C. Fan, and X. Yu, “Audio2head: Audio- driven one-shot talking-head generation with natural head motion,” in International Joint Conference on Artificial Intelligence. IJCAI, 2021

    2021

  54. [62]

    One-shot talking face generation from single-speaker audio-visual correlation learning,

    S. Wang, L. Li, Y . Ding, and X. Yu, “One-shot talking face generation from single-speaker audio-visual correlation learning,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 3, 2022, pp. 2531–2539

    2022

  55. [63]

    Talking- head generation with rhythmic head motion,

    L. Chen, G. Cui, C. Liu, Z. Li, Z. Kou, Y . Xu, and C. Xu, “Talking- head generation with rhythmic head motion,” inEuropean Conference on Computer Vision. Springer, 2020, pp. 35–51

    2020

  56. [64]

    A morphable model for the synthesis of 3d faces,

    V . Blanz and T. Vetter, “A morphable model for the synthesis of 3d faces,” inProceedings of the 26th annual conference on Computer graphics and interactive techniques, 1999, pp. 187–194

    1999

  57. [65]

    Lip movements generation at a glance,

    L. Chen, Z. Li, R. K. Maddox, Z. Duan, and C. Xu, “Lip movements generation at a glance,” inProceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 520–535

    2018

  58. [66]

    Talking face generation by conditional recurrent adversarial network,

    Y . Song, J. Zhu, D. Li, A. Wang, and H. Qi, “Talking face generation by conditional recurrent adversarial network,” in Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, Aug 2019. [Online]. Available: http: //dx.doi.org/10.24963/ijcai.2019/129

    2019 doi

  59. [67]

    Talking face generation by adversarially disentangled audio-visual representation,

    H. Zhou, Y . Liu, Z. Liu, P. Luo, and X. Wang, “Talking face generation by adversarially disentangled audio-visual representation,” inProceed- ings of the AAAI conference on artificial intelligence, vol. 33, no. 01, 2019, pp. 9299–9306

    2019

  60. [68]

    Vast: Vivify your talking avatar via zero-shot expressive facial style transfer,

    L. Chen, Z. Wu, R. Li, W. Bao, J. Ling, X. Tan, and S. Zhao, “Vast: Vivify your talking avatar via zero-shot expressive facial style transfer,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 2977–2987

    2023

  61. [69]

    Lipformer: High-fidelity and generalizable talking face generation with a pre-learned facial codebook,

    J. Wang, K. Zhao, S. Zhang, Y . Zhang, Y . Shen, D. Zhao, and J. Zhou, “Lipformer: High-fidelity and generalizable talking face generation with a pre-learned facial codebook,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 13 844–13 853

    2023

  62. [70]

    Difftalk: Crafting diffusion models for generalized audio-driven portraits anima- tion,

    S. Shen, W. Zhao, Z. Meng, W. Li, Z. Zhu, J. Zhou, and J. Lu, “Difftalk: Crafting diffusion models for generalized audio-driven portraits anima- tion,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 1982–1991

    2023

  63. [71]

    A lip sync expert is all you need for speech to lip generation in the wild,

    K. Prajwal, R. Mukhopadhyay, V . P. Namboodiri, and C. Jawahar, “A lip sync expert is all you need for speech to lip generation in the wild,” inProceedings of the 28th ACM International Conference on Multimedia, 2020, pp. 484–492

    2020

  64. [72]

    Emotion- controllable generalized talking face generation,

    S. Sinha, S. Biswas, R. Yadav, and B. Bhowmick, “Emotion- controllable generalized talking face generation,” inInternational Joint Conference on Artificial Intelligence. IJCAI, 2021

    2021

  65. [73]

    Mead: A large-scale audio-visual dataset for emotional talking-face generation,

    K. Wang, Q. Wu, L. Song, Z. Yang, W. Wu, C. Qian, R. He, Y . Qiao, and C. C. Loy, “Mead: A large-scale audio-visual dataset for emotional talking-face generation,” inComputer Vision–ECCV 2020: 16th Euro- pean Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXI. ...

    2020

  66. [74]

    Eamm: One-shot emotional talking face via audio-based emotion-aware motion model,

    X. Ji, H. Zhou, K. Wang, Q. Wu, W. Wu, F. Xu, and X. Cao, “Eamm: One-shot emotional talking face via audio-based emotion-aware motion model,” inACM SIGGRAPH 2022 Conference Proceedings, 2022, pp. 1–10

    2022

  67. [75]

    Styletalk: One-shot talking head generation with controllable speaking styles,

    Y . Ma, S. Wang, Z. Hu, C. Fan, T. Lv, Y . Ding, Z. Deng, and X. Yu, “Styletalk: One-shot talking head generation with controllable speaking styles,”arXiv preprint arXiv:2301.01081, 2023

    2023 arXiv

  68. [76]

    Flow-guided one-shot talking face generation with a high-resolution audio-visual dataset,

    Z. Zhang, L. Li, Y . Ding, and C. Fan, “Flow-guided one-shot talking face generation with a high-resolution audio-visual dataset,” inPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 3661–3670

    2021

  69. [77]

    Perceptual losses for real-time style transfer and super-resolution,

    J. Johnson, A. Alahi, and L. Fei-Fei, “Perceptual losses for real-time style transfer and super-resolution,” inComputer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11- 14, 2016, Proceedings, Part II 14. Springer, 2016, pp. 694–711

    2016

  70. [78]

    The unreasonable effectiveness of deep features as a perceptual metric,

    R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, “The unreasonable effectiveness of deep features as a perceptual metric,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 586–595

    2018

  71. [79]

    Very deep convolutional networks for large-scale image recognition,

    K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,”arXiv preprint arXiv:1409.1556, 2014

    2014 arXiv

  72. [80]

    Arbitrary style transfer in real-time with adaptive instance normalization,

    X. Huang and S. Belongie, “Arbitrary style transfer in real-time with adaptive instance normalization,” inProceedings of the IEEE international conference on computer vision, 2017, pp. 1501–1510

    2017

  73. [81]

    Synctalkface: Talking face generation with precise lip-syncing via audio-lip memory,

    S. J. Park, M. Kim, J. Hong, J. Choi, and Y . M. Ro, “Synctalkface: Talking face generation with precise lip-syncing via audio-lip memory,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 2, 2022, pp. 2062–2070

    2022

  74. [82]

    Out of time: automated lip sync in the wild,

    J. S. Chung and A. Zisserman, “Out of time: automated lip sync in the wild,” inComputer Vision–ACCV 2016 Workshops: ACCV 2016 International Workshops, Taipei, Taiwan, November 20-24, 2016, Revised Selected Papers, Part II 13. Springer, 2017, pp. 251–263

    2016

  75. [83]

    Talkclip: Talking head generation with text-guided expressive speaking styles,

    Y . Ma, S. Wang, Y . Ding, B. Ma, T. Lv, C. Fan, Z. Hu, Z. Deng, and X. Yu, “Talkclip: Talking head generation with text-guided expressive speaking styles,”arXiv preprint arXiv:2304.00334, 2023

    2023 arXiv

  76. [84]

    A fine-tuned wav2vec 2.0/hu- bert benchmark for speech emotion recognition, speaker verification and spoken language understanding,

    Y . Wang, A. Boumadane, and A. Heba, “A fine-tuned wav2vec 2.0/hu- bert benchmark for speech emotion recognition, speaker verification and spoken language understanding,”arXiv preprint arXiv:2111.02735, 2021

    2021 arXiv

  77. [85]

    Hubert: Self-supervised speech representation learning by masked prediction of hidden units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, “Hubert: Self-supervised speech representation learning by masked prediction of hidden units,”IEEE/ACM Transactions on JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 17 Audio, Speech...

    2021

  78. [86]

    Emoberta: Speaker-aware emotion recognition in conversation with roberta,

    T. Kim and P. V ossen, “Emoberta: Speaker-aware emotion recognition in conversation with roberta,”arXiv preprint arXiv:2108.12009, 2021

    2021 arXiv

  79. [87]

    Seeing what you said: Talking face generation guided by a lip reading expert,

    J. Wang, X. Qian, M. Zhang, R. T. Tan, and H. Li, “Seeing what you said: Talking face generation guided by a lip reading expert,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 14 653–14 662

    2023

  80. [88]

    V oxceleb2: Deep speaker recognition,

    J. S. Chung, A. Nagrani, and A. Zisserman, “V oxceleb2: Deep speaker recognition,”arXiv preprint arXiv:1806.05622, 2018

    2018 arXiv

  81. [89]

    Lip reading in the wild,

    J. S. Chung and A. Zisserman, “Lip reading in the wild,” inCom- puter Vision–ACCV 2016: 13th Asian Conference on Computer Vision, Taipei, Taiwan, November 20-24, 2016, Revised Selected Papers, Part II 13. Springer, 2017, pp. 87–103

    2016

  82. [90]

    Vfhq: A high- quality dataset and benchmark for video face super-resolution,

    L. Xie, X. Wang, H. Zhang, C. Dong, and Y . Shan, “Vfhq: A high- quality dataset and benchmark for video face super-resolution,” in The IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2022

    2022

  83. [91]

    Musetalk: Real-time high-fidelity video dubbing via spatio-temporal sampling,

    Y . Zhang, Z. Zhong, M. Liu, Z. Chen, B. Wu, Y . Zeng, C. Zhan, Y . He, J. Huang, and W. Zhou, “Musetalk: Real-time high-fidelity video dubbing via spatio-temporal sampling,”arxiv, 2025

    2025

  84. [92]

    V-express: Conditional dropout for progressive training of portrait video generation,

    C. Wang, K. Tian, J. Zhang, Y . Guan, F. Luo, F. Shen, Z. Jiang, Q. Gu, X. Han, and W. Yang, “V-express: Conditional dropout for progressive training of portrait video generation,” 2024

    2024

  85. [93]

    Echomimic: Lifelike audio-driven portrait animations through editable landmark conditions,

    Z. Chen, J. Cao, Z. Chen, Y . Li, and C. Ma, “Echomimic: Lifelike audio-driven portrait animations through editable landmark conditions,” arXiv preprint arXiv:2407.08136, 2024

    2024 arXiv

  86. [94]

    Hallo: Hierarchical audio-driven visual synthesis for portrait image animation,

    M. Xu, H. Li, Q. Su, H. Shang, L. Zhang, C. Liu, J. Wang, Y . Yao, and S. Zhu, “Hallo: Hierarchical audio-driven visual synthesis for portrait image animation,”arXiv preprint arXiv:2406.08801, 2024

    2024 arXiv

  87. [95]

    Emotionally enhanced talking face generation,

    S. Goyal, S. Bhagat, S. Uppal, H. Jangra, Y . Yu, Y . Yin, and R. R. Shah, “Emotionally enhanced talking face generation,” inProceedings of the 1st International Workshop on Multimedia Content Generation and Evaluation: New Methods and Practice, 2023, pp. 81–90

    2023

  88. [96]

    Say anything with any style,

    S. Tan, B. Ji, Y . Ding, and Y . Pan, “Say anything with any style,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 5, 2024, pp. 5088–5096

    2024

  89. [97]

    Flowvqtalker: High-quality emotional talking face generation through normalizing flow and quantization,

    S. Tan, B. Ji, and Y . Pan, “Flowvqtalker: High-quality emotional talking face generation through normalizing flow and quantization,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 26 317–26 327

    2024

  90. [98]

    Image quality assessment: from error visibility to structural similarity,

    Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,”IEEE Transactions on Image Processing, 2004

    2004

  91. [99]

    pytorch-fid: FID Score for PyTorch,

    M. Seitzer, “pytorch-fid: FID Score for PyTorch,” https://github.com/ mseitzer/pytorch-fid, August 2020, version 0.3.0

    2020

  92. [100]

    Frame attention networks for facial expression recognition in videos,

    D. Meng, X. Peng, K. Wang, and Y . Qiao, “Frame attention networks for facial expression recognition in videos,” in2019 IEEE international conference on image processing (ICIP). IEEE, 2019, pp. 3866–3870

    2019

  93. [101]

    Deep audio-visual speech recognition,

    T. Afouras, J. S. Chung, A. Senior, O. Vinyals, and A. Zisserman, “Deep audio-visual speech recognition,”IEEE transactions on pattern analysis and machine intelligence, vol. 44, no. 12, pp. 8717–8727, 2018

    2018

  94. [102]

    Emotion- controllable generalized talking face generation

    S. Sinha, S. Biswas, R. Yadav, and B. Bhowmick, “Emotion- controllable generalized talking face generation.”

  95. [103]

    Pirenderer: Controllable portrait image generation via semantic neural rendering,

    Y . Ren, G. Li, Y . Chen, T. H. Li, and S. Liu, “Pirenderer: Controllable portrait image generation via semantic neural rendering,” inProceed- ings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 13 759–13 768

    2021

  96. [104]

    One-shot free-view neural talking-head synthesis for video conferencing,

    T.-C. Wang, A. Mallya, and M.-Y . Liu, “One-shot free-view neural talking-head synthesis for video conferencing,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 10 039–10 049

    2021

  97. [105]

    Implicit identity representation conditioned memory compensation network for talking head video generation,

    F.-T. Hong and D. Xu, “Implicit identity representation conditioned memory compensation network for talking head video generation,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 23 062–23 072

    2023

  98. [106]

    Efficient video portrait reenactment via grid-based codebook,

    K. Wang, H. Zhou, Q. Wu, J. Tang, Z. Xu, B. Liang, T. Hu, E. Ding, J. Liu, Z. Liuet al., “Efficient video portrait reenactment via grid-based codebook,” inACM SIGGRAPH 2023 Conference Proceedings, 2023, pp. 1–9

    2023

  99. [107]

    High- resolution image synthesis with latent diffusion models,

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High- resolution image synthesis with latent diffusion models,” 2021

    2021

  100. [108]

    What makes fake images detectable? understanding properties that generalize,

    L. Chai, D. Bau, S.-N. Lim, and P. Isola, “What makes fake images detectable? understanding properties that generalize,”Lecture Notes in Computer Science, 2020

    2020

  101. [109]

    Deepfake video detection using recurrent neural networks,

    D. Guera and E. J. Delp, “Deepfake video detection using recurrent neural networks,”Advanced Video and Signal Based Surveillance, 2018

    2018

  102. [110]

    Look, listen and learn,

    R. Arandjelovic and A. Zisserman, “Look, listen and learn,” inProceed- ings of the IEEE international conference on computer vision, 2017, pp. 609–617

    2017

  103. [111]

    Joint audio-visual deepfake detection,

    Y . Zhou and S.-N. Lim, “Joint audio-visual deepfake detection,” International Conference on Computer Vision, 2021

    2021

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.