Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

VQTalker: Towards Multilingual Talking Avatars through Facial Motion Tokenization

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read VQTalker claims that discretizing facial motion into a finite token vocabulary, rather than a continuous stream, is what lets talking-head avatars lip-sync across languages and at roughly 11 kbps.

desk verdict Plausible, well-engineered system; the multilingual and state-of-the-art claims outrun the evidence, but the GRFSQ idea and the low-bitrate result merit a serious look. read the letter →

arxiv 2412.09892 v2 pith:J6HXMMQO submitted 2024-12-13 cs.CV

Tao Liu , Ziyang Ma , Qi Chen , Feilong Chen , Shuai Fan , Xie Chen , Kai Yu This is my paper · ORCID
classification cs.CV
keywords talkingheadgenerationfacialmotiontokenizationvectorquantizationGRFSQmultilinguallipsynchronizationvisemelow-bitrateanimationcoarse-to-fine
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

VQTalker sets out to prove that a talking head should not model facial motion as a continuous stream but as a finite vocabulary of discrete motion tokens, analogous to the way speech is composed of a finite set of phonemes and corresponding visemes. The paper argues that continuous representations are the reason existing systems struggle with non-Indo-European languages: they allow infinite variation, overfit the dominant training languages, and leak identity. The proposed framework quantizes the difference between source and target face motion with a Group Residual Finite Scalar Quantization (GRFSQ) bottleneck, then generates motion tokens from speech tokens through a coarse-to-fine transformer. The authors report state-of-the-art results on both video reconstruction and speech-driven generation, with a sharper improvement on their own six-language non-Indo-European evaluation set, at roughly 11 kbps for 512x512 facial motion. A sympathetic reader would take the core proposal as: make the representation discrete and the model will transfer across languages with less data and less bandwidth.

What carries the argument

The central object is Group Residual Finite Scalar Quantization (GRFSQ), a codebook-free quantizer that splits the 120-dimensional motion difference into 12 groups, applies 4 rounds of residual quantization, and at each step projects a low-dimensional residual onto a finite grid of levels (e.g., 5x5x5x5) via finite scalar quantization; the product grid forms an implicit codebook of 625 entries per group. Because there is no explicit learned codebook, it avoids codebook collapse and needs no auxiliary losses. On top of it, the paper builds a coarse-to-fine generation loop: for each of the 4 residual layers, a shared transformer with a layer indicator predicts all frame tokens non-autoregressively within the layer, while layers are generated autoregressively from coarse to fine, so early layers capture head pose and large motion and later layers add lip detail and suppress jitter.

What would settle it

The central claim would be falsified by a multilingual benchmark with dozens of speakers per language and phoneme-balanced test sentences that shows the discrete method's loss gap between Indo-European and non-Indo-European languages is no smaller than that of continuous baselines.

Watch

Extended reading notes

Core claim

On its own terms, the paper's claim is that facial motion can be lossily but usefully reduced to a discrete codebook sequence, and that doing so is not merely a compression trick but the mechanism that enables multilingual generalization. The tokenizer encodes motion as the difference between source and target facial features, splits it into groups, and applies repeated residual finite scalar quantization, producing code indices that a shared transformer predicts from speech tokens plus controls for head pose, gaze, and blinks. In experiments the discrete-discrete pipeline (speech tokens in, motion tokens out) beats continuous alternatives on identity preservation, sharpness, and video distribution distance, and, crucially, the gap between Indo-European and non-Indo-European test scores is smaller for VQTalker than for comparison systems. The authors further claim a bitrate of about 11 kbps, roughly 70% of the lowest continuous-representation bitrate cited, at 512x512 resolution.

Load-bearing premise

The claim that discrete tokens generalize across languages is tested on a corpus of only 30 short clips, five per language for six languages, collected by the authors; if those clips do not represent each language's range of sounds and mouth shapes, the cross-lingual result could disappear.

Editorial extensions

If this is right

  • If the central claim holds, multilingual talking-head systems no longer need large per-language training corpora; a finite motion-token vocabulary learned once should transfer to new languages through discrete speech tokens.
  • Facial motion can be stored or transmitted at about 11 kbps for 512x512 avatars, well below the 16 kbps floor cited for continuous representations, making real-time streaming avatars cheaper.
  • The discrete bottleneck doubles as an identity-leakage guard, so cross-identity reenactment can use a driving video without transferring skin tone or other appearance attributes.
  • The coarse-to-fine residual structure gives explicit control: separate layers carry pose, gaze, blink, and fine lip detail, so editing or retargeting one aspect without regenerating the whole sequence becomes possible.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not test zero-shot transfer to a language absent from its evaluation set, but if the discrete-token argument is right, a model trained only on Indo-European speech should animate a speaker of a completely unseen language about as well as one of the six tested languages; that is a direct, checkable extension.
  • Because the method separates motion tokens from texture rendering, its 11 kbps bitrate is a motion-stream cost, not a full video cost; comparing end-to-end systems would require adding the renderer's output bitrate, so the headline number should be read as a motion-representation result, our reading rather than the paper's.
  • The phoneme-viseme framing predicts that languages with similar articulatory inventories should transfer best to each other; a per-viseme or per-phoneme error analysis on a larger corpus would reveal whether transfer is truly organized by shared mouth shapes or by something else in the discrete code.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. VQTalker proposes a two-stage talking-head framework: a self-supervised facial motion tokenizer that quantizes motion differences with Group Residual Finite Scalar Quantization (GRFSQ), and a BERT-based coarse-to-fine generator that maps discrete speech tokens to four residual motion codebooks. The paper claims state-of-the-art performance in both video reconstruction and speech-driven generation, improved cross-lingual generalization from discretization, and a low bitrate of approximately 11 kbps at 512x512 resolution. Evaluation is carried out on HDTF and on a newly collected Multilingual Non-Indo-European Talking Head Evaluation Corpus (MNTE), with ablations over codebook design and discrete versus continuous representations.

Significance. If the central claims are validated, the paper would provide a useful architectural contribution: a codebook-free quantizer for facial motion, a clean separation between motion tokenization and rendering, and a low-bitrate discrete representation that may generalize across languages. Strengths of the manuscript include the transparent bitrate calculation (4 x 12 x log2(625) x 25), the detailed model configuration and hyperparameters, the ablation of different codebook designs, and an explicit limitations and ethics discussion. The central empirical claims, however, rest on comparisons that are partly confounded and on a very small, unreleased evaluation corpus; these need to be addressed before the stated conclusions can be accepted.

major comments (4)
  1. [Table 2 and Abstract] The abstract claims "state-of-the-art performance in both video-driven and speech-driven scenarios," but Table 2 shows that on MNTE, EDTalker outperforms VQTalker on SSIM (0.745 vs 0.730) and LMD (0.107 vs 0.146), while on HDTF VQTalker wins. This unqualified claim is not supported by the reported numbers and should be rephrased to identify the specific metrics and datasets where VQTalker is best, with a discussion of where it is not.
  2. [Ablation Study, Table 4] The central causal claim that discrete facial motion tokens improve cross-lingual generalization is not isolated in Table 4: the D-D condition changes both the audio encoder (CosyVoice vs Whisper vs VQ-Wav2vec) and the motion representation (vector vs GRFSQ), while the C-D and D-C rows do not hold the other component fixed. In addition, Table 4 reports only HDTF, which is Indo-European, so it says nothing directly about the multilingual setting. An experiment that varies only the motion representation while keeping the audio encoder fixed, evaluated on MNTE, is needed to support the paper's main claim.
  3. [Experiment, Evaluation Metrics and MNTE] No confidence intervals, per-sample breakdowns, or significance tests are reported for any of the quantitative results in Tables 1, 2, 3, 4, or 8. This is particularly problematic for the multilingual claim because MNTE contains only 30 videos total, 5 per language for 6 languages, and the authors state that the corpus is not released. Many reported differences are small (e.g., Table 1 SSIM 0.874 vs AniTalker 0.905), so without variance estimates or significance testing the conclusions are not statistically grounded. The authors should report confidence intervals or error bars, ideally with a released or expanded evaluation set.
  4. [Table 2 and Table 5] The speech-driven comparison in Table 2 is a full-system comparison in which baselines are trained on different datasets (Table 5), use different decoders, and differ in audio features. Therefore the observed gains over EDTalker and EchoMimic cannot be attributed to the discrete motion representation or to GRFSQ specifically. A controlled comparison with a continuous-representation variant trained on the same data, or an ablation that isolates the proposed components under matched conditions, is necessary to support the attribution.
minor comments (6)
  1. [Facial Motion Tokenizer] In the section headed "Facial Motion Tokenizer," the text says "Table 7 presents the results," but the table referenced is Table 1 in the main text; the table numbering is inconsistent with the supplementary material and should be corrected.
  2. [Limitations and Future Work] The Limitations section refers to a "wrapping method" that causes blur; this appears to be a typo for "warping method" and should be fixed.
  3. [Training and Inference] The statement "we did not incorporate any pre-trained parameters" is confusing because the framework uses a pretrained speech tokenizer from CosyVoice, Whisper, and VQ-Wav2vec in ablations, plus 3DDFA and landmark predictors; please clarify that the claim applies only to the generator and renderer, not to the entire pipeline.
  4. [Algorithm 1] Algorithm 1 initializes "I ← empty tensor of shape (G, R, ∗)" but does not define how the FSQ level vector [l1, ..., ld] is converted into index tuples or what the exact shapes are; adding a concrete example or a precise index formula would improve reproducibility.
  5. [Figure 5] The language distribution percentages in Figure 5 do not obviously sum to 100%: the listed categories sum to about 97.45% before the additional 2.55% and 4.30% labels, so the figure legend or the percentages should be corrected.
  6. [Table 3] In Table 3, the VQ row reports "# Codes 8196" with CSIM equal to MSI (0.588 and 0.588); this looks like a typo and should be verified, and the units or meaning of "Util." should be stated more precisely.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: VQTalker's bitrate, tokenizer, and evaluations are not defined in terms of their outputs; self-citations to CosyVoice and AniTalker are reproducible components rather than load-bearing circular premises.

full rationale

The paper's central claims are empirical architecture results, not derivations from fitted parameters. The low-bitrate claim is computed directly from chosen hyperparameters (12 groups, 4 residual layers, 625 entries per group): 4 x 12 x log2(625) x 25 = 11,150 bits/s, approximately 11 kbps. This is an arithmetic consequence of the configuration, not a prediction fitted to the outcome. GRFSQ is presented as a combination of Group VQ, Residual VQ, and Finite Scalar Quantization, with no invoked uniqueness theorem and no ansatz smuggled in via citation. The multilingual claim is tested against external baselines on HDTF and the authors' MNTE corpus; the central mechanism is not defined in terms of any of those baselines. The closest concern is Table 4, which switches audio representation and motion representation simultaneously (Whisper+Vector vs. CosyVoice+GRFSQ), so the reported gain cannot cleanly be attributed to the discrete motion tokens. That is an experimental confound and a correctness risk, not circularity: no quantity in the paper is defined in terms of another, and no fitted input is renamed as a prediction. Self-citations appear (AniTalker as a baseline, CosyVoice as a pretrained audio tokenizer), but CosyVoice is a released, code-reproducible model used as a system component, and AniTalker is used as a comparison method, not as the justification for VQTalker's claims. Under the stated rules, such citations are real evidence and do not raise the circularity score. No step in the derivation chain reduces to its own inputs by construction.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim relies on several hand-picked hyperparameters (G, R, FSQ levels, loss weights) that are tuned on the evaluation datasets, plus domain assumptions about phoneme-viseme universality and the benefit of discrete representations. No new physical or mathematical entities are introduced.

free parameters (5)
  • Number of groups G = 12
    Chosen via ablation on HDTF (Table 3); directly affects bitrate and quality.
  • Number of residual quantizers R = 4
    Chosen via ablation (Table 3); controls coarse-to-fine granularity and bitrate.
  • FSQ levels per dimension = [5, 5, 5, 5]
    From ablation (Table 3); determines implicit codebook size and bitrate: 4*12*log2(625)*25.
  • Loss weights lambda1, lambda2, lambda3, lambda4 = 1000, 1000, 0.01, 1
    Reported as determined through extensive experimentation in supplementary, tuned on validation data.
  • Audio downsampling ratio = 50 Hz to 25 Hz
    Chosen to synchronize speech tokens with 25 fps video; a design choice not derived from theory.
assumptions (4)
  • domain assumption Human speech comprises a finite set of distinct sound units (phonemes) and corresponding visual articulations (visemes) that share commonalities across languages.
    Used in the Introduction and Abstract to motivate the discrete representation. Not proven in the paper.
  • domain assumption Discrete representations reduce overfitting to language-specific variations and improve cross-lingual generalization.
    Central to the method; supported only by the small ablation in Table 4, not by a theoretical argument.
  • domain assumption Global semantic vector quantization is superior to patch-based VQ for facial motion.
    Argued in the supplementary section on Patch-based VQ vs. Global Semantic VQ without empirical comparison.
  • standard math Straight-Through Estimator provides a valid gradient approximation for the discrete quantization step.
    A standard technique in VQ training, but the paper provides no analysis of its convergence or bias in this setting.

how reviews work

0 comments
Cite this review

Pith. "Pith review of VQTalker: Towards Multilingual Talking Avatars through Facial Motion Tokenization." pith.science (2026). https://pith.science/paper/J6HXMMQO

@misc{pith2026241209892,
  author       = {Pith},
  title        = {Pith review of: VQTalker: Towards Multilingual Talking Avatars through Facial Motion Tokenization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/J6HXMMQO}},
  note         = {Machine review of arXiv:2412.09892}
}
read the original abstract

We present VQTalker, a Vector Quantization-based framework for multilingual talking head generation that addresses the challenges of lip synchronization and natural motion across diverse languages. Our approach is grounded in the phonetic principle that human speech comprises a finite set of distinct sound units (phonemes) and corresponding visual articulations (visemes), which often share commonalities across languages. We introduce a facial motion tokenizer based on Group Residual Finite Scalar Quantization (GRFSQ), which creates a discretized representation of facial features. This method enables comprehensive capture of facial movements while improving generalization to multiple languages, even with limited training data. Building on this quantized representation, we implement a coarse-to-fine motion generation process that progressively refines facial animations. Extensive experiments demonstrate that VQTalker achieves state-of-the-art performance in both video-driven and speech-driven scenarios, particularly in multilingual settings. Notably, our method achieves high-quality results at a resolution of 512*512 pixels while maintaining a lower bitrate of approximately 11 kbps. Our work opens new possibilities for cross-lingual talking face generation. Synthetic results can be viewed at https://x-lance.github.io/VQTalker.

Figures

Figures reproduced from arXiv: 2412.09892 by the authors.

Figure 1
Figure 1. VQTalker converts audio tokens to facial motion [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The VQTalker framework consists of two main components: (1) A quantized facial motion codec that learns a uni [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Codebook interleaving patterns. Hybrid facial mo [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Qualitative comparison of speech-driven talking head generation methods across non-Indo-European languages from [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Language distribution in our constructed training [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 7
Figure 7. Figure 7: FSQ Quantization Details Details on Other Datasets To analyze the datasets used by various speech-driven base￾lines, we compiled statistics on audio-driven methods and their training datasets, as shown in [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Ablation study on eye gaze and mouth shape mod [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 9
Figure 9. Figure 9: Comparison of continuous (a) and discrete (b) rep [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. FixTalk: Taming Identity Leakage for High-Quality Talking Head Generation in Extreme Cases

    cs.CV 2025-07 conditional novelty 5.0 of 10

    FixTalk adds two modules to a real-time GAN talking-head model, decoupling identity from motion to stop identity leakage while using a memory to recover details and reduce artifacts.

Reference graph

Works this paper leans on

56 extracted references · 39 canonical work pages · cited by 1 Pith paper

  1. [1]

    Baevski, A.; Schneider, S.; and Auli, M. 2020. vq-wav2vec: Self-supervised learning of discrete speech representations. International Conference on Learning Representations (ICLR)

    work page 2020

  2. [2]

    Bohr, P.; Gargote, R.; Vhorkate, R.; Yawle, R.; and Bairagi, V. 2013. A no reference image blur detection using cumulative probability blur detection (cpbd) metric. International Journal of Science and Modern Engineering, 1(5)

    work page 2013

  3. [3]

    Chen, L.-W.; Watanabe, S.; and Rudnicky, A. 2023. A vector quantized approach for text to speech synthesis on real-world spontaneous speech. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, 12644--12652

    work page 2023

  4. [4]

    Chen, Z.; Cao, J.; Chen, Z.; Li, Y.; and Ma, C. 2024. EchoMimic: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditions. arXiv preprint arXiv:2407.08136

    arXiv 2024

  5. [5]

    Copet, J.; Kreuk, F.; Gat, I.; Remez, T.; Kant, D.; Synnaeve, G.; Adi, Y.; and D \'e fossez, A. 2024. Simple and controllable music generation. Advances in Neural Information Processing Systems, 36

    2024

  6. [6]

    Cortacero, K.; Fischer, T.; and Demiris, Y. 2019. RT-BENE: A dataset and baselines for real-time blink estimation in natural environments. In Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, 0--0

    work page 2019

  7. [7]

    Davenport, M.; and Hannahs, S. J. 2020. Introducing phonetics and phonology. Routledge

    work page 2020

  8. [8]

    Devlin, J.; Chang, M.-W.; Lee, K.; and Toutanova, K. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 4171--4186

    2019

Show all 56 references
  1. [9]

    B.; Vougioukas, K.; Landgraf, Z.; Petridis, S.; and Pantic, M

    Drobyshev, N.; Casademunt, A. B.; Vougioukas, K.; Landgraf, Z.; Petridis, S.; and Pantic, M. 2024. EMOPortraits: Emotion-enhanced Multimodal One-shot Head Avatars. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8498--8507

    2024

  2. [10]

    Drobyshev, N.; Chelishev, J.; Khakhulin, T.; Ivakhnenko, A.; Lempitsky, V.; and Zakharov, E. 2022 a . Megaportraits: One-shot megapixel neural head avatars. In Proceedings of the 30th ACM International Conference on Multimedia (ACM MM), 2663--2671

    2022

  3. [11]

    Drobyshev, N.; Chelishev, J.; Khakhulin, T.; Ivakhnenko, A.; Lempitsky, V.; and Zakharov, E. 2022 b . Megaportraits: One-shot megapixel neural head avatars. In Proceedings of the 30th ACM International Conference on Multimedia, 2663--2671

    2022

  4. [12]

    Du, Z.; Chen, Q.; Zhang, S.; Hu, K.; Lu, H.; Yang, Y.; Hu, H.; Zheng, S.; Gu, Y.; Ma, Z.; et al. 2024. CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokens. arXiv preprint arXiv:2407.05407

    2024 arXiv

  5. [13]

    Esser, P.; Rombach, R.; and Ommer, B. 2021. Taming transformers for high-resolution image synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 12873--12883

    2021

  6. [14]

    Fabian, M.; David, M.; Eirikur, A.; and Michael, T. 2024. Finite Scalar Quantization: VQ-VAE Made Simple. International Conference on Learning Representations (ICLR)

    2024

  7. [15]

    Feng, Z.-H.; Kittler, J.; Awais, M.; Huber, P.; and Wu, X.-J. 2018. Wing loss for robust facial landmark localisation with convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2235--2245

    2018

  8. [16]

    Gan, Y.; Yang, Z.; Yue, X.; Sun, L.; and Yang, Y. 2023. Efficient emotional adaptation for audio-driven talking-head generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 22634--22645

    2023

  9. [17]

    Guo, J.; Zhang, D.; Liu, X.; Zhong, Z.; Zhang, Y.; Wan, P.; and Zhang, D. 2024. LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Control. arXiv preprint arXiv:2407.03168

    2024 arXiv

  10. [18]

    Guo, J.; Zhu, X.; Yang, Y.; Yang, F.; Lei, Z.; and Li, S. Z. 2020. Towards Fast, Accurate and Stable 3D Dense Face Alignment. In Proceedings of the European Conference on Computer Vision (ECCV)

    2020

  11. [19]

    He, T.; Guo, J.; Yu, R.; Wang, Y.; Zhu, J.; An, K.; Li, L.; Tan, X.; Wang, C.; Hu, H.; Wu, H.; Zhao, S.; and Bian, J. 2024. GAIA: Zero-shot Talking Avatar Generation. The International Conference on Learning Representations (ICLR)

    2024

  12. [20]

    Huang, H.; Wu, Z.; Kang, S.; Dai, D.; Jia, J.; Fu, T.; Tuo, D.; Lei, G.; Liu, P.; Su, D.; et al. 2021. Speaker independent and multilingual/mixlingual speech-driven talking head generation using phonetic posteriorgrams. In 2021 Asia-Pacific Signal and Information Processing As...

    2021

  13. [21]

    Ling, J.; Tan, X.; Chen, L.; Li, R.; Zhang, Y.; Zhao, S.; and Song, L. 2023. Stableface: Analyzing and improving motion stability for talking face generation. IEEE Journal of Selected Topics in Signal Processing

    2023

  14. [22]

    Liu, T.; Chen, F.; Fan, S.; Du, C.; Chen, Q.; Chen, X.; and Yu, K. 2024. AniTalker: Animate Vivid and Diverse Talking Faces through Identity-Decoupled Facial Motion Encoding. In Proceedings of the 32th ACM International Conference on Multimedia (ACM MM)

    2024

  15. [23]

    Ma, Y.; Wang, S.; Hu, Z.; Fan, C.; Lv, T.; Ding, Y.; Deng, Z.; and Yu, X. 2023. Styletalk: One-shot talking head generation with controllable speaking styles. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, 1896--1904

    2023

  16. [24]

    McGurk, H.; and MacDonald, J. 1976. Hearing lips and seeing voices. Nature

    1976

  17. [25]

    S.; and Zisserman, A

    Nagrani, A.; Chung, J. S.; and Zisserman, A. 2017. Voxceleb: a large-scale speaker identification dataset. arXiv preprint arXiv:1706.08612

    2017 arXiv

  18. [26]

    Pang, Y.; Zhang, Y.; Quan, W.; Fan, Y.; Cun, X.; Shan, Y.; and Yan, D.-m. 2023. Dpe: Disentanglement of pose and expression for general video portrait editing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 427--436

    2023

  19. [27]

    W.; Xu, T.; Brockman, G.; McLeavey, C.; and Sutskever, I

    Radford, A.; Kim, J. W.; Xu, T.; Brockman, G.; McLeavey, C.; and Sutskever, I. 2023. Robust speech recognition via large-scale weak supervision. In International conference on machine learning, 28492--28518. PMLR

    2023

  20. [28]

    Siarohin, A.; Lathuili \`e re, S.; Tulyakov, S.; Ricci, E.; and Sebe, N. 2019. First order motion model for image animation. Advances in neural information processing systems, 32

    2019

  21. [29]

    H.; Lee, J.; Yang, S.; Cho, H.; Lee, Y.; Choi, D.; and Kim, K.-w

    Song, H.-K.; Woo, S. H.; Lee, J.; Yang, S.; Cho, H.; Lee, Y.; Choi, D.; and Kim, K.-w. 2022. Talking face generation with multilingual tts. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 21425--21430

    2022

  22. [30]

    Sung-Bin, K.; Chae-Yeon, L.; Son, G.; Hyun-Bin, O.; Ju, J.; Nam, S.; and Oh, T.-H. 2024. MultiTalk: Enhancing 3D Talking Head Generation Across Languages with Multilingual Video Dataset. arXiv preprint arXiv:2406.14272

    2024 arXiv

  23. [31]

    Tan, S.; Ji, B.; Bi, M.; and Pan, Y. 2024 a . EDTalk: Efficient Disentanglement for Emotional Talking Head Synthesis. Proceedings of the European conference on computer vision (ECCV)

    2024

  24. [32]

    Tan, S.; Ji, B.; Ding, Y.; and Pan, Y. 2024 b . Say anything with any style. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 5088--5096

    2024

  25. [33]

    Tan, S.; Ji, B.; and Pan, Y. 2024. FlowVQTalker: High-Quality Emotional Talking Face Generation through Normalizing Flow and Quantization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 26317--26327

    2024

  26. [34]

    Tao, J.; Wang, B.; Ge, T.; Jiang, Y.; Li, W.; and Duan, L. 2022. Motion Transformer for Unsupervised Image Animation. In European Conference on Computer Vision (ECCV), 702--719. Springer

    2022

  27. [35]

    Tian, L.; Wang, Q.; Zhang, B.; and Bo, L. 2024. Emo: Emote portrait alive-generating expressive portrait videos with audio2video diffusion model under weak conditions. arXiv preprint arXiv:2402.17485

    2024 arXiv

  28. [36]

    Wang, D.; Deng, Y.; Yin, Z.; Shum, H.-Y.; and Wang, B. 2023. Progressive disentangled representation learning for fine-grained controllable talking head synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 17979--17989

    2023

  29. [37]

    Wang, K.; Wu, Q.; Song, L.; Yang, Z.; Wu, W.; Qian, C.; He, R.; Qiao, Y.; and Loy, C. C. 2020. Mead: A large-scale audio-visual dataset for emotional talking-face generation. In European Conference on Computer Vision (ECCV), 700--717. Springer

    2020

  30. [38]

    Wang, T.-C.; Mallya, A.; and Liu, M.-Y. 2021. One-shot free-view neural talking-head synthesis for video conferencing. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), 10039--10049

    2021

  31. [39]

    Wang, Y.; Yang, D.; Bremond, F.; and Dantcheva, A. 2022. Latent image animator: Learning to animate images via latent space navigation. Proceedings of the International Conference on Learning Representations (ICLR)

    2022

  32. [40]

    C.; Sheikh, H

    Wang, Z.; Bovik, A. C.; Sheikh, H. R.; and Simoncelli, E. P. 2004. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4): 600--612

    2004

  33. [41]

    J.; Bjontegaard, G.; and Luthra, A

    Wiegand, T.; Sullivan, G. J.; Bjontegaard, G.; and Luthra, A. 2003. Overview of the H. 264/AVC video coding standard. IEEE Transactions on circuits and systems for video technology, 13(7): 560--576

    2003

  34. [42]

    Xie, L.; Wang, X.; Zhang, H.; Dong, C.; and Shan, Y. 2022. Vfhq: A high-quality dataset and benchmark for video face super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 657--666

    2022

  35. [43]

    Xing, J.; Xia, M.; Zhang, Y.; Cun, X.; Wang, J.; and Wong, T.-T. 2023 a . Codetalker: Speech-driven 3d facial animation with discrete motion prior. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 12780--12790

    2023

  36. [44]

    Xing, J.; Xia, M.; Zhang, Y.; Cun, X.; Wang, J.; and Wong, T.-T. 2023 b . Codetalker: Speech-driven 3d facial animation with discrete motion prior. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 12780--12790

    2023

  37. [45]

    Xu, S.; Chen, G.; Guo, Y.-X.; Yang, J.; Li, C.; Zang, Z.; Zhang, Y.; Tong, X.; and Guo, B. 2024. Vasa-1: Lifelike audio-driven talking faces generated in real time. arXiv preprint arXiv:2404.10667

    2024 arXiv

  38. [46]

    C.; Denil, M.; et al

    Yang, Y.; Shillingford, B.; Assael, Y.; Wang, M.; Liu, W.; Chen, Y.; Zhang, Y.; Sezener, E.; Cobo, L. C.; Denil, M.; et al. 2020. Large-scale multilingual audio visual dubbing. arXiv preprint arXiv:2011.03530

    2020 arXiv

  39. [47]

    Yu, Q.; Weber, M.; Deng, X.; Shen, X.; Cremers, D.; and Chen, L.-C. 2024. An Image is Worth 32 Tokens for Reconstruction and Generation. arXiv preprint arXiv:2406.07550

    2024 arXiv

  40. [48]

    Zeghidour, N.; Luebs, A.; Omran, A.; Skoglund, J.; and Tagliasacchi, M. 2021. Soundstream: An end-to-end neural audio codec. IEEE/ACM Transactions on Audio, Speech, and Language Processing (TASLP), 30: 495--507

    2021

  41. [49]

    Zeng, B.; Liu, X.; Gao, S.; Liu, B.; Li, H.; Liu, J.; and Zhang, B. 2023. Face Animation with an Attribute-Guided Diffusion Model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 628--637

    2023

  42. [50]

    A.; Shechtman, E.; and Wang, O

    Zhang, R.; Isola, P.; Efros, A. A.; Shechtman, E.; and Wang, O. 2018. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)

    2018

  43. [51]

    Zhang, W.; Cun, X.; Wang, X.; Zhang, Y.; Shen, X.; Guo, Y.; Shan, Y.; and Wang, F. 2023 a . SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patt...

    2023

  44. [52]

    Zhang, Z.; Li, L.; Ding, Y.; and Fan, C. 2021. Flow-guided one-shot talking face generation with a high-resolution audio-visual dataset. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    2021

  45. [53]

    Zhang, Z.; et al. 2023 b . DINet: Deformation Inpainting Network for Realistic Face Visually Dubbing on High Resolution Video. Thirty-Seventh AAAI Conference on Artificial Intelligence (AAAI)

    2023

  46. [54]

    Zhu, H.; Wu, W.; Zhu, W.; Jiang, L.; Tang, S.; Zhang, L.; Liu, Z.; and Loy, C. C. 2022. CelebV-HQ: A large-scale video facial attributes dataset. In European conference on computer vision (ECCV), 650--667. Springer

    2022

  47. [55]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...

  48. [56]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.