Pith. sign in

REVIEW 5 major objections 6 minor 54 references

Democratizing High-Fidelity Co-Speech Gesture Video Generation

T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that predicting 2D skeletons from audio with a diffusion model and rendering them through a video generator yields high-fidelity gesture videos that beat prior methods in unpaired settings.

desk verdict CSG-405 is the real contribution and the method is sensible; the paper must specify the wav2vec-to-frame alignment and tone down the paired-quality SOTA claim. read the letter →

arxiv 2507.06812 v2 pith:XYM7J7IX submitted 2025-07-09 cs.CV cs.AI

Xu Yang , Shaoli Huang , Shenbo Xie , Xuelin Chen , Yifei Liu , Changxing Ding This is my paper · ORCID
classification cs.CVcs.AI
keywords co-speechgesturevideogenerationdiffusionmodel2Dskeletonaudio-to-skeletonCSG-405wav2vec2.0human
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes that high-fidelity co-speech gesture video can be produced by splitting the problem into two cheap parts: a diffusion model that predicts a 2D full-body skeleton sequence from audio plus the skeleton of a reference image, and an off-the-shelf human video generator that turns those skeletons into video. The central claim is that this skeleton bridge resolves the one-to-many audio-to-visual mapping without the private multi-thousand-hour training data and high-end GPU budgets used by industrial systems. The authors also release CSG-405, a 405-hour public dataset of 147,550 clips across 71 speech types with full-body skeleton annotations, which they argue is the first large-scale public resource for this task. If the claim holds, strong co-speech gesture video generation becomes a reproducible, comparatively low-cost research problem rather than an industrial monopoly.

What carries the argument

The mechanism is a two-stage generation: $S_g = G_s(s_r, A)$ predicts the skeleton sequence, and $V = G_v(S_g, R)$ renders the video. $G_s$ is a Diffusion Transformer; at each denoising step it receives the noisy skeleton sequence concatenated along the frame dimension with the reference skeleton $s_r$, and along the feature dimension with segment-level wav2vec 2.0 features (one 768-dimension vector per video frame), and it is trained to predict the clean skeleton $x_0$ with classifier-free guidance. Keypoints are expressed in a local motion representation with nose, wrists, and neck as roots, which disentangles subtle facial and hand motion from global body motion. $G_v$ is an unmodified or lightly fine-tuned human video generation model (StableAnimator, EchoMimicV2, or MimicMotion in the experiments) that renders the reference image under the predicted skeleton sequence.

What would settle it

Generate the same test audio with the wav2vec audio features deliberately shifted by one, two, or three frames relative to the skeleton sequence; if SyncNet audio-lip scores do not degrade sharply with these small offsets, then the one-to-one audio-frame conditioning is not what produces the reported synchronization.

Watch

Extended reading notes

Core claim

The paper's central claim is that a lightweight audio-to-skeleton diffusion model, whose input concatenates wav2vec 2.0 audio segments and the skeleton extracted from the speaker's reference image, produces skeletons that are both body-shape-consistent and frame-synchronized with the audio; when these skeletons drive an off-the-shelf video diffusion model, the resulting videos outperform prior methods in unpaired settings (audio and reference image from different sources) and match strong paired baselines. The authors treat the 2D skeleton as the load-bearing intermediate because it encodes expressive motion cheaply while remaining compatible with existing pose-conditioned video generators. They further claim that feature-level concatenation of audio and skeleton embeddings, rather than cross-attention, enforces one-to-one gesture-audio correspondence, and that the CSG-405 dataset is what makes such training feasible publicly.

Load-bearing premise

The method assumes the audio feature extractor produces exactly one feature vector per video frame at 25 frames per second, with each audio segment lining up one-to-one with a skeleton frame; if that alignment is only approximate, the claimed strict synchronization is not guaranteed.

Editorial extensions

If this is right

  • Unpaired audio-and-reference-image settings become practical: a user can drive any portrait with arbitrary speech without needing a paired video of that person gesturing.
  • Because the intermediate is a 2D skeleton, the approach inherits improvements in any pose-conditioned human video generator without retraining the audio-to-skeleton stage.
  • The feature-concatenation conditioning improves audio-lip and audio-gesture synchronization over cross-attention, as shown by the SyncNet ablation.
  • The public CSG-405 dataset allows training on four 32GB V100 GPUs, and fine-tuning an off-the-shelf generator on this data further improves its co-speech gesture quality.
  • Models trained on CSG-405 transfer to PATS and TED-talks without fine-tuning, indicating cross-dataset generalization.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the frame-wise audio-skeleton alignment is truly exact, the same feature-concatenation recipe could transfer to other one-to-one temporal modalities, such as dance or sign-language generation; this is an editorial inference beyond the paper.
  • The strong unpaired-setting results suggest that body-shape consistency from the reference skeleton, rather than audio alone, is doing much of the work; an untested extension is whether swapping the reference image while keeping the same audio changes gesture style but preserves timing.
  • Because the dataset was filtered with SyncNet and DWPose, part of the reported synchronization and skeleton quality may be inherited from dataset curation rather than the conditioning design; isolating that would require training the same model on unfiltered data.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes a two-stage framework for co-speech gesture video generation: first, a diffusion-based audio-to-skeleton model predicts a full-body 2D skeleton sequence from audio and a reference-image skeleton; second, an off-the-shelf video generator animates the reference image with the predicted skeleton. The authors also introduce CSG-405, a 405-hour public dataset of 147,550 clips with 2D full-body skeleton annotations. Experiments on PATS, TED-talks, and CSG-405 report unpaired FID/FVD/Sync-C/D improvements over several baselines, with ablations supporting the reference-skeleton condition, feature-concatenation audio conditioning, and local motion representation. The code, models, and dataset are promised for public release.

Significance. If the claims hold, the main contribution is a practical and relatively lightweight alternative to the large private pipelines used by recent industry systems, plus the first large-scale public dataset for co-speech gesture video generation. The extensive dataset construction pipeline, the use of external benchmarks for zero-shot evaluation, and the public release commitments are concrete strengths that could benefit the community. The technical insight of using fine-grained audio-skeleton concatenation rather than cross-attention is plausible and supported by the ablation trends. However, several load-bearing points need clarification and qualification before the stated contributions can be accepted as stated.

major comments (5)
  1. [3.1 Audio-to-Skeleton Prediction] The wav2vec 2.0 feature-to-frame alignment is unspecified. Section 3.1 states that the extracted audio features have 'the segments number is the same as the video frame number' (25 FPS), but wav2vec 2.0 natively outputs a feature sequence at a different temporal rate (typically 50 Hz for 16 kHz audio). No procedure is given for converting the native features to exactly one feature per video frame (e.g., strided sampling, average pooling, or a learned projection). Since the claimed 'strict coordination' and the feature-concatenation design depend on a one-to-one correspondence between audio segments and skeleton frames, this missing detail prevents reproduction and leaves open the possibility that the synchronization gains in Table 3 (Sync-C 6.28 vs. 0.77) reflect a favorable but unreported alignment choice rather than the conditioning mechanism itself.
  2. [Abstract and Section 4.2, Table 2] The abstract's unqualified claim that the method 'exceeds state-of-the-art approaches in visual quality and synchronization' is contradicted by the paper's own paired-setting results. In Table 2, StableAnimator achieves substantially higher SSIM and PSNR on all three datasets: PATS (0.75/22.93 vs. 0.56/16.35), TED-talks (0.81/25.50 vs. 0.61/17.18), and CSG-405 (0.83/24.27 vs. 0.65/16.26). The text in Section 4.2 acknowledges this, so the abstract and conclusion should be reworded to claim superiority only in the unpaired setting, and even there with the caveats noted below.
  3. [Table 2 and Section 4.2] All quantitative results are reported from single runs with no error bars or significance tests. Several differences in Table 2 are small (e.g., Sync-C 5.49 vs. 5.44 for Ours† vs. EchoMimicV2 on PATS; FVD 984.13 vs. 988.43 on CSG-405), so without variance estimates or multiple seeds the claimed state-of-the-art performance is not statistically supported. The authors should report mean and standard deviation over at least three runs, or otherwise justify why a single run is sufficient for these metrics.
  4. [4.1 Qualitative Comparisons] The comparisons with VLOGGER and CyberHost are only qualitative, using demo videos from the respective project homepages. This does not constitute a controlled comparison: the input references, audio, speakers, and video durations are not matched. Consequently, the abstract's claim of exceeding state-of-the-art methods cannot be extended to these two recent models, and the sentence in Section 4.1 that the approach 'surpasses both VLOGGER and CyberHost' is not supported by quantitative evidence.
  5. [3.2 Dataset Construction and Table 2] The CSG-405 test set was filtered during construction using SyncNet to remove clips with lip-audio misalignment. Since the same SyncNet measure is later used to compute Sync-C and Sync-D on that test set, the CSG-405 synchronization numbers may be inflated by selection. This is visible in Table 2: EchoMimicV2's Sync-C rises from 5.44 on PATS to 6.53 on CSG-405, even though the method was not trained on CSG-405. The authors should quantify this selection effect, for example by reporting Sync-C/D on a non-filtered subset or by discussing how much of the cross-dataset difference is attributable to the filtering rather than to the model.
minor comments (6)
  1. [Section 1, Contributions] Reference [50], cited to support 'a lightweight audio-to-skeleton prediction model', points to a super-resolution paper (Guo et al., TPAMI 2024), not to audio-to-skeleton prediction. Please replace with an appropriate reference to the diffusion architecture or motion-diffusion work actually used.
  2. [Section 3.1] The operations involving the reference skeleton sr are underspecified. The text says 'concatenate xt with sr along the frame dimension' even though sr is a single-frame skeleton (1 x 2K), and later says 'we concatenate a zero vector to sr' without specifying its length or where it is inserted. Please state the exact tensor shapes and broadcasting/concatenation operations.
  3. [Section 3.1 and 4.2] Hyperparameter values for classifier-free guidance scale α and for the temporal smoothing applied to keypoints are never reported. Since both affect the trade-off between fidelity and diversity, and smoothing affects the skeleton dynamics, the experimental setting is incomplete without these values.
  4. [Section 4, Datasets and Metrics] The random selection of 133 test clips from CSG-405 is not described with a seed or a reproducible protocol. Please provide the split details or a deterministic procedure so that other researchers can replicate the exact test set.
  5. [Section 4, Metrics] FID and FVD are computed on videos of different native resolutions (PATS at 256x256, TED-talks at 384x384, CSG-405 at 512x512). Please clarify whether these metrics are computed on the generated videos at their native resolution or after resizing, since this affects cross-dataset comparisons.
  6. [Throughout] Minor language and consistency issues: 'a audio-to-skeleton' in the Broader Impacts section should be 'an audio-to-skeleton', and the audio sampling rate should be written as 16 kHz, not '16K'. The dataset's 71 speech types in Table 1 should be reconciled with the six aggregated categories in Figure 2(a).

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the audio-to-skeleton model is trained on held-out data, evaluated on external benchmarks, and no prediction reduces to a fitted input or self-citation chain.

full rationale

The derivation chain in the paper is Sg = Gs(sr, A) and V = Gv(Sg, R). Gs is trained with a standard diffusion denoising loss on held-out CSG-405 clips, while Gv is an off-the-shelf external video generation model (StableAnimator, EchoMimicV2, or MimicMotion). The main quantitative claims are tested on external PATS and TED-talks splits without fine-tuning, and the in-house CSG-405 test split is a held-out portion of the dataset. No fitted parameter is renamed as a prediction: skeletons are generated from audio and then fed into a separately trained video generator, so the outputs are not equal to any training input by construction. The only self-citation is reference [27], used in a background statement that existing co-speech gesture methods impose audio conditions via cross-attention; this is not load-bearing. The use of SyncNet both as a dataset filter and as an evaluation metric on the CSG-405 test set is a methodological bias concern rather than circular reasoning, because the primary cross-dataset results on PATS and TED-talks are independent of that filtering. The unspecified procedure for aligning wav2vec 2.0 features (natively 50 Hz) to 25 FPS video frames in Section 3.1 is a reproducibility and correctness concern, but it does not make the derivation circular; it concerns an unstated implementation detail, not a self-referential definition or a fitted constant masquerading as a prediction.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim rests on standard diffusion training, on the sufficiency of a single reference skeleton, on an unspecified audio-to-frame alignment, on the robustness of off-the-shelf video generators, and on the representativeness of YouTube after automatic filtering. No new physical or conceptual entities are introduced; CSG-405 is a curated dataset, not an invented entity.

free parameters (3)
  • classifier-free guidance scale α = not reported
    Section 3.1 defines α as the trade-off between diversity and fidelity during inference; no value is given, so the inference setting behind Table 2 and Table 3 is not specified.
  • temporal smoothing parameters = not reported
    Section 3.2 says keypoints except mouth are temporally smoothed, but the filter type, window size, and smoothing strength are omitted; this preprocessing affects skeleton quality and downstream video quality.
  • audio feature to 25 FPS alignment = not reported
    Section 3.1 requires F audio segments for F frames, but the method for converting wav2vec 2.0 features to 25 Hz is not described; the alignment strategy is a free modeling choice.
assumptions (5)
  • standard math The diffusion denoising objective from [5] applied directly to keypoint-coordinate sequences yields high-fidelity skeletal motion.
    Section 3.1 adopts the x0-prediction loss without modification or verification for 2D full-body keypoints.
  • domain assumption The 2D skeleton extracted from a single reference image is sufficient to encode the target speaker's body shape for video generation.
    Section 3.1 concatenates sr with the noisy skeleton sequence along the frame dimension and relies on self-attention to propagate body-shape cues.
  • domain assumption wav2vec 2.0 features can be made frame-aligned with the video, with one feature vector per video frame.
    Section 3.1 asserts the segment number equals the frame number but does not state how wav2vec 2.0's native rate is adjusted.
  • domain assumption An off-the-shelf human video generator renders realistic, identity-preserving video from predicted skeletons without requiring audio conditioning.
    Section 3.1 feeds Sg and R into Gv (StableAnimator or EchoMimicV2) and assumes the generator handles any skeleton artifact or body shape.
  • domain assumption The automatic filters (DWPose, PyAnnote, SyncNet) leave a corpus representative of diverse speech types and speakers.
    Section 3.2 removes side/back views, static poses, multi-speaker overlaps, and lip-sync failures; the resulting selection bias is not analyzed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Democratizing High-Fidelity Co-Speech Gesture Video Generation." pith.science (2026). https://pith.science/paper/XYM7J7IX

@misc{pith2026250706812,
  author       = {Pith},
  title        = {Pith review of: Democratizing High-Fidelity Co-Speech Gesture Video Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XYM7J7IX}},
  note         = {Machine review of arXiv:2507.06812}
}
read the original abstract

Co-speech gesture video generation aims to synthesize realistic, audio-aligned videos of speakers, complete with synchronized facial expressions and body gestures. This task presents challenges due to the significant one-to-many mapping between audio and visual content, further complicated by the scarcity of large-scale public datasets and high computational demands. We propose a lightweight framework that utilizes 2D full-body skeletons as an efficient auxiliary condition to bridge audio signals with visual outputs. Our approach introduces a diffusion model conditioned on fine-grained audio segments and a skeleton extracted from the speaker's reference image, predicting skeletal motions through skeleton-audio feature fusion to ensure strict audio coordination and body shape consistency. The generated skeletons are then fed into an off-the-shelf human video generation model with the speaker's reference image to synthesize high-fidelity videos. To democratize research, we present CSG-405-the first public dataset with 405 hours of high-resolution videos across 71 speech types, annotated with 2D skeletons and diverse speaker demographics. Experiments show that our method exceeds state-of-the-art approaches in visual quality and synchronization while generalizing across speakers and contexts. Code, models, and CSG-405 are publicly released at https://mpi-lab.github.io/Democratizing-CSG/

Figures

Figures reproduced from arXiv: 2507.06812 by the authors.

Figure 1
Figure 1. Examples of co-speech gesture videos created by our framework. These examples demonstrate the advantages of our framework [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. More details of CSG-405. (a) The proportion of clips for each speech type. (b) Attribute distribution in gender, ethnicity, age, [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Overview of our co-speech gesture video generation framework. We concatenate the 2D skeleton of the reference image [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: An overview of our data collection pipeline. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Qualitative comparisons between our methods and state-of-the-art methods on our CSG-405 database. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Qualitative comparisons between CyberHost [ [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Qualitative comparisons between our methods and state-of-the-art methods on PATS [ [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

54 extracted references · 50 canonical work pages

  1. [50]

    Y . Guo, M. Tan, Z. Deng, J. Wang, Q. Chen, J. Cao, Y . Xu, J. Chen. Towards Lightweight Super-Resolution With Dual Regression Learning. In TPAMI, 2024. 3

    work page 2024

  2. [1]

    Ahuja, D

    C. Ahuja, D. Lee, Y . Nakano, L. Morency. Style trans- fer for co-speech gesture animation: A multi-speaker conditional-mixture approach. In ECCV, 2020. 2, 3, 6, 7, 8

    work page 2020

  3. [2]

    Karras, A

    J. Karras, A. Holynski, T. Wang, I. Kemelmacher- Shlizerman. Dreampose: Fashion image-to-video syn- thesis via stable diffusion. In CVPR, 2023. 3

    work page 2023

  4. [3]

    X. Liu, Q. Wu, H. Zhou, Y . Du, W. Wu, D. Lin, Z. Liu. Audio-Driven Co-Speech Gesture Video Generation. In NeurIPS, 2022. 2, 3, 4, 6

    work page 2022

  5. [4]

    Bookstein

    F. Bookstein. Principal warps: Thin-plate splines and the decomposition of deformations. In TPAMI, 1989. 3

    work page 1989

  6. [5]

    Tevet, S

    G. Tevet, S. Raab, B. Gordon, Y . Shafir, D. Cohen- Or, A. Bermano. Human motion diffusion model. In ICLR, 2023. 4

    work page 2023

  7. [6]

    Y . Yoon, W. Ko, M. Jang, J. Lee, J. Kim, G. Lee. Robots learn social skills: End-to-end learning of co- speech gesture generation for humanoid robots. In ICRA, 2019. 2, 3, 6, 7, 8

    work page 2019

  8. [7]

    Siarohin, O

    A. Siarohin, O. Woodford, J. Ren, M. Chai, S. Tulyakov. Motion representations for articulated ani- mation. In CVPR, 2021. 3

    work page 2021

Show all 54 references
  1. [8]

    J. Zhao, H. Zhang. Thin-plate spline motion model for image animation. In CVPR, 2022. 3

    2022

  2. [9]

    Z. Cao, T. Simon, S. Wei, Y . Sheikh. Realtime multi- person 2d pose estimation using part affinity fields. In CVPR, 2017. 3

    2017

  3. [10]

    Z. Yang, A. Zeng, C. Yuan, Y . Li. Effective whole- body pose estimation with two-stages distillation. In ICCV, 2023. 3, 5

    2023

  4. [11]

    Siarohin, S

    A. Siarohin, S. Lathuili `ere, S. Tulyakov, E. Ricci, N. Sebe. First order motion model for image animation. In NeurIPS, 2019. 3

    2019

  5. [12]

    J. Ho, A. Jain, P. Abbeel. Denoising diffusion proba- bilistic models. In NeurIPS, 2020. 2

    2020

  6. [13]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. Gomez, L. Kaiser, I. Polosukhin. Attention is all you need. In NeurIPS, 2017. 3, 5, 8

    2017

  7. [14]

    Kingma, J

    D. Kingma, J. Ba. Adam: A method for stochastic op- timization. In ICLR, 2015. 6

    2015

  8. [15]

    Unterthiner, S

    T. Unterthiner, S. VanSteenkiste, K. Kurach, R. Marinier, M. Michalski, S. Gelly. Fvs: a new metric for video generation. In ICLR, 2019. 6

    2019

  9. [16]

    L. Hu, X. Gao, P. Zhang, K. Sun, B. Zhang, L. Bo. An- imate Anyone: Consistent and Controllable Image-to- Video Synthesis for Character Animation. In CVPR,

  10. [17]

    Rombach, A

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, B. Ommer, High-resolution image synthesis with latent diffusion models. In CVPR, 2022. 2, 3, 4

    2022

  11. [18]

    Hogue, C

    S. Hogue, C. Zhang, H. Daruger, Y . Tian, X. Guo. DiffTED: One-shot Audio-driven TED Talk Video Generation with Diffusion-based Co-speech Gestures. In CVPR Workshop, 2024. 2, 3, 6, 7

    2024

  12. [19]

    X. He, Q. Huang, Z. Zhang, Z. Lin, Z. Wu, S. Yang, M. Li, Z. Chen, S. Xu, X. Wu. Co-Speech Gesture Video Generation via Motion-Decoupled Diffusion Model. In CVPR, 2024. 2, 3, 4, 6, 7

    2024

  13. [20]

    Z. Xu, J. Zhang, J. Liew, H. Yan, J. Liu, C. Zhang, J. Feng, M. Shou. Magicanimate: Temporally consis- tent human image animation using diffusion model. In CVPR, 2024. 2, 3

    2024

  14. [21]

    Zhang, J

    Y . Zhang, J. Gu, L. Wang, H. Wang, J. Cheng, Y . Zhu, F. Zou. Mimicmotion: High-quality human motion video generation with confidence-aware pose guid- ance. In ICML, 2024. 2, 3, 5, 6, 7

    2024

  15. [22]

    S. Zhu, J. Chen, Z. Dai, Y . Xu, X. Cao, Y . Yao, H. Zhu, S. Zhu. Champ: Controllable and consistent hu- man image animation with 3d parametric guidance. In ECCV, 2024. 2, 3

    2024

  16. [23]

    G. Lin, J. Jiang, C. Liang, T. Zhong, J. Yang, Y . Zheng. CyberHost: Taming Audio-driven Avatar Dif- fusion Model with Region Codebook Attention. In ICLR, 2025. 2, 3, 6, 8

    2025

  17. [24]

    Corona, A

    E. Corona, A. Zanfir, E. Bazavan, N. Kolotouros, T. Alldieck, C. Sminchisescu. VLOGGER: Multi- modal diffusion for embodied avatar synthesis. In arXiv:2403.08764, 2024. 2, 3, 6, 8

    2024 arXiv

  18. [25]

    Peebles, S

    W. Peebles, S. Xie. Scalable diffusion models with transformers. In ICCV, 2023. 4

    2023

  19. [26]

    S. Qian, Z. Tu, Y . Zhi, W. Liu, S. Gao. Speech drives templates: Co-speech gesture synthesis with learned templates. In ICCV, 2021. 5, 8

    2021

  20. [27]

    Y . Liu, Q. Cao, Y . Wen, H. Jiang, C. Ding. Towards variable and coordinated holistic co-speech motion generation. In CVPR, 2024. 4

    2024

  21. [28]

    Z. Chen, J. Cao, Z. Chen, Y . Li, C. Ma. Echomimic: Lifelike audio-driven portrait animations through ed- itable landmark conditions. In AAAI, 2025. 2, 3, 4

    2025

  22. [29]

    L. Tian, Q. Wang, B. Zhang, L. Bo. Emo: Emote por- trait alive-generating expressive portrait videos with audio2video diffusion model under weak conditions. In ECCV, 2024. 3, 4

    2024

  23. [30]

    Y . Ma, H. Liu, H. Wang, H. Pan, Y . He, J. Yuan, A. Zeng, C. Cai, H. Shum, W. Liu, Q. Chen. Follow-Your-Emoji: Fine-Controllable and Expres- sive Freestyle Portrait Animation. In SIGGRAPH Asia, 2024. 2, 3, 4

    2024

  24. [31]

    Loper, N

    M. Loper, N. Mahmood, J. Romero, G. Pons-Moll, M. Black. SMPL: A skinned multi-person linear model. In TOG, 2015. 3

    2015

  25. [32]

    Pavlakos, V

    G. Pavlakos, V . Choutas, N. Ghorbani, T. Bolkart, A. Osman, D. Tzionas, M. Black. Expressive body cap- ture: 3d hands, face, and body from a single image. In CVPR, 2019. 3

    2019

  26. [33]

    Karras, T

    T. Karras, T. Aila, S. Laine, J. Lehtinen. Progressive 9 growing of gans for improved quality, stability, and variation. In ICLR, 2018. 5

    2018

  27. [34]

    H. Zhu, W. Wu, W. Zhu, L. Jiang, S. Tang, L. Zhang, Z. Liu, C. Loy. CelebV-HQ: A large-scale video facial attributes dataset. In ECCV, 2022. 5

    2022

  28. [35]

    Serengil, A

    S. Serengil, A. Ozpinar. A Benchmark of Facial Recognition Pipelines and Co-Usability Performances of Modules. In Bilisim Teknolojileri Dergisi, 2024. 5

    2024

  29. [36]

    G ¨uler, N

    R. G ¨uler, N. Neverova, I. Kokkinos. Densepose: Dense human pose estimation in the wild. In CVPR,

  30. [37]

    Zhang, X

    W. Zhang, X. Cun, X. Wang, Y . Zhang, X. Shen, Y . Guo, Y . Shan, F. Wang. Sadtalker: Learning realistic 3d motion coefficients for stylized audio-driven single image talking face animation. In CVPR, 2023. 3, 4

    2023

  31. [38]

    H. Wei, Z. Yang, Z. Wang. Aniportrait: Audio- driven synthesis of photorealistic portrait animation. In arXiv:2403.17694, 2024. 3, 4

    2024 arXiv

  32. [39]

    Egger, W

    B. Egger, W. Smith, A. Tewari, S. Wuhrer, M. Zoll- hoefer, T. Beeler, F. Bernard, T. Bolkart, A. Ko- rtylewski, S. Romdhani, C. Theobalt, V . Blanz, T. Vet- ter. 3d morphable face models—past, present, and fu- ture. In TOG, 2020. 4

    2020

  33. [40]

    Heusel, H

    M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, S. Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In NeurIPS,

  34. [41]

    Zhang, Z

    H. Zhang, Z. Liang, R. Fu, B. Liu, Z. Wen, X. Liu, C. Li, Y . Liang. Efficient Long-duration Talking Video Synthesis with Linear Diffusion Transformer under Multimodal Guidance. In https://arxiv.org/abs/2411.16748, 2024. 2, 4

    2024 arXiv

  35. [42]

    R. Meng, X. Zhang, Y . Li, C. Ma. EchoMimicV2: Towards Striking, Simplified, and Semi-Body Human Animation. In CVPR, 2025. 2, 3, 5, 6, 7

    2025

  36. [43]

    Achiam, S

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, Gpt-4 technical report. In arXiv:2303.08774, 2023. 5

    2023 arXiv

  37. [44]

    S. Jin, L. Xu, J. Xu, C. Wang, W. Liu, C. Qian, W. Ouyang, P. Luo. Whole-body human pose estimation in the wild. In ECCV, 2020. 5

    2020

  38. [45]

    Plaquet, H

    A. Plaquet, H. Bredin. Powerset multi-class cross en- tropy loss for neural speaker diarization. In INTER- SPEECH, 2023. 5

    2023

  39. [46]

    J. hung, A. Zisserman. Out of time: automated lip sync in the wild. In ACCV Workshop, 2016. 5, 6

    2016

  40. [47]

    Baevski, Y

    A. Baevski, Y . Zhou, A. Mohamed, M. Auli. wav2vec 2.0: A framework for self-supervised learning of speech representations. In NeurIPS, 2020. 4

    2020

  41. [48]

    J. Ho, T. Salimans. Classifier-free diffusion guidance. In NeurIPS Workshop, 2021. 5

    2021

  42. [49]

    S. Tu, Z. Xing, X. Han, Z. Cheng, Q. Dai, C. Luo, Z. Wu. StableAnimator: High-Quality Identity- Preserving Human Image Animation. In CVPR, 2025. 2, 3, 4, 5, 6, 7

    2025

  43. [51]

    J. Guan, Q. Yang, K. Wang, H. Zhou, S. He, Z Xu, H. Feng, E. Ding, J. Wang, H. Xie, Y . Zhao, Z. Liu. TALK-Act: Enhance Textural-Awareness for 2D Speaking Avatar Reenactment with Diffusion Model. In SIGGRAPH Asia, 2024. 3

    2024

  44. [52]

    Huang, F

    Z. Huang, F. Tang, Y . Zhang, X. Cun, J. Cao, J. Li, T. Lee. Make-your-anchor: A diffusion-based 2d avatar generation framework. In CVPR, 2024. 3

    2024

  45. [53]

    Z. Wang, A. Bovik, H. Sheikh, E. Simoncelli. Image quality assessment: from error visibility to structural similarity. In TIP, 2004. 6

    2004

  46. [54]

    Hor ´e, D

    A. Hor ´e, D. Ziou. Image quality metrics: PSNR vs. SSIM. In ICPR, 2010. 6 10

    2010

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.