Pith. sign in

REVIEW 5 major objections 5 minor 13 cited by

Long-Term TalkingFace Generation via Motion-Prior Conditional Diffusion Model

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Two motion priors keep talking-face video stable for 7,200 frames: MCDM conditions a diffusion video generator on an archived clip and a present-clip motion diffusion model, adding a memory-updated temporal attention that prevents drift.

desk verdict Credible systems paper with a genuinely useful dataset, but an unspecified training procedure in Stage3 could undercut the headline gains. read the letter →

arxiv 2502.09533 v1 pith:YBGCXD2D submitted 2025-02-13 cs.CV

Fei Shen , Cong Wang , Junyao Gao , Qin Guo , Jisheng Dang , Jinhui Tang , Tat-Seng Chua This is my paper
classification cs.CV
keywords talkingfacegenerationlong-termvideodiffusionmodelsmotionpriortemporalconsistencyaudio-drivenanimationmultilingualdatasetidentitypreservation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that long-term audio-driven talking-face generation can be made stable and identity-preserving by conditioning the video generator on two kinds of motion priors: an archived clip of historical frames and a present clip produced by a small auxiliary diffusion model that predicts head, lip, and expression tokens. If true, this would let a single reference image and audio track produce coherent minutes-long talking-head video without drift, blurring, or identity leakage, a bottleneck for virtual avatars and dubbing. The authors also release a multilingual dataset of over 200 hours across 10 languages, and report top scores on established benchmarks plus the new set. The central claim includes the specific mechanism that separating identity and motion and updating a motion memory prevents the error accumulation that limits existing short-horizon diffusion approaches.

What carries the argument

The central object is the two-motion-prior conditioning scheme. The archived-clip motion-prior uses frame-aligned attention where each historical frame's tokens $F^i_a$ are paired with the reference tokens $F_x$ as key and value, giving each generated frame access to long-term identity context. The present-clip motion-prior is a separate small diffusion model optimized by the objective in Eq. (3) to denoise noise-added head, lip, and expression tokens, thereby decoupling motion from identity. The memory-efficient temporal attention maintains a running memory $M_f$ that exponentially blends archived features and feeds them together with present-clip features into a Performer-style fast attention along the time axis, which is what prevents error accumulation over thousands of frames.

What would settle it

Run the trained model on a 7,200-frame sequence after replacing the present-clip motion tokens with fixed Gaussian noise of the same shape. If Sync-C and SSIM stay within the reported margins, the present-clip motion-prior is not carrying the causal motion information the paper claims; if they collapse, the mechanism is load-bearing. A complementary check is to measure correlation between predicted motion tokens and ground-truth landmark trajectories on a held-out set: when that correlation decays with sequence length, the long-term causal claim fails.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that long-term talking-face generation can be cast as a conditional diffusion problem with two motion-prior streams. The archived-clip motion-prior runs frame-aligned attention between a reference frame and historical frames, giving the UNet a compact long-term identity and context anchor. The present-clip motion-prior is a separate small diffusion model that denoises tokens for head, lip, and expression motion (Eq. 3), conditioned on audio, reference image, and optional landmarks, thereby separating identity from motion. A memory-efficient temporal attention keeps a running motion memory $M_f = \alpha M_{f-1} + (1-\alpha)F^{\text{ref}}_{ac}$, concatenates it with the present-clip features, and applies fast attention over the time axis. With these three modules, MCDM reports the best FID, FVD, Sync-C, Sync-D, SSIM, and E-FID on HDTF, CelebV-HQ, and the new TalkingFace-Wild dataset, and shows SSIM staying stable out to 7,200 frames when the archived clip is used.

Load-bearing premise

The pipeline rests on the assumption that the motion tokens denoised by the small present-clip model stay informative when injected into the main video UNet, so that token-level motion prediction actually improves pixel-level head, lip, and expression motion rather than just conditioning noise.

Editorial extensions

If this is right

  • Long sequences (thousands of frames) can be generated from a single reference image and audio with stable structural similarity, assuming the archived clip is maintained.
  • The present-clip motion-prior lets the model accept or skip landmark guidance without retraining, so the same pipeline covers both driven and fully audio-driven animation.
  • The TalkingFace-Wild dataset provides a 10-language, 200+ hour benchmark for testing generalization beyond English-centric datasets.
  • Because identity and motion are decoupled, the method should be applicable to other avatar animation tasks that currently rely on short frame histories.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable prediction follows from the decoupling claim: corrupting only the motion tokens should lower lip-sync and head-motion metrics while leaving identity metrics roughly intact; corrupting only the identity tokens should do the reverse. The paper shows token-level heatmaps and clusters but does not run this causal swap experiment.
  • The memory weight $\alpha=0.1$ is a sensitive hyperparameter; the paper's own plot suggests that values near 0.9 erode the benefit, which implies that in practice the memory update needs tuning per video length or speaking style.
  • The SSIM-vs-length curves (Figure 5) are measured against a reference video; a fully open-set long-term evaluation without ground truth would require a no-reference metric, so the practical stability claim depends on how faithfully the reference video is followed.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes MCDM, a diffusion-based framework for long-term audio-driven talking-face video generation. The architecture combines three components: an archived-clip motion-prior that injects long-term historical frames via frame-aligned attention, a present-clip motion-prior diffusion model that predicts head/lip/expression motion tokens from audio and image tokens, and a memory-efficient temporal attention module with a fixed exponential update rule to reduce error accumulation over long sequences. Training is split into three stages, and inference first samples motion tokens from the present-clip prior and then denoises video latents conditioned on these tokens. The paper also introduces TalkingFace-Wild, a multilingual dataset of over 200 hours across 10 languages. Experiments compare MCDM with seven baselines on HDTF, CelebV-HQ, and TalkingFace-Wild, reporting improvements across FID, FVD, Sync-C, Sync-D, SSIM, and E-FID, together with ablations, a user study, and a long-sequence SSIM analysis up to 7200 frames.

Significance. If the reported results hold, MCDM would be a meaningful advance for long-term talking-face generation, and the TalkingFace-Wild dataset would be a useful multilingual benchmark. The paper has clear strengths: the three-stage design is well motivated, the ablations are logically organized, the comparisons cover the main recent baselines, and the authors commit to releasing code, models, and data. However, the central mechanism for transferring motion-prior tokens into the video UNet is underspecified and untested, the main quantitative claims lack error bars or significance tests, and the long-term evidence is limited to an internal ablation. These issues are fixable but currently prevent acceptance.

major comments (5)
  1. [3.5 (Stage3)] The paper does not state whether the present-clip reference attention in Stage3 is trained on ground-truth head/lip/expression tokens extracted by the Section 3.3 encoders or on tokens sampled from the Stage2 motion-prior. At inference, only sampled tokens are available, so if training used ground-truth tokens, the measured Sync-C/Sync-D and SSIM gains in Tables 2 and 3 could be a teacher-forcing artifact. The w/o Fpc ablation in Table 4 replaces the pathway with a directly injected audio-attention module and therefore changes the architecture rather than the token source; Figure 6 shows token-space heatmaps and t-SNE clusters, not pixel-level benefit. The token-to-video transfer step is thus unspecified and untested and must be clarified and ablated.
  2. [Tables 2-5] No error bars, confidence intervals, or significance tests are reported for any metric. Several claimed wins are small in absolute terms, for example Sync-D 8.04 vs 8.55 on HDTF (Table 2) and SSIM 0.779 vs 0.745 on TalkingFace-Wild (Table 3). Without repeated runs or statistical testing, the claim that MCDM achieves the top results across all metrics is not empirically supported.
  3. [4.3 and Table 3] The memory coefficient alpha in Eq. 5 is tuned by sweeping SSIM on the same TalkingFace-Wild evaluation set that is later used for the main comparison in Table 3 (Figure 7). This selection-on-test-set procedure can inflate the reported gains, and the sentence 'Consequently, we set alpha = 0.1 as the default value' should be backed by a held-out validation split or a nested evaluation protocol.
  4. [Figure 5] The central long-term claim of the paper is supported only by the internal w/ Fa vs w/o Fa ablation with SSIM reported on frames of a single sequence up to 7200 frames. There is no long-sequence comparison against the baselines from Tables 2 and 3, and no FVD or Sync metrics at long horizons. The paper should add long-term quantitative comparisons with baselines to substantiate the 'long-term TalkingFace generation' claim.
  5. [4.1] The symbol alpha is used inconsistently: the text says 'we set alpha = 16' for the archived-clip frame count, while Eq. 5 defines alpha in [0,1] and the same paragraph later sets 'the weighting factor alpha in Eq. 5' to 0.1. This makes the memory update and the archived-clip configuration ambiguous; the archived frame count should use a distinct symbol such as a, and the memory coefficient should consistently be alpha.
minor comments (5)
  1. [3.3, Eq. 3] The expectation notation Et,Fp,zt,epsilon is not standard; it should be written as E_{t,Fp,zt,epsilon} with the distributions of all variables specified. The current notation leaves unclear whether the expectation is over the noise, the timestep, the conditioning tokens, or all of them.
  2. [4.2] The user study is described only as 20 participants providing rank-3 preferences on 10 cases; please report inter-subject agreement or confidence intervals, and clarify how ties are handled.
  3. [1 and Table 1] There is a typo in the language distribution figure: 'Spanlish' should be 'Spanish'. In addition, the dataset statistics in Table 1 should specify the license and the de-identification procedure for the YouTube-sourced footage.
  4. [4.3] The description of w/o Fpc as 'adds an audio attention module for audio feature input' is confusing: the notation suggests the present-clip prior is removed, but the text does not clearly state whether the remaining audio conditioning replaces Fpc tokens or is added in parallel. Please rephrase for clarity.
  5. [5] The conclusion states that MCDM 'sets new benchmarks' without qualifying that the claims depend on the alpha selection and training details described in the major comments; a brief limitation paragraph on the teacher-forcing question and the lack of error bars would improve the manuscript.

Circularity Check

0 steps flagged · score 2.0 of 10

No derivation-level circularity; the empirical SOTA claim is self-contained against external benchmarks. Score reflects one minor non-load-bearing self-citation (V-Express) and a flagged teacher-forcing specification gap.

full rationale

MCDM is presented as an empirical system, not a formal derivation. The central claims — Tables 2 and 3, and Figure 5 — are held-out comparisons against external benchmarks and baselines; no equation in the paper defines an output in terms of the claimed result, and no fitted parameter is renamed as a prediction. The motion-prior loss in Eq. (3) is a standard denoising objective on head/lip/expression tokens, the memory update in Eqs. (4)-(5) is a fixed recurrence, and the archived-clip attention in Eqs. (1)-(2) is a conventional cross-attention mechanism with no identity between input and output. The one self-citation is V-Express (Wang et al., 2024), co-authored by two current authors (Cong Wang and Fei Shen), but it is used only as a comparison baseline; the claimed advantage is measured by external metrics on held-out data, so this is a disclosure/fairness concern rather than a circular reduction. I also flag a non-circular limitation: Section 3.5 does not state whether Stage3 trains the present-clip reference attention on ground-truth motion tokens or on tokens sampled from the Stage2 motion-prior, and the w/o Fpc ablation in Table 4 changes the architecture instead of varying the token source, so the reported gains could partly reflect a teacher-forcing mismatch. This is an omitted specification, not a self-referential derivation. Score 2 rather than 0 solely because of the minor self-citation and the need for explicit disclosure; the derivation chain itself is not circular.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The central claim rests on several domain assumptions: the SD v1.5 latent space transfers to multi-frame video, the token-level motion-prior loss produces useful conditioning for pixel generation, the fixed exponential memory summarizes long-term context, and the chosen metrics reflect human-preferred quality. No new physical entities are introduced. The only explicitly hand-tuned values that affect the headline results are the memory coefficient alpha and the archived frame count, both set on the authors' datasets.

free parameters (3)
  • Memory update coefficient alpha (Eq. 5) = 0.1
    Chosen by inspecting SSIM versus alpha on TalkingFace-Wild in Figure 7; the paper then fixes this value for the main results. This is model selection on the evaluation set.
  • Archived clip frame count (written alpha=16 in Section 4.1) = 16 frames
    Number of historical frames stored in the archived-clip motion-prior; chosen by the authors. The text uses alpha where the architecture diagram and Eq. 1 use a, creating an inconsistency.
  • Number of patchify tokens m and query tokens n = m=256, n=16
    Hand-set architectural dimensions in the archived-clip module that affect the representation capacity.
assumptions (6)
  • domain assumption Stable Diffusion v1.5 VAE and UNet can serve as a multi-frame video denoising backbone without architectural changes beyond added attention layers.
    Section 3.1 says the denoising UNet 'resembles SD v1.5'; the paper does not justify why image-latent priors transfer to temporal video latents.
  • domain assumption The Lprior MSE denoising loss in Eq. 3 trains motion tokens that capture head, lip, and expression information sufficient for later video generation.
    Section 3.3 defines the loss; no independent evidence links token-level reconstruction to downstream video quality.
  • domain assumption The fixed exponential moving average in Eq. 5 is a sufficient summary of long-term motion context.
    Section 3.4; the choice of alpha is empirical and no theoretical bound on error accumulation is given.
  • domain assumption FID, FVD, Sync-C, Sync-D, SSIM and E-FID are reliable proxies for talking-face quality.
    Section 4.1 lists these metrics without validating them against human perception for this specific task.
  • domain assumption SyncNet filtering and manual inspection produce a dataset with sufficient audio-lip alignment to train and evaluate.
    Section 4.1 'Data Processing'; this curation may remove hard cases and bias results.
  • standard math Performer fast attention provides a sufficient approximation to exact softmax attention in the temporal layer.
    Section 3.4 replaces temporal self-attention with fast attention (Choromanski et al., 2020); quality loss is not measured.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Long-Term TalkingFace Generation via Motion-Prior Conditional Diffusion Model." pith.science (2026). https://pith.science/paper/YBGCXD2D

@misc{pith2026250209533,
  author       = {Pith},
  title        = {Pith review of: Long-Term TalkingFace Generation via Motion-Prior Conditional Diffusion Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YBGCXD2D}},
  note         = {Machine review of arXiv:2502.09533}
}
read the original abstract

Recent advances in conditional diffusion models have shown promise for generating realistic TalkingFace videos, yet challenges persist in achieving consistent head movement, synchronized facial expressions, and accurate lip synchronization over extended generations. To address these, we introduce the \textbf{M}otion-priors \textbf{C}onditional \textbf{D}iffusion \textbf{M}odel (\textbf{MCDM}), which utilizes both archived and current clip motion priors to enhance motion prediction and ensure temporal consistency. The model consists of three key elements: (1) an archived-clip motion-prior that incorporates historical frames and a reference frame to preserve identity and context; (2) a present-clip motion-prior diffusion model that captures multimodal causality for accurate predictions of head movements, lip sync, and expressions; and (3) a memory-efficient temporal attention mechanism that mitigates error accumulation by dynamically storing and updating motion features. We also release the \textbf{TalkingFace-Wild} dataset, a multilingual collection of over 200 hours of footage across 10 languages. Experimental results demonstrate the effectiveness of MCDM in maintaining identity and motion continuity for long-term TalkingFace generation. Code, models, and datasets will be publicly available.

Figures

Figures reproduced from arXiv: 2502.09533 by the authors.

Figure 1
Figure 1. Our MCDM architecture. On the upper, the archived-clip motion-prior leverages frame-aligned attention with archived-clip, enhancing identity coherence over extended sequences. On the right, the present-clip motion-prior diffusion model uses multimodal causality and temporal interactions to decouple and predict motion states, covering head, lip, and expression movements while maintaining a clear separation of identit… view at source ↗
Figure 2
Figure 2. The overview of memory-efficient temporal attention. It can dynamically update and integrate historical motion features with current ones. defined as: Lprior = Et,Fp,zt,ϵ, ∥ϵ − ϵθ (zt, t, Fp)∥ 2 . (3) Without landmark guidance, Fp represent multimodal in￾teraction tokens from audio and the reference frame. zt represent noise-added tokens for head, lip, and expression movements at timestep t. With landmark guidance, … view at source ↗
Figure 3
Figure 3. Qualitative comparison on HDTF and CelebV-HQ. Our method achieves the best generation results, particularly in identity consistency and motion detail. 0 0.2 0.4 0.6 0.8 1 A B C D E F G H Rank-3 Identity Consistency 0 0.2 0.4 0.6 0.8 1 A B C D E F G H Rank-3 Motion Synchronization 0 0.2 0.4 0.6 0.8 1 A B C D E F G H Rank-3 Video Quality A: Audio2Head B: V-Express C: AniPortrait D: SadTalker E: Hallo F: EchoMimic G: M… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: User study results of identity consistency, motion synchronization, and video quality. Higher values indicate better performance. ally, unlike Hallo (Xu et al., 2024) and MegActor-Σ (Yang et al., 2024), Our method accurately captures subtle facial actions, including bl…
Figure 7
Figure 7. Figure 7: SSIM results for different α values. Our method per￾forms comparably well when the α value is smaller than 0.9. Method FID↓ FVD↓ Sync-C↑ Sync-D↓ SSIM↑ E-FID↓ B1 42.49 668.24 7.69 6.78 0.771 2.02 B2 47.12 721.17 7.30 6.84 0.732 2.29 Ours 42.08 656.71 7.84 6.69 0.779 1.9…
Figure 6
Figure 6. Figure 6: Lip heatmap and expression cluster. We find that w/ Fp effectively tracks the lip region and conveys expressions. ure 5(a) indicates that without the archived-clip (w/o Fa), identity consistency worsens with frame progression, result￾ing in visible artifacts and incons…

Discussion (0). Sign in to comment.

Forward citations

Cited by 13 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Geometry-guided Emotion Modulation for Controllable and Photorealistic Emotional Talking Face Generation

    cs.CV 2026-08 conditional novelty 6.0 of 10

    A diffusion-based talking-face generator uses 3D blendshape coefficients to continuously control the emotion intensity of generated facial expressions.

  2. ACM-UNet: Adaptive Integration of CNNs and Mamba for Efficient Medical Image Segmentation

    cs.CV 2025-05 conditional novelty 5.0 of 10

    ACM-UNet, a UNet variant combining pretrained CNN and Mamba backbones via lightweight adapters and a wavelet decoder module, reports 85.12% Dice on Synapse and 92.29% on ACDC.

  3. FaceEditTalker: Controllable Talking Head Generation with Facial Attribute Editing

    cs.CV 2025-05 conditional novelty 5.0 of 10

    A single framework can edit predefined facial attributes in audio-synchronized talking head videos while preserving identity and lip-sync quality.

  4. PQ-DAF: Pose-driven Quality-controlled Data Augmentation for Data-scarce Driver Distraction Detection

    cs.CV 2025-08 reject novelty 4.0 of 10

    PQ-DAF uses pose-conditioned diffusion generation plus CogVLM filtering to augment few-shot driver distraction training data, and reports large accuracy gains that are compromised by a non-standard train/test protocol.

  5. Hybrid Compact Least-Squares and Central Weighted Essentially Non-Oscillatory Schemes for Hyperbolic Conservation Laws on Structured Curvilinear Grids

    physics.flu-dyn 2025-08 reject novelty 4.0 of 10

    No verifiable result: the abstract and body address unrelated topics, so the claimed CLS-CWENO schemes appear without derivation, experiments, or benchmarks.

  6. FashionPose: Unified Text-Driven Fashion Synthesis with Joint Geometric and Photometric Control

    cs.CV 2025-07 reject novelty 4.0 of 10

    A single caption can drive pose generation, person-image synthesis, and relighting through a three-stage FashionPose pipeline, with reported text-to-pose gains on DF-PASS that are undermined by inconsistent tables.

  7. DiffFit: Disentangled Garment Warping and Texture Refinement for Virtual Try-On

    cs.CV 2025-06 reject novelty 4.0 of 10

    DiffFit synthesizes virtual try-on images by separately warping the garment geometry and then refining texture with a conditional diffusion model, reporting SOTA metrics on VITON-HD and DressCode but with inconsistent...

  8. O2Former:Direction-Aware and Multi-Scale Query Enhancement for SAR Ship Instance Segmentation

    cs.CV 2025-06 conditional novelty 4.0 of 10

    O2Former adds a multi-scale query generator and an orientation-aware module to Mask2Former and reports improved SAR ship instance segmentation on SSDD and HRSID.

  9. Dual Attention Residual U-Net for Accurate Brain Ultrasound Segmentation in IVH Detection

    eess.IV 2025-05 conditional novelty 4.0 of 10

    A residual U-Net with CBAM and a dual-branch sparse/dense attention layer reports Dice 89.04 and IoU 81.84 on brain ultrasound ventricle segmentation.

  10. YOLO-FDA: Integrating Hierarchical Attention and Detail Enhancement for Surface Defect Detection

    cs.CV 2025-06 conditional novelty 3.0 of 10

    A YOLOv5 variant with BiFPN, directional detail enhancement, and two attention fusion modules reports state-of-the-art mAP on GC10-DET and DAGM2007.

  11. MCFNet: A Multimodal Collaborative Fusion Network for Fine-Grained Semantic Classification

    cs.CV 2025-05 conditional novelty 3.0 of 10

    MCFNet fuses ALBERT text features and ViT image features with dropout, L1/L2 regularization, hybrid self/cross attention, and multi-loss training, claiming state-of-the-art accuracy on Con-Text and Drink Bottle.

  12. YOLO-SPCI: Enhancing Remote Sensing Object Detection via Selective-Perspective-Class Integration

    cs.CV 2025-05 conditional novelty 3.0 of 10

    YOLO-SPCI, a YOLOv8 variant with a three-branch attention module at backbone stages P3 and P5, reports 92.0% mAP50 on NWPU VHR-10 versus 88.9% for the baseline.

  13. FreqU-FNet: Frequency-Aware U-Net for Imbalanced Medical Image Segmentation

    eess.IV 2025-05 reject novelty 3.0 of 10

    FreqU-FNet mixes frequency-domain filters and adaptive upsampling inside a U-Net, and claims better minority-class segmentation, although key reported results are internally inconsistent.

Reference graph

Works this paper leans on

19 extracted references · 2 canonical work pages · cited by 13 Pith papers

  1. [6]

    Animatediff: Animate your personalized text-to-image diffusion models without specific tuning

    Guo, Y ., Yang, C., Rao, A., Liang, Z., Wang, Y ., Qiao, Y ., Agrawala, M., Lin, D., and Dai, B. Animatediff: Animate your personalized text-to-image diffusion models without specific tuning. arXiv preprint arXiv:2307.04725,

    arXiv

  2. [7]

    Loopy: Taming audio-driven portrait avatar with long-term motion dependency

    Jiang, J., Liang, C., Yang, J., Lin, G., Zhong, T., and Zheng, Y . Loopy: Taming audio-driven portrait avatar with long-term motion dependency. arXiv preprint arXiv:2409.02634,

    arXiv

  3. [9]

    Follow-your- emoji: Fine-controllable and expressive freestyle portrait animation

    Ma, Y ., Liu, H., Wang, H., Pan, H., He, Y ., Yuan, J., Zeng, A., Cai, C., Shum, H.-Y ., Liu, W., et al. Follow-your- emoji: Fine-controllable and expressive freestyle portrait animation. arXiv preprint arXiv:2406.01900,

    arXiv

  4. [10]

    and Osindero, S

    Mirza, M. and Osindero, S. Conditional generative adver- sarial nets. arXiv preprint arXiv:1411.1784,

    arXiv

  5. [11]

    Vividtalk: One- shot audio-driven talking head generation based on 3d hybrid prior

    Sun, X., Zhang, L., Zhu, H., Zhang, P., Zhang, B., Ji, X., Zhou, K., Gao, D., Bo, L., and Cao, X. Vividtalk: One- shot audio-driven talking head generation based on 3d hybrid prior. arXiv preprint arXiv:2312.01841,

    arXiv

  6. [12]

    Emo: Emote por- trait alive-generating expressive portrait videos with au- dio2video diffusion model under weak conditions

    Tian, L., Wang, Q., Zhang, B., and Bo, L. Emo: Emote por- trait alive-generating expressive portrait videos with au- dio2video diffusion model under weak conditions. arXiv preprint arXiv:2402.17485,

    arXiv

  7. [13]

    V-express: Con- ditional dropout for progressive training of portrait video generation

    Wang, C., Tian, K., Zhang, J., Guan, Y ., Luo, F., Shen, F., Jiang, Z., Gu, Q., Han, X., and Yang, W. V-express: Con- ditional dropout for progressive training of portrait video generation. arXiv preprint arXiv:2406.02511,

    arXiv

  8. [14]

    Audio2head: Audio-driven one-shot talking-head generation with nat- ural head motion

    10 Motion-Prior Conditional Diffusion Model Wang, S., Li, L., Ding, Y ., Fan, C., and Yu, X. Audio2head: Audio-driven one-shot talking-head generation with nat- ural head motion. arXiv preprint arXiv:2107.09293 , 2021a. Wang, T.-C., Mallya, A., and Liu, M.-Y . One-shot free-view neural talking-head synthesis for video conferencing. In Proceedings of the I...

    arXiv

Show all 19 references
  1. [16]

    Hallo: Hierarchical audio-driven visual synthesis for portrait image animation

    Xu, M., Li, H., Su, Q., Shang, H., Zhang, L., Liu, C., Wang, J., Van Gool, L., Yao, Y ., and Zhu, S. Hallo: Hierarchical audio-driven visual synthesis for portrait image animation. arXiv preprint arXiv:2406.08801,

    arXiv

  2. [17]

    Megactor-sigma: Unlocking flexible mixed-modal control in portrait animation with diffusion transformer

    Yang, S., Li, H., Wu, J., Jing, M., Li, L., Ji, R., Liang, J., Fan, H., and Wang, J. Megactor-sigma: Unlocking flexible mixed-modal control in portrait animation with diffusion transformer. arXiv preprint arXiv:2408.14975,

    arXiv

  3. [19]

    Real3d-portrait: One- shot realistic 3d talking portrait synthesis

    Ye, Z., Zhong, T., Ren, Y ., Yang, J., Li, W., Huang, J., Jiang, Z., He, J., Huang, R., Liu, J., et al. Real3d-portrait: One- shot realistic 3d talking portrait synthesis. arXiv preprint arXiv:2401.08503,

    arXiv

  4. [2004]

    Aniportrait: Audio-driven synthesis of photorealistic portrait animation

    Wei, H., Yang, Z., and Wang, Z. Aniportrait: Audio-driven synthesis of photorealistic portrait animation. arXiv preprint arXiv:2403.17694,

    arXiv

  5. [2014]

    Sample and computation redistribution for efficient face detection

    Guo, J., Deng, J., Lattas, A., and Zafeiriou, S. Sample and computation redistribution for efficient face detection. arXiv preprint arXiv:2105.04714,

    arXiv

  6. [2018]

    Crafting papers on machine learning

    9 Motion-Prior Conditional Diffusion Model Langley, P. Crafting papers on machine learning. In Langley, P. (ed.),Proceedings of the 17th International Conference on Machine Learning (ICML 2000), pp. 1207–1216, Stan- ford, CA,

    2000

  7. [2020]

    Chung, J. S. and Zisserman, A. Out of time: automated lip sync in the wild. In Computer Vision–ACCV 2016 Workshops: ACCV 2016 International Workshops, Taipei, Taiwan, November 20-24, 2016, Revised Selected Papers, Part II 13, pp. 251–263. Springer,

    2016

  8. [2021]

    Liveportrait: Efficient portrait animation with stitching and retargeting control

    Guo, J., Zhang, D., Liu, X., Zhong, Z., Zhang, Y ., Wan, P., and Zhang, D. Liveportrait: Efficient portrait animation with stitching and retargeting control. arXiv preprint arXiv:2407.03168,

    arXiv

  9. [2022]

    Echomimic: Lifelike audio-driven portrait animations through editable landmark conditions

    Chen, Z., Cao, J., Chen, Z., Li, Y ., and Ma, C. Echomimic: Lifelike audio-driven portrait animations through editable landmark conditions. arXiv preprint arXiv:2407.08136,

    arXiv

  10. [2023]

    Geneface: Generalized and high-fidelity audio-driven 3d talking face synthesis

    Ye, Z., Jiang, Z., Ren, Y ., Liu, J., He, J., and Zhao, Z. Geneface: Generalized and high-fidelity audio-driven 3d talking face synthesis. arXiv preprint arXiv:2301.13430,

    arXiv

  11. [2024]

    Rethinking attention with performers

    Choromanski, K., Likhosherstov, V ., Dohan, D., Song, X., Gane, A., Sarlos, T., Hawkins, P., Davis, J., Mohiuddin, A., Kaiser, L., et al. Rethinking attention with performers. arXiv preprint arXiv:2009.14794,

    2009 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.