Pith. sign in

REVIEW 3 major objections 5 minor 228 references

A five-phase pipeline organizes human motion video generation, with LLMs as motion planners.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-05 10:33 UTC pith:4HP6NQRJ

load-bearing objection Useful survey with a valuable taxonomy; its benchmark is the weak link and should be revised or dropped. the 3 major comments →

arxiv 2509.03883 v1 pith:4HP6NQRJ submitted 2025-09-04 cs.CV cs.MM

Human Motion Video Generation: A Survey

Haiwei Xue , Xiangyang Luo , Zhanghao Hu , Xin Zhang , Xunzhi Xiang , Yuqin Dai , Jianzhuang Liu , Zhensong Zhang
show 7 more authors
Minglei Li Jian Yang Fei Ma Zhiyong Wu Changpeng Yang Zonghong Dai Fei Richard Yu
This is my paper
classification cs.CV cs.MM
keywords human motion video generationsurveyfive-phase pipelinelarge language modelsmotion planningdiffusion modelspose-guided dance generationtalking head synthesis
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to give the field a shared structure by decomposing human motion video generation into five phases: input, motion planning, motion video generation, refinement, and output. It organizes more than two hundred papers into vision-, text-, and audio-driven categories and claims to be the first survey to discuss large language models as motion planners. If this framing is accepted, researchers gain a unified map for comparing methods, a taxonomy of diffusion-model architectures, and a new quantitative baseline for pose-guided dance video generation. The survey also collects 64 human-centered datasets and identifies common failure modes across current methods.

Core claim

The paper claims that existing surveys cover individual subtasks but miss the whole generative process, so it proposes a five-phase pipeline spanning every step from input to deployed output. Within this pipeline, it distinguishes two motion-planning strategies: implicit feature mapping and LLM-based planning, with the latter shown in two forms—retrieval of fine-grained motion descriptions and direct projection into a motion latent space. For the generation phase, it classifies diffusion-based pose-driven methods by what feeds the main diffusion branch (pure noise, a reference image, or guided conditions) and by attention-fusion variants. It also contributes a comparative benchmark of nine o

What carries the argument

The central object is the five-phase generation pipeline (input, motion planning, motion video generation, refinement, output), used as the organizing scaffold for the whole survey. The paper's key analytic tools are a three-way modality classification (audio-, text-, and vision-driven) and a taxonomy of diffusion-based frameworks, including three input schemes for the main diffusion branch and five attention-fusion arrangements. For motion planning, the distinguishing mechanism is the LLM planner, which either retrieves motion descriptions from a database or projects them into a generative model's latent space.

Load-bearing premise

The comparative benchmark assumes that an unofficial reimplementation of Animate Anyone faithfully represents the original model and that ten TikTok test videos are enough to rank the nine methods.

What would settle it

Rerun the nine pose-guided methods on a larger and independently annotated dance test set using official model weights wherever they exist; if the top rankings across SSIM, PSNR, LPIPS, FID, and FID-VID change materially, the survey's comparative conclusions about MagicAnimate and UniAnimate would not generalize.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Any new method can be located within the five-phase pipeline and compared against its neighbors, giving the field a common coordinate system.
  • LLM-based motion planning becomes a distinct research branch, with retrieval-style and generative-style planners as two identifiable architectures.
  • The nine-method benchmark gives practitioners a quantitative starting point: MagicAnimate for structural fidelity, UniAnimate for perceptual and temporal quality, and Animate Anyone for pixel-level closeness to ground truth.
  • The catalog of common failure modes—hand blur, facial distortion, identity or background inconsistency, limb dislocation, and low resolution—serves as a concrete checklist for evaluating new methods.
  • The 64-dataset collection supplies a fast route to training data, including resolution, duration, and task-support annotations.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the five-phase framing takes hold, motion planning may be evaluated separately from video rendering, leading to benchmarks that score planner quality (e.g., retrieval rank, instruction adherence) independently of pixel-level metrics.
  • The LLM-planner direction suggests a future separation of semantic motion specification from appearance rendering, which would let planners and generators be upgraded independently.
  • The benchmark could naturally extend to audio-driven and text-driven methods under the same metrics, enabling cross-modal comparison that the current pose-guided-only setup does not provide.
  • The survey's deliberate exclusion of 3D skeleton motion, NeRF, and 3D Gaussian splatting leaves room for a complementary survey that bridges the 2D video and 3D representation worlds.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This manuscript surveys human motion video generation, organizing the field into five phases (input, motion planning, motion video generation, refinement, output) and three driving modalities (vision, text, audio). It reviews over 200 papers, provides a table of 64 datasets, summarizes metrics, and reports a new comparative benchmark of nine pose-guided dance-video generation methods evaluated on 10 TikTok test videos with six metrics. The paper claims to be the first survey to discuss LLM-based motion planning and to provide a complete pipeline-oriented taxonomy.

Significance. If the survey's coverage and taxonomy hold, it would be a useful reference: the five-phase decomposition gives a unified vocabulary, the dataset table is extensive, and the LLM-planning discussion addresses an emerging topic. The paper ships an open repository of reviewed models. The independent benchmark is a potentially valuable practitioner resource, but its reliability is currently limited by the fragile experimental setup; it should not be used to rank methods until variance and faithfulness of the implementations are addressed.

major comments (3)
  1. [Section VI.B, Table VII, Fig. 12] The quantitative ranking relies on an unofficial reimplementation of Animate Anyone (footnote 1) and a 10-video test set. No error bars or significance tests are provided. For example, the gap between the best SSIM (MagicAnimate, 0.7558) and the second-best (UniAnimate, 0.7436) is 0.0122, which may be within run-to-run variance on 10 videos. This makes Table VII's rankings unsupported as stated. Please report variance/confidence intervals and per-video breakdowns, verify the unofficial implementation against the original (or relabel the column as the reimplementation), and either enlarge the test set or explicitly describe the benchmark as illustrative rather than comparative.
  2. [Table VII, Fig. 12, Section II scope note] Table VII lists 'StableAnimation [208]' while reference [208] is StableAnimator, and Fig. 12's caption says 'five open-source methods' although the text lists nine. These inconsistencies indicate the experimental presentation has not been carefully checked. Moreover, StableAnimator (arXiv:2411.17697) and Animate-X (arXiv:2410.10306) have arXiv dates after the stated August 30, 2024 cutoff, contradicting the paper's note that all statistics are current as of that date. Please correct the naming and reconcile the benchmark with the stated cutoff, or revise the scope statement.
  3. [Fig. 2 caption and Tables V-VI] The paper states it does not cover video generation through NeRF or 3DGS rendering, yet Table V includes SD-NeRF [138] (a NeRF-based talking-head method) and Table VI includes TalkingGaussian [142] and GaussianTalker [143] (3D Gaussian-based). This creates an internal inconsistency in the taxonomy. Either clarify the inclusion criteria for these 3D-representation methods so they are consistent with the stated scope, or remove them from the tables.
minor comments (5)
  1. [Author affiliations] Typo: 'Artificial Intelligence Innovation and Incubation (Al’) Institute' should be 'AI Institute'.
  2. [Fig. 12 caption] The caption says 'five open-source methods' but the text and Table VII describe nine methods; please align the caption with the actual content.
  3. [Tables III and V] The real-time support column is labeled 'Estimated FPS based on NVIDIA RTX 4090 GPU' but the basis of the estimate is not stated. Please specify whether these are reported by the original papers or measured here.
  4. [Fig. 11] The legend contains garbled/overlapping text, e.g., 'text Consider (ignore) the temporal dim.(text)'. Please redraw the figure for legibility.
  5. [Reference [110]] The Mamba reference (Gu and Dao) lacks a full bibliographic venue/year entry; please complete it. Also change 'StableAnimation' to 'StableAnimator' in Table VII.

Circularity Check

0 steps flagged

No significant circularity: the survey taxonomy and benchmark are independent; self-citations are not load-bearing.

full rationale

This paper is a literature survey, not a derivation chain. Its five-phase pipeline (input, motion planning, motion video generation, refinement, output) is an organizing taxonomy proposed by the authors; it is not defined in terms of a quantity the paper later predicts, and the survey does not attempt to derive the taxonomy from first principles. The novelty claims (first comprehensive five-phase survey; first survey discussing LLMs for motion planning) are literature claims supported by comparison with prior surveys [10]–[14] and by the cited LLM-based motion planning works [17]–[19]; they do not reduce to the paper's own definitions. The comparative benchmark in Section VI.B is an external empirical evaluation on the TikTok test set: the reported L1, PSNR, SSIM, LPIPS, FID, and FID-VID values are measurements of nine methods, not fitted parameters renamed as predictions. Including methods co-authored by members of this survey team (e.g., DreaMoving [97], MotionFollower [99], PoseAnimate [109], StableAnimator [208]) is normal citation practice, and none of those citations is load-bearing for the survey's central organizational or novelty claims. The concerns raised about the unofficial Animate Anyone reimplementation and the small 10-video test set are validity/reliability issues for the benchmark, not circularity: the benchmark does not assume the conclusion it reports. Accordingly, no circular step is present.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The central claim of this paper is organizational, not a quantitative derivation. It has no fitted free parameters or invented entities. Its main premises are the classification rule for driving modalities and the exclusion of NeRF/3DGS rendering, both of which are explicit scope choices.

axioms (2)
  • domain assumption Any method that includes audio input is classified as audio-driven, even if it also uses text or vision (Section I).
    This classification rule shapes the entire organization of the survey and determines which categories methods fall into. It is a reasonable editorial choice but is not derived from external constraints.
  • domain assumption The survey excludes 3D reconstruction from images and NeRF/3DGS rendering, limiting coverage to direct video generation methods (Section I).
    This scoping decision defines the survey's boundaries and affects the claim of comprehensiveness.

pith-pipeline@v1.4.0-alltime-deepseek-medium · 35528 in / 8020 out tokens · 69973 ms · 2026-08-05T10:33:18.743062+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Human Motion Video Generation: A Survey." pith.science (2026). https://pith.science/paper/4HP6NQRJ

@misc{pith2026250903883,
  author       = {Pith},
  title        = {Pith review of: Human Motion Video Generation: A Survey},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4HP6NQRJ}},
  note         = {Machine review of arXiv:2509.03883}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Human motion video generation has garnered significant research interest due to its broad applications, enabling innovations such as photorealistic singing heads or dynamic avatars that seamlessly dance to music. However, existing surveys in this field focus on individual methods, lacking a comprehensive overview of the entire generative process. This paper addresses this gap by providing an in-depth survey of human motion video generation, encompassing over ten sub-tasks, and detailing the five key phases of the generation process: input, motion planning, motion video generation, refinement, and output. Notably, this is the first survey that discusses the potential of large language models in enhancing human motion video generation. Our survey reviews the latest developments and technological trends in human motion video generation across three primary modalities: vision, text, and audio. By covering over two hundred papers, we offer a thorough overview of the field and highlight milestone works that have driven significant technological breakthroughs. Our goal for this survey is to unveil the prospects of human motion video generation and serve as a valuable resource for advancing the comprehensive applications of digital humans. A complete list of the models examined in this survey is available in Our Repository https://github.com/Winn1y/Awesome-Human-Motion-Video-Generation.

Figures

Figures reproduced from arXiv: 2509.03883 by Changpeng Yang, Fei Ma, Fei Richard Yu, Haiwei Xue, Jian Yang, Jianzhuang Liu, Minglei Li, Xiangyang Luo, Xin Zhang, Xunzhi Xiang, Yuqin Dai, Zhanghao Hu, Zhensong Zhang, Zhiyong Wu, Zonghong Dai.

Figure 1
Figure 1. Figure 1: Quantity of papers in the four categories reviewed in this survey, [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Pipeline of generating human motion videos, which can be divided into five key phases. Initially, diverse input sources such as vision cues, text [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Timeline of key advances in vision-, text-, and audio-driven human motion video generation methods. [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Different human data representations. formation of different body parts, facilitating the decoupling of distinct features. This approach enables targeted enhancements and independent manipulation of each body part, thereby refining the representation and interaction dynamics within human-centric models [78]. Optical Flow. Directly optimizing the model on a noisy dataset often leads to background instabilit… view at source ↗
Figure 6
Figure 6. Figure 6: Overview of InstructAvatar [19], which employs GPT-4 and diffusion [PITH_FULL_IMAGE:figures/full_fig_p005_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Comparative overview of different generative frameworks based on diffusion models, where pure noise (A), a reference image (B), and guided [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Different attention fusion methods of diffusion-based vision-driven [PITH_FULL_IMAGE:figures/full_fig_p009_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Different Text2Face pipelines for first-personal scripts. [PITH_FULL_IMAGE:figures/full_fig_p010_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Paradigm summary of audio-driven human motion video generation. [PITH_FULL_IMAGE:figures/full_fig_p011_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Overview of common metrics in human motion video generation. [PITH_FULL_IMAGE:figures/full_fig_p013_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Quantitative performances of five open-source methods across [PITH_FULL_IMAGE:figures/full_fig_p014_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Main challenges in human motion video generation: subpar fidelity with examples like hand blur and facial distortion, poor consistency with identity [PITH_FULL_IMAGE:figures/full_fig_p015_13.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

228 extracted references · 33 canonical work pages · 14 internal anchors

  1. [1]

    Deep video portraits,

    H. Kim, P. Garrido, A. Tewari, W. Xu, J. Thies, M. Niessner, P. Pérez, C. Richardt, M. Zollhöfer, and C. Theobalt, “Deep video portraits,” TOG, vol. 37, no. 4, pp. 1–14, 2018

    2018

  2. [2]

    Faceformer:Speech- driven 3d facial animation with transformers,

    Y.Fan,Z.Lin,J.Saito,W.Wang,andT.Komura,“Faceformer:Speech- driven 3d facial animation with transformers,” inCVPR, 2022

    2022

  3. [3]

    Make-a-video: Text-to-video generation without text-video data,

    U. Singer, A. Polyak, T. Hayes, X. Yin, J. An, S. Zhang, Q. Hu, H. Yang, O. Ashual, O. Gafni,et al., “Make-a-video: Text-to-video generation without text-video data,”arXiv preprint arXiv:2209.14792, 2022

    Pith/arXiv arXiv 2022

  4. [4]

    Animatediff: Animate your personalized text-to- image diffusion models without specific tuning,

    Y. Guo, C. Yang, A. Rao, Z. Liang, Y. Wang, Y. Qiao, M. Agrawala, D. Lin, and B. Dai, “Animatediff: Animate your personalized text-to- image diffusion models without specific tuning,” inICLR, 2024

    2024

  5. [5]

    Stable video diffusion: Scaling latent video diffusion models to large datasets,

    A. Blattmann, T. Dockhorn, S. Kulal, D. Mendelevitch, M. Kilian, D. Lorenz, Y. Levi, Z. English, V. Voleti, A. Letts,et al., “Stable video diffusion: Scaling latent video diffusion models to large datasets,”arXiv preprint arXiv:2311.15127, 2023

    Pith/arXiv arXiv 2023

  6. [6]

    Faces that speak: Jointly synthesising talking face and speech from text,

    Y. Jang, J.-H. Kim, J. Ahn, D. Kwak, H.-S. Yang, Y.-C. Ju, I.-H. Kim, B.-Y. Kim, and J. S. Chung, “Faces that speak: Jointly synthesising talking face and speech from text,” inCVPR, 2024

    2024

  7. [7]

    Magicdance: Realistic human dance video gen- eration with motions & facial expressions transfer,

    D. Chang, Y. Shi, Q. Gao, J. Fu, H. Xu, G. Song, Q. Yan, X. Yang, and M. Soleymani, “Magicdance: Realistic human dance video gen- eration with motions & facial expressions transfer,” arXiv preprint arXiv:2311.12052, 2023

    Pith/arXiv arXiv 2023

  8. [8]

    Nerf: Representing scenes as neural radiance fields for view synthesis,

    B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoor- thi, and R. Ng, “Nerf: Representing scenes as neural radiance fields for view synthesis,”Communications of the ACM, vol. 65, no. 1, pp. 99– 106, 2021

    2021

  9. [9]

    3d gaussian splatting for real-time radiance field rendering,

    B. Kerbl, G. Kopanas, T. Leimkühler, and G. Drettakis, “3d gaussian splatting for real-time radiance field rendering,”TOG, vol. 42, no. 4, pp. 1–14, 2023

    2023

  10. [10]

    Deep person generation: A survey from the perspective of face, pose, and cloth synthesis,

    T. Sha, W. Zhang, T. Shen, Z. Li, and T. Mei, “Deep person generation: A survey from the perspective of face, pose, and cloth synthesis,”ACM Comput. Surv., vol. 55, no. 12, pp. 1–37, 2023

    2023

  11. [11]

    A comprehensive survey on human video generation: Challenges, methods, and insights,

    W. Lei, J. Wang, F. Ma, G. Huang, and L. Liu, “A comprehensive survey on human video generation: Challenges, methods, and insights,” arXiv preprint arXiv:2407.08428, 2024

    Pith/arXiv arXiv 2024

  12. [12]

    Image-based virtual try-on: A survey,

    D. Song, X. Zhang, J. Zhou, W. Nie, R. Tong, M. Kankanhalli, and A.-A. Liu, “Image-based virtual try-on: A survey,”IJCV, 2024

    2024

  13. [13]

    A Comprehensive Taxonomy and Analysis of Talking Head Synthesis: Techniques for Portrait Generation, Driving Mechanisms, and Editing

    M. Meng, Y. Zhao, B. Zhang, Y. Zhu, W. Shi, M. Wen, and Z. Fan, “A comprehensive taxonomy and analysis of talking head synthesis: Techniques for portrait generation, driving mechanisms, and editing,” arXiv preprint arXiv:2406.10553, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  14. [14]

    Multilingual video dubbing—a technology review and current challenges,

    D. Bigioi and P. Corcoran, “Multilingual video dubbing—a technology review and current challenges,”Frontiers in Signal Processing, vol. 3, p. 1230755, 2023

    2023

  15. [15]

    Difftalk: Crafting diffusion models for generalized audio-driven portraits anima- tion,

    S. Shen, W. Zhao, Z. Meng, W. Li, Z. Zhu, J. Zhou, and J. Lu, “Difftalk: Crafting diffusion models for generalized audio-driven portraits anima- tion,” inCVPR, 2023

    2023

  16. [16]

    Identity-preserving talking face generation with landmark and appear- ance priors,

    W. Zhong, C. Fang, Y. Cai, P. Wei, G. Zhao, L. Lin, and G. Li, “Identity-preserving talking face generation with landmark and appear- ance priors,” inCVPR, 2023

    2023

  17. [17]

    Af- fective faces for goal-driven dyadic communication,

    S. Geng, R. Teotia, P. Tendulkar, S. Menon, and C. Vondrick, “Af- fective faces for goal-driven dyadic communication,”arXiv preprint arXiv:2301.10939, 2023

    Pith/arXiv arXiv 2023

  18. [18]

    AgentAvatar: Disentangling Planning, Driving and Rendering for Photorealistic Avatar Agents

    D. Wang, B. Dai, Y. Deng, and B. Wang, “Agentavatar: Disentangling planning, driving and rendering for photorealistic avatar agents,”arXiv preprint arXiv:2311.17465, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  19. [19]

    Instructavatar: Text-guided emotion and motion control for avatar generation,

    Y. Wang, J. Guo, J. Bai, R. Yu, T. He, X. Tan, X. Sun, and J. Bian, “Instructavatar: Text-guided emotion and motion control for avatar generation,” inAAAI, 2025

    2025

  20. [20]

    Human motion generation: A survey,

    W. Zhu, X. Ma, D. Ro, H. Ci, J. Zhang, J. Shi, F. Gao, Q. Tian, and Y. Wang, “Human motion generation: A survey,”IEEE TPAMI, 2023

    2023

  21. [21]

    A survey of talking-head generation technology and its applications,

    H. Li, L. Zhang, X. Zou, J. Lin, X. Li, Y. Li, X. Chang, and J. Zheng, “A survey of talking-head generation technology and its applications,” in ICEAAI, 2025

    2025

  22. [22]

    Unsupervised high-resolution portrait gaze correction and animation,

    J. Zhang, J. Chen, H. Tang, E. Sangineto, P. Wu, Y. Yan, N. Sebe, and W. Wang, “Unsupervised high-resolution portrait gaze correction and animation,” IEEE TIP, vol. 31, pp. 5272–5286, 2022

    2022

  23. [23]

    Expression domain translation network for cross-domain head reenactment,

    T. Kang, J. Oh, J. Lee, S. Park, and J. Choo, “Expression domain translation network for cross-domain head reenactment,” inICASSP, 2024

    2024

  24. [24]

    Otavatar: One-shot talking face avatar with controllable tri-plane rendering,

    Z. Ma, X. Zhu, G.-J. Qi, Z. Lei, and L. Zhang, “Otavatar: One-shot talking face avatar with controllable tri-plane rendering,” in CVPR, 2023

    2023

  25. [25]

    Follow-your-emoji: Fine-controllable and expressive freestyle portrait animation,

    Y. Ma, H. Liu, H. Wang, H. Pan, Y. He, J. Yuan, A. Zeng, C. Cai, H.- Y. Shum, W. Liu, and Q. Chen, “Follow-your-emoji: Fine-controllable and expressive freestyle portrait animation,” inSIGGRAPH Asia, 2024

    2024

  26. [26]

    Liveportrait: Efficient portrait animation with stitching and retargeting control,

    J. Guo, D. Zhang, X. Liu, Z. Zhong, Y. Zhang, P. Wan, and D. Zhang, “Liveportrait: Efficient portrait animation with stitching and retargeting control,” arXiv preprint arXiv:2407.03168, 2024

    Pith/arXiv arXiv 2024

  27. [27]

    X-portrait: Expressive portrait animation with hierarchical motion attention,

    Y. Xie, H. Xu, G. Song, C. Wang, Y. Shi, and L. Luo, “X-portrait: Expressive portrait animation with hierarchical motion attention,” in SIGGRAPH, 2024

    2024

  28. [28]

    MobilePortrait: Real-Time One-Shot Neural Head Avatars on Mobile Devices

    J. Jiang, G. Lin, Z. Rong, C. Liang, Y. Zhu, J. Yang, and T. Zhong, “Mobileportrait: Real-time one-shot neural head avatars on mobile devices,” arXiv preprint arXiv:2407.05712, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  29. [29]

    Everybody dance now,

    C. Chan, S. Ginosar, T. Zhou, and A. A. Efros, “Everybody dance now,” inICCV, 2019

    2019

  30. [30]

    Human motionformer: Transferring human motions with vision transformers,

    H. Liu, X. Han, C. Jin, L. Qian, H. Wei, Z. Lin, F. Wang, H. Dong, Y. Song, J. Xu, and Q. Chen, “Human motionformer: Transferring human motions with vision transformers,” inICLR, 2023

    2023

  31. [31]

    Bidirectional temporal diffusion model for temporally consistent human animation,

    T. Adiya, J. S. Yoon, J. Lee, S. Kim, and H. Lim, “Bidirectional temporal diffusion model for temporally consistent human animation,” in ICLR, 2024

    2024

  32. [32]

    Disco: Disentangled control for realistic human dance generation,

    T. Wang, L. Li, K. Lin, Y. Zhai, C.-C. Lin, Z. Yang, H. Zhang, Z. Liu, and L. Wang, “Disco: Disentangled control for realistic human dance generation,” inCVPR, 2024

    2024

  33. [33]

    Animate anyone: Consistent and controllable image-to-video synthesis for character animation,

    L. Hu, “Animate anyone: Consistent and controllable image-to-video synthesis for character animation,” inCVPR, 2024

    2024

  34. [34]

    Follow-your-pose v2: Multiple- condition guided character image animation for stable pose control,

    J. Xue, H. Wang, Q. Tian, Y. Ma, A. Wang, Z. Zhao, S. Min, W. Zhao, K. Zhang, H.-Y. Shum,et al., “Follow-your-pose v2: Multiple- condition guided character image animation for stable pose control,” arXiv preprint arXiv:2406.03035, 2024

    Pith/arXiv arXiv 2024

  35. [35]

    Human4dit: Free-view human video generation with 4d diffusion transformer,

    R. Shao, Y. Pang, Z. Zheng, J. Sun, and Y. Liu, “Human4dit: Free-view human video generation with 4d diffusion transformer,”arXiv preprint arXiv:2405.17405, 2024

    Pith/arXiv arXiv 2024

  36. [36]

    Mimicmotion: High-quality human motion video generation with confidence-aware pose guidance,

    Y. Zhang, J. Gu, L.-W. Wang, H. Wang, J. Cheng, Y. Zhu, and F. Zou, “Mimicmotion: High-quality human motion video generation with confidence-aware pose guidance,” arXiv preprint arXiv:2406.19680, 2024

    Pith/arXiv arXiv 2024

  37. [37]

    I2v-adapter: A general image-to-video adapter for diffusion models,

    X. Guo, M. Zheng, L. Hou, Y. Gao, Y. Deng, P. Wan, D. Zhang, Y. Liu, W. Hu, and Z. Zha, “I2v-adapter: A general image-to-video adapter for diffusion models,” inSIGGRAPH, 2024

    2024

  38. [38]

    Vivid: Video virtual try-on using diffusion models,

    Z. Fang, W. Zhai, A. Su, H. Song, K. Zhu, M. Wang, Y. Chen, Z. Liu, Y. Cao, and Z.-J. Zha, “Vivid: Video virtual try-on using diffusion models,” arXiv preprint arXiv:2405.11794, 2024

    Pith/arXiv arXiv 2024

  39. [39]

    Dreampose: Fashion image-to-video synthesis via stable diffusion,

    J. Karras, A. Holynski, T.-C. Wang, and I. Kemelmacher-Shlizerman, “Dreampose: Fashion image-to-video synthesis via stable diffusion,” in ICCV, 2023

    2023

  40. [40]

    Make-your-anchor: A diffusion-based 2d avatar generation frame- work,

    Z. Huang, F. Tang, Y. Zhang, X. Cun, J. Cao, J. Li, and T.-Y. Lee, “Make-your-anchor: A diffusion-based 2d avatar generation frame- work,” inCVPR, 2024

    2024

  41. [41]

    Write-a-speaker: Text-based emotional and rhythmic talking-head gen- eration,

    L. Li, S. Wang, Z. Zhang, Y. Ding, Y. Zheng, X. Yu, and C. Fan, “Write-a-speaker: Text-based emotional and rhythmic talking-head gen- eration,” inAAAI, 2021

    2021

  42. [42]

    Id-animator: Zero-shot identity-preserving human video generation,

    X. He, Q. Liu, S. Qian, X. Wang, T. Hu, K. Cao, K. Yan, M. Zhou, and J. Zhang, “Id-animator: Zero-shot identity-preserving human video generation,” arXiv preprint arXiv:2404.15275, 2024

    Pith/arXiv arXiv 2024

  43. [43]

    Edit-your-motion: Space-time diffusion decoupling learning for video motion editing,

    Y. Zuo, L. Li, L. Jiao, F. Liu, X. Liu, W. Ma, S. Yang, and Y. Guo, “Edit-your-motion: Space-time diffusion decoupling learning for video motion editing,”arXiv preprint arXiv:2405.04496, 2024. XUE et al.: HUMAN MOTION VIDEO GENERATION: A SURVEY 17

    Pith/arXiv arXiv 2024

  44. [44]

    Follow your pose: Pose-guided text-to-video generation using pose- free videos,

    Y. Ma, Y. He, X. Cun, X. Wang, S. Chen, X. Li, and Q. Chen, “Follow your pose: Pose-guided text-to-video generation using pose- free videos,” inAAAI, 2024

    2024

  45. [45]

    Text2performer: Text-driven human video generation,

    Y. Jiang, S. Yang, T. L. Koh, W. Wu, C. C. Loy, and Z. Liu, “Text2performer: Text-driven human video generation,” inICCV, 2023

    2023

  46. [46]

    Styleheat: One-shot high-resolution editable talking face generation via pre-trained stylegan,

    F. Yin, Y. Zhang, X. Cun, M. Cao, Y. Fan, X. Wang, Q. Bai, B. Wu, J. Wang, and Y. Yang, “Styleheat: One-shot high-resolution editable talking face generation via pre-trained stylegan,” inECCV, 2022

    2022

  47. [47]

    Pose- controllable talking face generation by implicitly modularized audio- visual representation,

    H. Zhou, Y. Sun, W. Wu, C. C. Loy, X. Wang, and Z. Liu, “Pose- controllable talking face generation by implicitly modularized audio- visual representation,” inCVPR, 2021

    2021

  48. [48]

    Edtalk: Efficient disentanglement for emotional talking head synthesis,

    S. Tan, B. Ji, M. Bi, and Y. Pan, “Edtalk: Efficient disentanglement for emotional talking head synthesis,” inECCV, 2024

    2024

  49. [49]

    Echomimic: Lifelike audio-drivenportraitanimationsthrougheditablelandmarkconditions,

    Z. Chen, J. Cao, Z. Chen, Y. Li, and C. Ma, “Echomimic: Lifelike audio-drivenportraitanimationsthrougheditablelandmarkconditions,” arXiv preprint arXiv:2407.08136, 2024

    Pith/arXiv arXiv 2024

  50. [50]

    Emo: Emote portrait alive generating expressive portrait videos with audio2video diffusion model under weak conditions,

    L. Tian, Q. Wang, B. Zhang, and L. Bo, “Emo: Emote portrait alive generating expressive portrait videos with audio2video diffusion model under weak conditions,” inECCV, 2024

    2024

  51. [51]

    Hallo: Hierarchical audio-driven visual synthesis for portrait image animation,

    M. Xu, H. Li, Q. Su, H. Shang, L. Zhang, C. Liu, J. Wang, L. Van Gool, Y. Yao, and S. Zhu, “Hallo: Hierarchical audio-driven visual synthesis for portrait image animation,”arXiv preprint arXiv:2406.08801, 2024

    Pith/arXiv arXiv 2024

  52. [52]

    Emotional conversation: Empowering talking faces with cohesive expression, gaze and pose generation,

    J. Liang and F. Lu, “Emotional conversation: Empowering talking faces with cohesive expression, gaze and pose generation,”arXiv preprint arXiv:2406.07895, 2024

    Pith/arXiv arXiv 2024

  53. [53]

    Makeittalk: Speaker-aware talking-head animation,

    Y. Zhou, X. Han, E. Shechtman, J. Echevarria, E. Kalogerakis, and D. Li, “Makeittalk: Speaker-aware talking-head animation,” TOG, vol. 39, no. 6, pp. 1–15, 2020

    2020

  54. [54]

    Live speech portraits: Real-time photore- alistic talking-head animation,

    Y. Lu, J. Chai, and X. Cao, “Live speech portraits: Real-time photore- alistic talking-head animation,”TOG, vol. 40, no. 6, pp. 1–17, 2021

    2021

  55. [55]

    Vlogger: Multimodal diffusion for embodied avatar synthesis,

    E. Corona, A. Zanfir, E. G. Bazavan, N. Kolotouros, T. Alldieck, and C. Sminchisescu, “Vlogger: Multimodal diffusion for embodied avatar synthesis,” arXiv preprint arXiv:2403.08764, 2024

    Pith/arXiv arXiv 2024

  56. [56]

    Dance Any Beat: Blending Beats with Visuals in Dance Video Generation

    X. Wang, H. Wang, D. Liu, and W. Cai, “Dance any beat: Blend- ing beats with visuals in dance video generation,” arXiv preprint arXiv:2405.09266, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  57. [57]

    Auto-encoding variational bayes,

    D. P. Kingma and M. Welling, “Auto-encoding variational bayes,”arXiv preprint arXiv:1312.6114, 2013

    Pith/arXiv arXiv 2013

  58. [58]

    Geneface: Generalized and high-fidelity audio-driven 3d talking face synthesis,

    Z. Ye, Z. Jiang, Y. Ren, J. Liu, J. He, and Z. Zhao, “Geneface: Generalized and high-fidelity audio-driven 3d talking face synthesis,” in ICLR, 2023

    2023

  59. [59]

    Geneface++: Generalized and stable real-time audio-driven 3d talking face generation,

    Z. Ye, J. He, Z. Jiang, R. Huang, J. Huang, J. Liu, Y. Ren, X. Yin, Z. Ma, and Z. Zhao, “Geneface++: Generalized and stable real-time audio-driven 3d talking face generation,” arXiv preprint arXiv:2305.00787, 2023

    Pith/arXiv arXiv 2023

  60. [60]

    Neural discrete representation learning,

    A. Van Den Oord, O. Vinyals,et al., “Neural discrete representation learning,” inNeurIPS, 2017

    2017

  61. [61]

    Generative adversarial nets,

    I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in NeurIPS, 2014

    2014

  62. [62]

    A style-based generator architecture for generative adversarial networks,

    T. Karras, S. Laine, and T. Aila, “A style-based generator architecture for generative adversarial networks,” inCVPR, 2019

    2019

  63. [63]

    Analyzing and improving the image quality of stylegan,

    T. Karras, S. Laine, M. Aittala, J. Hellsten, J. Lehtinen, and T. Aila, “Analyzing and improving the image quality of stylegan,” inCVPR, 2020

    2020

  64. [64]

    An identity-preserved framework for human motion transfer,

    J. Ma, X. Zhang, and S. Yu, “An identity-preserved framework for human motion transfer,” IEEE Trans. Inf. Forensics Secur., vol. 19, pp. 3495–3509, 2024

    2024

  65. [65]

    Deep unsupervised learning using nonequilibrium thermodynamics,

    J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli, “Deep unsupervised learning using nonequilibrium thermodynamics,” in ICML, 2015

    2015

  66. [66]

    Improved techniques for training score-based generative models,

    Y. Song and S. Ermon, “Improved techniques for training score-based generative models,” inNeurIPS, 2020

    2020

  67. [67]

    Improved denoising diffusion proba- bilistic models,

    A. Q. Nichol and P. Dhariwal, “Improved denoising diffusion proba- bilistic models,” inICML, 2021

    2021

  68. [68]

    Denoising diffusion implicit mod- els,

    J. Song, C. Meng, and S. Ermon, “Denoising diffusion implicit mod- els,” inICLR, 2020

    2020

  69. [69]

    Diffusion models beat gans on image synthesis,

    P. Dhariwal and A. Nichol, “Diffusion models beat gans on image synthesis,” inNeurIPS, 2021

    2021

  70. [70]

    A survey on generative diffusion models,

    H. Cao, C. Tan, Z. Gao, Y. Xu, G. Chen, P.-A. Heng, and S. Z. Li, “A survey on generative diffusion models,”IEEE Trans. Knowl. Data Eng., vol. 36, no. 7, p. 2814–2830, 2024

    2024

  71. [71]

    Denoising diffusion probabilistic models,

    J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” inNeurIPS, 2020

    2020

  72. [72]

    Dance Your Latents: Consistent Dance Generation through Spatial-temporal Subspace Attention Guided by Motion Flow

    H. Fang, Z. Sun, Z. Huang, F. Tang, J. Cao, and S. Tang, “Dance your latents: Consistent dance generation through spatial-temporal subspace attention guided by motion flow,”arXiv preprint arXiv:2310.14780, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  73. [73]

    Human Modelling and Pose Estimation Overview

    P. Knap, “Human modelling and pose estimation overview,” arXiv preprint arXiv:2406.19290, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  74. [74]

    Champ: Controllable and consistent human image animation with 3d parametric guidance,

    S. Zhu, J. L. Chen, Z. Dai, Z. Dong, Y. Xu, X. Cao, Y. Yao, H. Zhu, and S. Zhu, “Champ: Controllable and consistent human image animation with 3d parametric guidance,” inECCV, 2024

    2024

  75. [75]

    Openpose: Realtime multi-person 2d pose estimation using part affinity fields,

    Z. Cao, G. Hidalgo, T. Simon, S.-E. Wei, and Y. Sheikh, “Openpose: Realtime multi-person 2d pose estimation using part affinity fields,” IEEE TPAMI, vol. 43, no. 1, pp. 172–186, 2021

    2021

  76. [76]

    Effective whole-body pose estimation with two-stages distillation,

    Z. Yang, A. Zeng, C. Yuan, and Y. Li, “Effective whole-body pose estimation with two-stages distillation,” inICCVW, 2023

    2023

  77. [77]

    Vividpose: Advancing stable video diffusion for realistic human image animation,

    Q. Wang, Z. Jiang, C. Xu, J. Zhang, Y. Wang, X. Zhang, Y. Cao, W. Cao, C. Wang, and Y. Fu, “Vividpose: Advancing stable video diffusion for realistic human image animation,”arXiv preprint arXiv:2405.18156, 2024

    Pith/arXiv arXiv 2024

  78. [78]

    Magicanimate: Temporally consistent human image animation using diffusion model,

    Z. Xu, J. Zhang, J. H. Liew, H. Yan, J.-W. Liu, C. Zhang, J. Feng, and M. Z. Shou, “Magicanimate: Temporally consistent human image animation using diffusion model,” inCVPR, 2024

    2024

  79. [79]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark,et al., “Learning transferable visual models from natural language supervision,” inICML, 2021

    2021

  80. [80]

    Conformer: Convolution- augmented transformer for speech recognition,

    A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y. Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y. Wu, and R. Pang, “Conformer: Convolution- augmented transformer for speech recognition,” inInterspeech, 2020

    2020

Showing first 80 references.