Pith. sign in

REVIEW 4 major objections 5 minor 49 references

Fleximo: Towards Flexible Text-to-Human Motion Video Generation

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Fleximo generates human motion videos from one photo and text

desk verdict A plausible new-task pipeline whose comparative claim is undermined by a static-reference evaluation and a tiny, unvalidated motion metric. read the letter →

arxiv 2411.19459 v1 pith:K3JR6LI2 submitted 2024-11-29 cs.CV cs.LG

Yuhang Zhang , Yuan Zhou , Zeyu Liu , Yuxuan Cai , Qiuyue Wang , Aidong Men , Huan Yang This is my paper · ORCID
classification cs.CVcs.LG
keywords text-to-humanmotionvideogenerationtext-to-motionmotion-to-videoskeletonadapteranchorpointrescaleLLMplanningBenchScore
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces a new task, text-to-human motion video generation, where the input is a single reference image plus natural language and the output is a video of that person performing the described motion. The authors claim that by combining pretrained text-to-3D motion models with a motion-to-video generator, they can produce high-quality motion videos without needing millions of text-video training pairs. The key moves are projecting text-generated 3D skeletons to 2D, rescaling them to match the reference person via anchor-point alignment, and using a skeleton adapter to fill in missing hand and face details. The paper also contributes MotionBench, a 400-video benchmark, and MotionScore, a metric for motion-text alignment, and reports that Fleximo outperforms existing text-conditioned image-to-video methods.

What carries the argument

The load-bearing mechanism is the anchor-point based rescale combined with the skeleton adapter. The rescale method chooses the neck as a fixed anchor and then rescales each connected body part (arms, legs, hands) individually to match the reference image's limb lengths, after a global affine alignment; this is what makes the text-generated skeleton match the identity in the reference image. The skeleton adapter is a latent video diffusion model that takes a handless skeleton video and a reference skeleton image and generates a complete skeleton video with realistic hand keypoints, bridging the format gap between T2M-GPT's 18 body points and the full-body keypoints required by motion-to-video models.

What would settle it

Run Fleximo on a motion with large positional shift (e.g., "A person is playing basketball" with side-facing movement) and measure frame-to-frame variation of the per-bone scaling factors and face identity similarity against the reference image. If the scaling variance is high and identity similarity drops sharply relative to smaller motions, the anchor-point rescale assumption is violated and the central claim of consistent high-quality generation fails.

Watch

Extended reading notes

Core claim

The central claim is that a reference image plus natural language is sufficient to generate realistic human motion videos, and that this can be achieved by bridging two existing model families: text-to-motion and motion-to-video. Concretely, Fleximo projects skeletons from a pretrained text-to-3D motion model into 2D, rescales them frame-by-frame to the reference image using an affine transform followed by per-bone anchoring at the neck, enriches the sparse skeleton with a skeleton adapter that synthesizes hand and face keypoints, and refines the final video by re-extracting DWPose skeletons from a first-pass anchor video. With LLM-based planning, long text is decomposed into discrete motion segments that can be concatenated into videos of any desired length. The paper reports that this pipeline outperforms text-conditioned image-to-video baselines on identity fidelity, video quality, and motion following.

Load-bearing premise

The pipeline assumes that a text-generated skeleton can be aligned to the reference person by an affine transform followed by per-bone rescaling, and that this alignment stays consistent for all frames of a motion; when a motion involves large positional shifts, the scaling becomes inconsistent and identity is lost.

Editorial extensions

If this is right

  • Users can animate any reference photo with natural language, without sourcing pose videos, making motion control as easy as typing a sentence.
  • Because the pipeline reuses large pretrained models and only needs a modest training set for the adapter, text-to-human motion video becomes feasible without millions of text-video pairs.
  • Long and compound instructions can be generated by the LLM planner, enabling multi-motion videos that concatenate seamlessly.
  • The MotionScore metric offers a way to judge motion following that avoids the bias of binary or forced-choice questioning.
  • The refinement loop improves identity consistency by reusing DWPose skeletons from a first-generated anchor video, suggesting a general self-improvement strategy for skeleton-guided animation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If scale alignment is the main failure mode, a depth-aware or learned skeleton-rescale step could extend Fleximo to motions with large translation and out-of-plane rotation, such as running toward the camera.
  • The LLM planning template is not motion-specific and could be applied to other long-horizon generation tasks where a text instruction must be segmented into executable sub-steps.
  • MotionScore could be repurposed as a general motion-following evaluation for any generated human video, though its reliability depends on the captioning LLM's ability to describe motion accurately.
  • The skeleton adapter's training scheme—animate a full skeleton from a handless one—is a general way to inject fine-grained missing details and could be applied to facial expressions, props, or object interactions in other animation pipelines.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces a new task, text-to-human motion video generation, in which a single reference image and a natural-language description are used to synthesize a video of the reference identity performing the described motion. To avoid training on large text-video datasets, Fleximo first generates 3D human motion with a pretrained text-to-motion model (T2M-GPT), projects the resulting meshes to 2D skeletons, and aligns those skeletons to the reference image using an affine transformation followed by a neck-anchored per-bone rescaling (Section 3.2). A skeleton adapter trained on HumanVid fills in missing hand and face keypoints (Section 3.3), and the resulting skeleton video is fed to a MimicMotion-style motion-to-video model together with the reference image; an optional refinement pass and LLM-based decomposition into motion segments support longer or multi-part motions. The paper also introduces MotionBench, a benchmark of 400 videos over 20 identities and 20 motions, and MotionScore, an LLM-plus-CLIP metric for motion-text alignment. Quantitative comparisons against I2VGen-XL, VideoCrafter, and DynamiCrafter, a small user study, ablations, and failure cases are reported.

Significance. The proposed task is well motivated and the modular pipeline is a sensible way to sidestep the lack of large text-motion-video paired data: reusing T2M-GPT and a pretrained motion-to-video model is practical, and the skeleton adapter addresses a real gap between 18-point text-to-motion skeletons and the full-body keypoints expected by motion-to-video models. If the comparative claims were properly supported, Fleximo would be a useful baseline and the MotionBench/MotionScore package would be a community resource. However, the current quantitative evidence does not establish the central claim of outperforming existing methods: the main image-similarity metrics are computed against the static reference image, which rewards stillness, MotionScore is an author-proposed metric with no validation and a very small margin, and the user study is too small and lacks statistical analysis. The paper's own failure case for large positional shifts further limits the scope of the claimed flexibility. These issues are fixable with additional experiments, which is why I am not recommending rejection.

major comments (4)
  1. [Section 5.2, Table 1] The PSNR, SSIM, LPIPS, and DreamSim numbers in Table 1 are computed by comparing each generated frame to the single reference image, as stated in Section 5.2. For a motion-generation task this is not a valid notion of ground truth: a video that simply freezes on the reference image would score perfectly, and any real motion necessarily reduces pixel-level similarity. The large reported gap (PSNR 16.647 for Fleximo vs. 9.607 for DynamiCrafter) is therefore consistent with Fleximo staying closer to the reference and cannot support the claim that Fleximo generates better motion videos. These metrics should be replaced or supplemented with motion-aware evaluations, for example comparing extracted pose sequences against the text-to-motion skeletons or against ground-truth motion from reference videos, and per-video error bars or confidence intervals should be reported.
  2. [Section 4, MotionScore] MotionScore is an author-proposed metric whose validity is not demonstrated. The paper reports that MotionLLM is prompted to describe the motion in the generated video and that CLIP cosine similarity between that description and the input text is computed, but it does not show that MotionScore correlates with human judgments, does not report per-motion or per-video distributions, and does not provide any statistical test. The reported advantage over DynamiCrafter is 0.6990 versus 0.6868, a difference of 0.0122 that is likely within the noise of an LLM-plus-CLIP pipeline. The authors should validate MotionScore against human ratings, show that it detects failures such as the basketball case in Figure 9, and report confidence intervals and significance tests.
  3. [Section 5.4, Table 2 and Supplementary Section 9] The user study does not support the strength of the claims drawn from it. There are only 8 participants, and with 800 videos and each participant scoring 100 videos, each video appears to receive a single rating; no inter-rater agreement, significance test, or confidence interval is reported. The 2.3-point gap in motion alignment between Fleximo and DynamiCrafter therefore cannot be separated from rating noise. The study should be expanded or, at minimum, the authors should report per-item statistics, a repeated-measures analysis, and inter-rater agreement.
  4. [Section 5.6, Figure 9] The acknowledged failure case is load-bearing for the central claim. The paper states that for motions with large positional shifts, such as playing basketball, the skeleton scaling becomes inconsistent, causing scale jitter and poor identity preservation. This is precisely the kind of dynamic motion that the task is meant to support, and several motions in MotionBench (running, jumping, boxing, playing golf) are likely to trigger the same behavior. The metrics in Table 1 would not detect such failures because they reward proximity to the reference image. The authors should quantify how often the pipeline degrades on each motion type, for example by reporting per-motion MotionScore or per-motion human ratings, and should qualify the claims of 'flexible' and 'any desired length' accordingly.
minor comments (5)
  1. [Section 3.2, Eq. (3)] The notation yd and yr is described as 'y-coordinates of detected bodies and the reference body,' but it is not clear how many keypoints are used in the polyfit or whether both arms and legs are included; please specify the exact set of keypoints used for the affine fit.
  2. [Section 3.3 vs. Figure 2] The paper uses both 'skeleton adapter' and 'skeleton adaptor'; please unify the terminology.
  3. [Section 5.5] The ablation study is presented as Figure 6 in the main text but the text refers to it as 'Fig. 14'; please correct the cross-reference.
  4. [References] References [7] and [8] are the same paper (Guo et al., 'Generating diverse and natural 3d human motions from text') and should be merged.
  5. [Supplementary Section 9] The sentence 'The results in Tab. 2 are averaged on 200 videos' is imprecise; it should read 'averaged over the 200 videos generated by each method,' and the fact that each video receives a single rating should be stated explicitly.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation found; the pipeline composes pretrained external models and trained adapters, and the debatable evaluation choices are validity concerns rather than circularity.

full rationale

Fleximo's derivation chain is an empirical pipeline: LLM planning, T2M-GPT 3D motion generation, orthographic projection, affine/anchor-point skeleton alignment, a trained skeleton adapter, motion-to-video generation, and optional refinement. No equation defines a predicted output as a fitted input. Equations (3)-(6) fit affine parameters to reference-image keypoints, but this is a conditioning/registration step, not a prediction later reported as evidence. The skeleton adapter is trained on HumanVid and validated against external baselines; the refinement uses DWPose, an external pose estimator. Self-citations to MimicMotion and Allegro are not load-bearing for the central novelty, because the anchor rescale and skeleton adapter are evaluated comparatively against I2VGen-XL, VideoCrafter, and DynamiCrafter. MotionScore is an unvalidated evaluation design and the reference-image similarity metrics in Table 1 may be poor measures for a motion-generation task, but these are evaluation-validity concerns, not circularity: they do not make Fleximo's outputs resemble its inputs by construction. No fitted parameter is renamed as a prediction, and no uniqueness claim is imported from the authors' prior work. Hence no significant circularity.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The pipeline relies on pretrained text-to-motion (T2M-GPT) and motion-to-video (MimicMotion) models, plus a fine-tuned skeleton adapter. Its per-frame affine and per-bone rescaling parameters are computed from the input reference image and are not global fitted constants. The main unproven support is the generalization of these modules and the validity of the newly proposed MotionScore metric.

free parameters (2)
  • per-frame affine scale and offset (ax, ay, bx, by) = computed per frame via linear regression (polyfit) between T2M-GPT skeleton and reference keypoints
    Equations 3-6: these parameters align the text-generated skeleton to the reference image; they are fit on the fly, not learned globally, and directly determine the movement scale.
  • per-bone length rescaling factors (neck, arms, legs, hands) = computed to match reference skeleton bone lengths per frame
    Section 3.2: after affine alignment, each body part length is rescaled to the reference image's bone length; this choice affects motion quality and is cited as the source of scale jitter for large movements.
assumptions (4)
  • domain assumption A single affine transformation plus per-bone rescaling can align the projected T2M-GPT skeleton to the reference image for any pose
    Section 3.2; the paper admits in Sec. 5.6 that large positional shifts break this alignment, causing scale jitter and identity loss.
  • domain assumption T2M-GPT 3D meshes projected to 2D provide motion guidance that MimicMotion-style M2V models can follow
    Section 3.1-3.3; the whole pipeline depends on the pretrained text-to-motion model's output being a usable motion prior; the paper only fine-tunes the video side.
  • domain assumption MotionLLM's video description, embedded by CLIP, is a valid proxy for human-judged motion-text alignment
    Section 4; MotionScore is proposed without correlation analysis against human labels; the user study in Sec. 5.4 is separate.
  • domain assumption The skeleton adapter trained on HumanVid handless/with-hand skeleton pairs generalizes to out-of-domain text-generated skeletons and identities
    Section 3.3 and 5.1; training distribution is HumanVid; test identities are online/Kling images, some out-of-domain.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Fleximo: Towards Flexible Text-to-Human Motion Video Generation." pith.science (2026). https://pith.science/paper/K3JR6LI2

@misc{pith2026241119459,
  author       = {Pith},
  title        = {Pith review of: Fleximo: Towards Flexible Text-to-Human Motion Video Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/K3JR6LI2}},
  note         = {Machine review of arXiv:2411.19459}
}
read the original abstract

Current methods for generating human motion videos rely on extracting pose sequences from reference videos, which restricts flexibility and control. Additionally, due to the limitations of pose detection techniques, the extracted pose sequences can sometimes be inaccurate, leading to low-quality video outputs. We introduce a novel task aimed at generating human motion videos solely from reference images and natural language. This approach offers greater flexibility and ease of use, as text is more accessible than the desired guidance videos. However, training an end-to-end model for this task requires millions of high-quality text and human motion video pairs, which are challenging to obtain. To address this, we propose a new framework called Fleximo, which leverages large-scale pre-trained text-to-3D motion models. This approach is not straightforward, as the text-generated skeletons may not consistently match the scale of the reference image and may lack detailed information. To overcome these challenges, we introduce an anchor point based rescale method and design a skeleton adapter to fill in missing details and bridge the gap between text-to-motion and motion-to-video generation. We also propose a video refinement process to further enhance video quality. A large language model (LLM) is employed to decompose natural language into discrete motion sequences, enabling the generation of motion videos of any desired length. To assess the performance of Fleximo, we introduce a new benchmark called MotionBench, which includes 400 videos across 20 identities and 20 motions. We also propose a new metric, MotionScore, to evaluate the accuracy of motion following. Both qualitative and quantitative results demonstrate that our method outperforms existing text-conditioned image-to-video generation methods. All code and model weights will be made publicly available.

Figures

Figures reproduced from arXiv: 2411.19459 by the authors.

Figure 1
Figure 1. Given a reference image and a motion text, our method [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The framework of Fleximo. We use LLM to plan long motion texts. The text-to-motion module generates 3D mesh vertices [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The structure of our proposed skeleton adapter. The [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Qualitative results of Fleximo (first three rows) compared to the SOTA text-conditioned image-to-video generation method, [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 6
Figure 6. Figure 6: The ablation study of Fleximo. Reference image is [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: The generated video could only respond to a small mo [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Generated videos with and without refinement. With [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: Failure case of Fleximo. When the motion involves large [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 10
Figure 10. Figure 10: The identities in our MotionBench. The last 5 identities are generated by Kling AI, and all other identities are sourced online. [PITH_FULL_IMAGE:figures/full_fig_p011_10.png]
Figure 11
Figure 11. Figure 11: Different motions generated by Fleximo. We show a same identity performing 10 different motions to illustrate the motion [PITH_FULL_IMAGE:figures/full_fig_p013_11.png]
Figure 12
Figure 12. Figure 12: Qualitative comparison with other methods. The reference image is shown on the left. We use various methods to generate [PITH_FULL_IMAGE:figures/full_fig_p014_12.png]
Figure 13
Figure 13. Figure 13: Qualitative comparison with other methods. The reference image is shown on the left. We use various methods to generate [PITH_FULL_IMAGE:figures/full_fig_p015_13.png]
Figure 14
Figure 14. Figure 14: More results of the ablation study. The reference image is presented in the left, and the motion texts are ”a woman is running” [PITH_FULL_IMAGE:figures/full_fig_p016_14.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

49 extracted references · 20 canonical work pages

  1. [1]

    Stable video diffusion: Scaling latent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023. 4

    arXiv 2023

  2. [2]

    Magicpose: Realistic human poses and facial expressions retargeting with identity-aware diffusion

    Di Chang, Yichun Shi, Quankai Gao, Hongyi Xu, Jessica Fu, Guoxian Song, Qing Yan, Yizhe Zhu, Xiao Yang, and Mo- hammad Soleymani. Magicpose: Realistic human poses and facial expressions retargeting with identity-aware diffusion. In Forty-first International Conference on Machine Learn- ing, 2023. 1, 3

    work page 2023

  3. [3]

    Videocrafter1: Open diffusion models for high-quality video generation, 2023

    Haoxin Chen, Menghan Xia, Yingqing He, Yong Zhang, Xiaodong Cun, Shaoshu Yang, Jinbo Xing, Yaofang Liu, Qifeng Chen, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter1: Open diffusion models for high-quality video generation, 2023. 2, 5, 6, 1

    work page 2023

  4. [4]

    Motionllm: Understanding human behaviors from human motions and videos

    Ling-Hao Chen, Shunlin Lu, Ailing Zeng, Hao Zhang, Benyou Wang, Ruimao Zhang, and Lei Zhang. Motionllm: Understanding human behaviors from human motions and videos. arXiv preprint arXiv:2405.20340, 2024. 5

    arXiv 2024

  5. [5]

    Executing your commands via motion diffusion in latent space

    Xin Chen, Biao Jiang, Wen Liu, Zilong Huang, Bin Fu, Tao Chen, and Gang Yu. Executing your commands via motion diffusion in latent space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 18000–18010, 2023. 2

    2023

  6. [6]

    Dream- sim: Learning new dimensions of human visual similar- ity using synthetic data

    Stephanie Fu, Netanel Tamir, Shobhita Sundaram, Lucy Chai, Richard Zhang, Tali Dekel, and Phillip Isola. Dream- sim: Learning new dimensions of human visual similar- ity using synthetic data. arXiv preprint arXiv:2306.09344 ,

    arXiv

  7. [7]

    Generating diverse and natural 3d human motions from text

    Chuan Guo, Shihao Zou, Xinxin Zuo, Sen Wang, Wei Ji, Xingyu Li, and Li Cheng. Generating diverse and natural 3d human motions from text. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022. 2

    work page 2022

  8. [8]

    Generating diverse and natural 3d human motions from text

    Chuan Guo, Shihao Zou, Xinxin Zuo, Sen Wang, Wei Ji, Xingyu Li, and Li Cheng. Generating diverse and natural 3d human motions from text. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5152–5161, 2022. 2

    work page 2022

Show all 49 references
  1. [9]

    Gans trained by a two time-scale update rule converge to a local nash equilib- rium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium. In Advances in Neural Information Processing Sys- tems, 2017. 6

    2017

  2. [10]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. Advances in Neural Information Processing Systems, 33:6840–6851, 2020. 2

    2020

  3. [11]

    Image quality metrics: Psnr vs

    Alain Hore and Djemel Ziou. Image quality metrics: Psnr vs. ssim. In International Conference on Pattern Recognition , pages 2366–2369. IEEE, 2010. 6

    2010

  4. [12]

    Animate anyone: Consistent and controllable image- to-video synthesis for character animation

    Li Hu. Animate anyone: Consistent and controllable image- to-video synthesis for character animation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8153–8163, 2024. 1, 3

    2024

  5. [13]

    VBench: Com- prehensive benchmark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. VBench: Com- prehensive benchmark suite for video generative models....

    2024

  6. [14]

    Learning high fi- delity depths of dressed humans by watching social media dance videos

    Yasamin Jafarian and Hyun Soo Park. Learning high fi- delity depths of dressed humans by watching social media dance videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12753– 12762, 2021. 5

    2021

  7. [15]

    Digital image processing

    Bernd J ¨ahne. Digital image processing. Springer Science & Business Media, 2005. 3

    2005

  8. [16]

    Motiongpt: Human motion as a foreign lan- guage

    Biao Jiang, Xin Chen, Wen Liu, Jingyi Yu, Gang Yu, and Tao Chen. Motiongpt: Human motion as a foreign lan- guage. Advances in Neural Information Processing Systems, 36:20067–20079, 2023. 2

    2023

  9. [17]

    Auto-encoding variational bayes

    Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013. 4

    2013 arXiv

  10. [18]

    Motionrl: Align text-to-motion generation to human preferences with multi-reward reinforcement learning

    Xiaoyang Liu, Yunyao Mao, Wengang Zhou, and Houqiang Li. Motionrl: Align text-to-motion generation to human preferences with multi-reward reinforcement learning. arXiv preprint arXiv:2410.06513, 2024. 2

    2024 arXiv

  11. [19]

    Sora: A review on background, technology, limitations, and opportunities of large vision models

    Yixin Liu, Kai Zhang, Yuan Li, Zhiling Yan, Chujie Gao, Ruoxi Chen, Zhengqing Yuan, Yue Huang, Hanchi Sun, Jian- feng Gao, et al. Sora: A review on background, technology, limitations, and opportunities of large vision models. arXiv preprint arXiv:2402.17177, 2024. 2

    2024 arXiv

  12. [20]

    Temos: Generating diverse human motions from textual descriptions

    Mathis Petrovich, Michael J Black, and G ¨ul Varol. Temos: Generating diverse human motions from textual descriptions. In European Conference on Computer Vision , pages 480–

  13. [21]

    Movie gen: A cast of media foundation models

    Adam Polyak, Amit Zohar, Andrew Brown, Andros Tjandra, Animesh Sinha, Ann Lee, Apoorv Vyas, Bowen Shi, Chih- Yao Ma, Ching-Yao Chuang, et al. Movie gen: A cast of media foundation models. arXiv preprint arXiv:2410.13720,

    arXiv

  14. [22]

    Improving language understanding by gener- ative pre-training

    Alec Radford. Improving language understanding by gener- ative pre-training. 2018. 2

    2018

  15. [23]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. In Forty-first International Conference on Mach...

    2021

  16. [24]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 2, 4

    2022

  17. [25]

    U- net: Convolutional networks for biomedical image segmen- tation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U- net: Convolutional networks for biomedical image segmen- tation. In International Conference on Medical Image Com- puting and Computer-Assisted Intervention, pages 234–241. Springer, 2015. 4 9

    2015

  18. [26]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020. 2

    2010 arXiv

  19. [27]

    Motionclip: Exposing human motion generation to clip space

    Guy Tevet, Brian Gordon, Amir Hertz, Amit H Bermano, and Daniel Cohen-Or. Motionclip: Exposing human motion generation to clip space. In European Conference on Com- puter Vision, pages 358–374. Springer, 2022. 2

    2022

  20. [28]

    Human motion diffu- sion model

    Guy Tevet, Sigal Raab, Brian Gordon, Yoni Shafir, Daniel Cohen-or, and Amit Haim Bermano. Human motion diffu- sion model. In The Eleventh International Conference on Learning Representations, 2023. 2

    2023

  21. [29]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth´ee Lacroix, Baptiste Rozi`ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. 2, 3

    2023 arXiv

  22. [30]

    Towards accurate generative models of video: A new metric & chal- lenges

    Jonas Unterthiner, Bernhard Nessler, Georg Heigold, Tejas Kulkarni, Elena Buchatskaya, and Roland Hafner. Towards accurate generative models of video: A new metric & chal- lenges. In arXiv preprint arXiv:1812.01717, 2018. 6

    2018 arXiv

  23. [31]

    Neural discrete representation learning

    Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in Neural Information Processing Systems, 30, 2017. 2

    2017

  24. [32]

    Disco: Disentangled control for realistic human dance generation

    Tan Wang, Linjie Li, Kevin Lin, Yuanhao Zhai, Chung- Ching Lin, Zhengyuan Yang, Hanwang Zhang, Zicheng Liu, and Lijuan Wang. Disco: Disentangled control for realistic human dance generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition ,...

    2024

  25. [33]

    Motiongpt-2: A general-purpose motion- language model for motion generation and understanding

    Yuan Wang, Di Huang, Yaqi Zhang, Wanli Ouyang, Jile Jiao, Xuetao Feng, Yan Zhou, Pengfei Wan, Shixiang Tang, and Dan Xu. Motiongpt-2: A general-purpose motion- language model for motion generation and understanding. arXiv preprint arXiv:2410.21747, 2024. 2

    arXiv

  26. [34]

    Humanvid: Demystifying training data for camera-controllable human image animation

    Zhenzhi Wang, Yixuan Li, Yanhong Zeng, Youqing Fang, Yuwei Guo, Wenran Liu, Jing Tan, Kai Chen, Tianfan Xue, Bo Dai, et al. Humanvid: Demystifying training data for camera-controllable human image animation. arXiv preprint arXiv:2407.17438, 2024. 5

    2024 arXiv

  27. [35]

    Dynamicrafter: Animating open-domain images with video diffusion priors

    Jinbo Xing, Menghan Xia, Yong Zhang, Haoxin Chen, Xin- tao Wang, Tien-Tsin Wong, and Ying Shan. Dynamicrafter: Animating open-domain images with video diffusion priors

  28. [36]

    Magicanimate: Temporally consistent human im- age animation using diffusion model

    Zhongcong Xu, Jianfeng Zhang, Jun Hao Liew, Hanshu Yan, Jia-Wei Liu, Chenxu Zhang, Jiashi Feng, and Mike Zheng Shou. Magicanimate: Temporally consistent human im- age animation using diffusion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Reco...

    2024

  29. [37]

    Mastering text-to-image dif- fusion: Recaptioning, planning, and generating with multi- modal llms

    Ling Yang, Zhaochen Yu, Chenlin Meng, Minkai Xu, Ste- fano Ermon, and CUI Bin. Mastering text-to-image dif- fusion: Recaptioning, planning, and generating with multi- modal llms. In Forty-first International Conference on Ma- chine Learning, 2024. 5

    2024

  30. [38]

    Effec- tive whole-body pose estimation with two-stages distillation

    Zhendong Yang, Ailing Zeng, Chun Yuan, and Yu Li. Effec- tive whole-body pose estimation with two-stages distillation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4210–4220, 2023. 2, 5

    2023

  31. [39]

    Generating human motion from textual descrip- tions with discrete representations

    Jianrong Zhang, Yangsong Zhang, Xiaodong Cun, Yong Zhang, Hongwei Zhao, Hongtao Lu, Xi Shen, and Ying Shan. Generating human motion from textual descrip- tions with discrete representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, ...

    2023

  32. [40]

    Motiondif- fuse: Text-driven human motion generation with diffusion model

    Mingyuan Zhang, Zhongang Cai, Liang Pan, Fangzhou Hong, Xinying Guo, Lei Yang, and Ziwei Liu. Motiondif- fuse: Text-driven human motion generation with diffusion model. arXiv preprint arXiv:2208.15001, 2022. 2

    2022 arXiv

  33. [41]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni- tion, 2018. 6

    2018

  34. [42]

    I2vgen-xl: High-quality image-to-video synthesis via cascaded diffusion models

    Shiwei Zhang, Jiayu Wang, Yingya Zhang, Kang Zhao, Hangjie Yuan, Zhiwu Qin, Xiang Wang, Deli Zhao, and Jingren Zhou. I2vgen-xl: High-quality image-to-video synthesis via cascaded diffusion models. arXiv preprint arXiv:2311.04145, 2023. 2, 5, 6, 1

    2023 arXiv

  35. [43]

    Mim- icmotion: High-quality human motion video generation with confidence-aware pose guidance

    Yuang Zhang, Jiaxi Gu, Li-Wen Wang, Han Wang, Junqi Cheng, Yuefeng Zhu, and Fangyuan Zou. Mim- icmotion: High-quality human motion video generation with confidence-aware pose guidance. arXiv preprint arXiv:2406.19680, 2024. 1, 2, 3

    2024 arXiv

  36. [44]

    Al- legro: Open the black box of commercial-level video gener- ation model

    Yuan Zhou, Qiuyue Wang, Yuxuan Cai, and Huan Yang. Al- legro: Open the black box of commercial-level video gener- ation model. arXiv preprint arXiv:2410.15458, 2024. 2

    2024 arXiv

  37. [45]

    Champ: Controllable and consistent human image animation with 3d parametric guidance

    Shenhao Zhu, Junming Leo Chen, Zuozhuo Dai, Qingkun Su, Yinghui Xu, Xun Cao, Yao Yao, Hao Zhu, and Siyu Zhu. Champ: Controllable and consistent human image animation with 3d parametric guidance. arXiv preprint arXiv:2403.14781, 2024. 1, 3 10 Fleximo: Towards Flexible Text-to-H...

    2024 arXiv

  38. [46]

    MotionBench We present 20 distinct identities in MotionBench, as shown in Fig. 10. These identities represent a diverse spectrum of individuals, encompassing a range of genders, ages, and ethnic backgrounds, ensuring that our dataset is compre- hensive and inclusive. The ident...

  39. [47]

    Results of Fleximo and Other Methods We provide more results of Fleximo and other methods in Fig. 11, Fig. 12 and Fig. 13. We also provide the orig- inal videos in the zip file of the supplementary material. We show the same person performing 10 different mo- tions in Fig. 11,...

  40. [48]

    Eight participants evaluate each video across three criteria: video quality, identity preservation, and motion-text alignment

    Details of User Study We generate 800 videos using four different video gener- ation methods: I2VGen-XL, VideoCrafter, DynamiCrafter, and Fleximo. Eight participants evaluate each video across three criteria: video quality, identity preservation, and motion-text alignment. Eac...

  41. [49]

    More Results of Ablation Study We provide more results of ablation study in Fig. 14. As illustrated in Fig. 14 (a), directly combining the motion- guided video generation method MimicMotion [43] with the text-to-motion approach T2M-GPT [39] does not yield realistic human motio...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.