REVIEW 4 major objections 5 minor 8 cited by
DreamActor-H1: High-Fidelity Human-Product Demonstration Video Generation via Motion-designed Diffusion Transformers
T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read DreamActor-H1 claims it can generate a realistic human-product demonstration video from a single human photo and a single product photo, preserving both identities.
desk verdict A useful task formulation and an honest architecture, but the comparative evaluation is too weak to support the claimed edge over SOTA, and the motion template assumption is under-tested. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the masked object attention: a VAE-encoded product latent, held fixed rather than self-updating, is concatenated with video tokens, processed by self-attention, scaled by a product-region mask, and added back as a residual, which prevents the network from washing out fine product detail as the diffusion transformer deepens. Around it sit two supporting mechanisms: reference attention for human-product appearance injection, and motion guidance that concatenates a colored 3D body mesh template (SMPL-X) plus the product bounding box with the input noise, with inference-time template matching that picks and resizes motions from a pool covering products of roughly 1 to 40 centimeters.
What would settle it
Take a product with a non-standard shape, such as an umbrella or a transparent bottle, and a person image with no table in front of them, run the inference pipeline, and check whether the hand and product bounding boxes overlap and move coherently across frames: if the vision-language model misestimates size or the matched template cannot fit the scene, the generated frames will show the hand passing through the product or the product floating, contradicting the motion-realism claim.
Extended reading notes
Core claim
The central claim is that DreamActor-H1 simultaneously preserves human identity and fine-grained product identity while producing natural demonstration motions, something the authors argue existing pose-guided, human-object-interaction, and multi-subject customization methods cannot do from the same input format. The method encodes the human reference and product reference through a variational autoencoder; reference attention injects appearance, and an additional masked object attention injects the product latent as a residual update restricted to product-containing regions, so logos and textures survive the diffusion process. Motion comes from a 3D body mesh template and a product bounding box, with an automatic template-matching stage that selects and scales a predefined motion according to product size, body orientation, and aspect ratio. Structured dictionary-style text generated by a vision-language model adds category-level semantics. In quantitative comparisons and a user study, the full model reports the best CLIP-I, DINO-I, FaceSim-Arc, and win rates against the compared baselines.
Load-bearing premise
The load-bearing assumption is that a predefined pool of motion templates plus a rule-based match using product size, body orientation, and aspect ratio yields natural hand-product interactions for any given input pair; when that match fails, the video's gestures will not align with the product.
Editorial extensions
If this is right
- An e-commerce seller could generate a personalized 65-frame, 720 by 1280 demonstration video from two images, with no multi-view product capture or depth maps.
- Product details that normally break in video generation, such as logos, text, and textures on packaging, are claimed to survive because the masked object attention injects a fixed product latent rather than relying only on text prompts.
- The automatic motion-template selection means the same framework can adapt to products of different sizes and aspect ratios within the 1 to 40 centimeter range without requiring the user to supply a pose sequence.
- The structured text encoding is asserted to improve 3D consistency during small product rotations, so the product does not morph when the demonstrator turns it in hand.
Reading between the lines
- The authors' own limitation note says the template pool can produce unnatural interactions when the input scene does not match the template, such as a table-pickup template requiring a table in front of the person; this suggests replacing rule-based template matching with a learned contact-aware motion retargeting model as the natural next step.
- The masked object attention idea could transfer beyond products to any two-subject video generation task, such as a person interacting with a specific pet, instrument, or piece of furniture, whenever one subject's fine detail must be preserved.
- A testable extension is to measure identity preservation under exactly the failure mode the authors name: fast product rotation, where VAE temporal compression can make text and textures flicker.
- Because product-size estimation is delegated to a vision-language model, non-standard shapes are a known failure point; replacing that estimate with a direct metric estimate from the image or a user-supplied physical size would provide a cheap ablation.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. DreamActor-H1 proposes a Diffusion Transformer framework that takes a human reference image and a product reference image and generates a demonstration video. Appearance is injected through reference attention plus a masked object attention that adds fixed VAE-encoded product latents as residual updates; motion is guided by SMPL-X body templates and product bounding boxes, with a pre-defined template pool and rule-based template matching at inference; VLM-generated structured text captions supply category-level semantics. The model is initialized from Seaweed-7B and trained on a hybrid dataset of roughly 15,000 collected demonstration videos plus supplementary live-stream and Seaweed data. The paper reports quantitative and user-study comparisons with AnchorCrafter, Phantom, VACE, and UniAnimate-DiT, together with ablations of the object-attention and text modules.
Significance. If the claims held, this would be a practically useful contribution for e-commerce demonstration generation, because it addresses joint human/product identity preservation and motion guidance from only two input images. The architectural ideas, especially the masked object cross-attention and the template-plus-bounding-box motion guidance, are clearly motivated, and the authors are candid about limitations in Section 5. The paper is an empirical systems paper rather than a derivation paper, so the main burden is experimental evidence. That evidence is currently too weak to establish the central claims of outperforming state-of-the-art methods and generating realistic demonstration motions; the framework is plausible, but the evaluation is not yet conclusive.
major comments (4)
- [§4.2, Tables 1 and 2] The comparison with state-of-the-art methods is not sufficiently controlled to support the abstract's claim of outperforming SOTA. AnchorCrafter is evaluated on only 3 videos, as the authors state; Phantom and VACE run at different resolutions and on different base models; and UniAnimate-DiT is given the authors' first frames and pose sequences as inputs, so it is not a fair comparison for motion realism or end-to-end demonstration generation. All metrics in Table 1 are reported as single means without error bars, confidence intervals, or significance tests, and the user study in Table 2 reports only 'good' or 'same' rates with unequal sample sizes across methods. These issues are load-bearing for the headline claim and should be addressed with matched evaluation protocols, more baselines under identical input conditions, and statistical reporting.
- [§3.3 and §5] The motion realism claim rests on the inference-time template pool and rule-based matching algorithm described in §3.3, but the paper provides no evaluation of template-match success, VLM size-error rates, or coverage of arbitrary input pairs. The authors explicitly state in Section 5 that the system 'may produce unnatural interactions due to relying on pre-defined motions unrelated to specific products' and that the VLM 'may inaccurately judge product sizes, especially for non-standard shapes.' Because the test set is not described as including such difficult cases, the central motion contribution is validated only under favorable conditions. The paper should report failure rates of template matching on a diverse test set, break down results by product shape/size, and analyze cases where the VLM size prediction is incorrect.
- [§4.2] There is no direct metric for interaction correctness, despite 'realistic demonstration motions' and 'intuitive alignment of hand gestures with product placements' being central claims. The only motion-related metric is VBench 'motion smoothness,' which is near-ceiling for all methods and does not measure hand-object contact, spatial alignment, occlusion, or physical plausibility. The user study asks about human identity, product identity, and overall video quality, but does not isolate motion realism or interaction correctness. A dedicated human evaluation or automated hand-object alignment metric on a common set of inputs is needed to substantiate the motion contribution.
- [§4.3, Table 1] The ablation study reports single-run mean scores without variance or significance testing, so the improvements of 'Ours' over 'Ours baseline' and 'Ours w/o text' cannot be distinguished from noise. In addition, Section 4.3 refers to 'Tab. 4,' but no Table 4 is present in the manuscript; either the missing table must be added or the cross-reference corrected. The ablations are important because they support the two main architectural contributions, masked object attention and text guidance, and they deserve the same statistical rigor as the main comparison.
minor comments (5)
- [§3.1, §3.2] The abbreviation 'V AE' appears with a space in several places and should be written as 'VAE' for consistency.
- [Table 1 caption] The caption contains the typo 'we us our first frames' and should read 'we use our first frames.'
- [§4.2] The prompt description contains the typo 'graps' and should read 'grasps.'
- [Reference [56]] The author listing 'Team Seawead' appears to be a typo for 'Team Seaweed.'
- [§3.5, §4.2] The test set used for Tables 1 and 2 is not described: the number of test videos, product categories, and selection protocol are missing, which makes it difficult to assess sampling bias or generalizability.
Circularity Check
No significant circularity: DreamActor-H1 is an empirical image-to-video pipeline; the motion-template limitation is acknowledged in the paper and does not reduce a claimed derivation to its own inputs.
full rationale
DreamActor-H1 is a trained image-to-video generation system rather than a formal derivation, so the classic circularity patterns (result equal to fit by construction, uniqueness theorem imported from the authors, ansatz smuggled in via self-citation) do not apply. The appearance-preservation claim is evaluated with external metrics (CLIP-I, DINO-I, ArcFace) against reference images, and the motion branch uses a pre-defined template pool selected by VLM/rule-based matching. The authors explicitly flag this in Sec. 5: the system 'may produce unnatural interactions due to relying on pre-defined motions unrelated to specific products' and 'the VLM may inaccurately judge product sizes, especially for non-standard shapes.' These are stated limitations on generalizability, not circular steps. The only mild self-citation is the use of reference attention 'following [39,45]' in Sec. 3.2, where [45] (DreamActor-M1) and [39] (OmniHuman-1) share some authors with this paper; however, this is a design choice rather than a load-bearing uniqueness argument, and the paper's claimed contribution is the additional masked object attention and structured text conditioning. No fitted parameter is renamed as a prediction, and no evaluation metric reduces to the training target by construction. The system is therefore self-contained with respect to external benchmarks, and the acknowledged limitations should be weighed as correctness risk rather than circularity.
Assumptions & free parameters
free parameters (2)
- Classifier-free guidance scale =
2.5
- Region-specific loss weights for faces, hands, and products =
not reported
assumptions (5)
- domain assumption SMPL-X and FaceVerse models capture the range of human demonstration motions sufficiently for training and inference.
- domain assumption GroundingDINO and SAM2 provide accurate product detection and segmentation for generating motion guidance and for evaluation masks.
- domain assumption The vision-language model Seed1.5-VL correctly estimates product size, category, material, and text for structured text guidance and box scaling.
- domain assumption The pre-defined motion template pool, built from the training set, covers the space of demonstration motions and product sizes (1 to 40 centimeters).
- domain assumption The pretrained Seaweed-7B model supplies a strong video diffusion prior that transfers to the human-product demonstration task.
Cite this review
Pith. "Pith review of DreamActor-H1: High-Fidelity Human-Product Demonstration Video Generation via Motion-designed Diffusion Transformers." pith.science (2026). https://pith.science/paper/UBBHENJ6
@misc{pith2026250610568,
author = {Pith},
title = {Pith review of: DreamActor-H1: High-Fidelity Human-Product Demonstration Video Generation via Motion-designed Diffusion Transformers},
year = {2026},
howpublished = {\url{https://pith.science/paper/UBBHENJ6}},
note = {Machine review of arXiv:2506.10568}
}
read the original abstract
In e-commerce and digital marketing, generating high-fidelity human-product demonstration videos is important for effective product presentation. However, most existing frameworks either fail to preserve the identities of both humans and products or lack an understanding of human-product spatial relationships, leading to unrealistic representations and unnatural interactions. To address these challenges, we propose a Diffusion Transformer (DiT)-based framework. Our method simultaneously preserves human identities and product-specific details, such as logos and textures, by injecting paired human-product reference information and utilizing an additional masked cross-attention mechanism. We employ a 3D body mesh template and product bounding boxes to provide precise motion guidance, enabling intuitive alignment of hand gestures with product placements. Additionally, structured text encoding is used to incorporate category-level semantics, enhancing 3D consistency during small rotational changes across frames. Trained on a hybrid dataset with extensive data augmentation strategies, our approach outperforms state-of-the-art techniques in maintaining the identity integrity of both humans and products and generating realistic demonstration motions. Project page: https://lizhenwangt.github.io/DreamActor-H1/.
Figures
Figures from the paper (4 more)
Forward citations
Cited by 8 Pith papers
-
AgentHOI: Multi-Agent Reasoning for Human-Object-Interaction Video Generation via Implicit Representation Alignment
AgentHOI generates human-object interaction videos from text plus one human image and one object image, using multi-agent action planning and implicit text-to-motion feature alignment inside a video diffusion model.
-
StreamHOI: Interaction-aware Temporal Memory Adaptation for Streaming HOI Video Generation
A streaming image-to-video model that assigns block-specific sink/local memory and learned RoPE distance scaling preserves long human-object interactions under bounded latency.
-
HOMIE: Human-object Centric Video Personalization via Multimodal Intelligent Enhancement
HOMIE unifies inter- and intra-subject video personalization by injecting MLLM-derived relational features into DiT self-attention (GMG) and tagging tokens with modality/reference embeddings (MRE), reporting SOTA on a...
-
MVHOI: Bridge Multi-view Condition to Complex Human-Object Interaction Video Reenactment via 3D Foundation Model
Using a 3D foundation model to produce viewpoint-aware anchors plus multi-view reference textures enables realistic human-object-interaction reenactment with large out-of-plane rotations.
-
HairWeaver: Few-Shot Photorealistic Hair Motion Synthesis with Sim-to-Real Guided Video Diffusion
HairWeaver animates a single human photo with physically plausible hair motion by transferring simulated CG hair dynamics into a frozen video diffusion model via two lightweight LoRA adapters.
-
HOComp: Interaction-Aware Human-Object Composition
A diffusion-transformer method that composes a foreground object into a human image with MLLM-chosen interaction regions, pose keypoint supervision, and appearance/background consistency losses, plus a new paired dataset.
-
Real-Time Human-Centric World Modeling for Upper-Body Human-Object Interaction
A distilled video model jointly controls multi-scale upper-body motion latents and two discrete contact states to generate real-time human–object interaction at 25 FPS.
-
DreamSwapV: Mask-guided Subject Swapping for Any Customized Video Editing
DreamSwapV performs mask-guided, subject-agnostic subject swapping in videos using multiple conditions, an adaptive mask strategy, and a new benchmark.
Reference graph
Works this paper leans on
-
[1]
Stable video diffusion: Scaling latent video diffusion models to large datasets
Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023. 3
arXiv 2023
-
[2]
Magicpose: Realistic human poses and facial expressions retar- geting with identity-aware diffusion
Di Chang, Yichun Shi, Quankai Gao, Jessica Fu, Hongyi Xu, Guoxian Song, Qing Yan, Yizhe Zhu, Xiao Yang, and Mohammad Soleymani. Magicpose: Realistic human poses and facial expressions retar- geting with identity-aware diffusion. arXiv preprint arXiv:2311.12052, 2023. 2, 3
arXiv 2023
-
[3]
Binghui Chen, Chongyang Zhong, Wangmeng Xi- ang, Yifeng Geng, and Xuansong Xie. Virtualmodel: Generating object-id-retentive human-object interac- tion image by diffusion model for e-commerce mar- keting. arXiv preprint arXiv:2405.09985, 2024. 2
arXiv 2024
-
[4]
Demamba: Ai- generated video detection on million-scale genvideo benchmark
Haoxing Chen, Yan Hong, Zizheng Huang, Zhuoer Xu, Zhangxuan Gu, Yaohui Li, Jun Lan, Huijia Zhu, Jianfu Zhang, Weiqiang Wang, et al. Demamba: Ai- generated video detection on million-scale genvideo benchmark. arXiv preprint arXiv:2405.19707, 2024. 8
arXiv 2024
-
[5]
Multi-subject open-set per- sonalization in video generation
Tsai-Shien Chen, Aliaksandr Siarohin, Willi Mena- pace, Yuwei Fang, Kwot Sin Lee, Ivan Skorokhodov, Kfir Aberman, Jun-Yan Zhu, Ming-Hsuan Yang, and Sergey Tulyakov. Multi-subject open-set per- sonalization in video generation. arXiv preprint arXiv:2501.06187, 2025. 3
arXiv 2025
-
[6]
Arcface: Additive angular margin loss for deep face recognition
Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Arcface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4690–4699, 2019. 6
2019
-
[7]
Cinema: Coherent multi-subject video generation via mllm- based guidance
Yufan Deng, Xun Guo, Yizhi Wang, Jacob Zhiyuan Fang, Angtian Wang, Shenghai Yuan, Yiding Yang, Bo Liu, Haibin Huang, and Chongyang Ma. Cinema: Coherent multi-subject video generation via mllm- based guidance. arXiv preprint arXiv:2503.10391 ,
-
[8]
Cg-hoi: Contact- guided 3d human-object interaction generation
Christian Diller and Angela Dai. Cg-hoi: Contact- guided 3d human-object interaction generation. In Proc. Computer Vision and Pattern Recognition (CVPR), IEEE, 2024. 2
work page 2024
Show all 81 references
-
[9]
Scaling rectified flow transformers for high-resolution image synthesis
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international confer- ence on machi...
2024
-
[10]
Re-hold: Video hand object interaction reenactment via adaptive layout-instructed diffusion model
Yingying Fan, Quanwei Yang, Kaisiyuan Wang, Hang Zhou, Yingying Li, Haocheng Feng, Errui Ding, Yu Wu, and Jingdong Wang. Re-hold: Video hand object interaction reenactment via adaptive layout-instructed diffusion model. CVPR, 2025. 2
2025
-
[11]
Humandit: Pose-guided diffusion transformer for long-form human motion video generation
Qijun Gan, Yi Ren, Chen Zhang, Zhenhui Ye, Pan Xie, Xiang Yin, Zehuan Yuan, Bingyue Peng, and Jianke Zhu. Humandit: Pose-guided diffusion transformer for long-form human motion video generation. arXiv preprint arXiv:2502.04847, 2025. 2, 3
2025 arXiv
-
[12]
Seedream 3.0 techni- cal report
Yu Gao, Lixue Gong, Qiushan Guo, Xiaoxia Hou, Zhichao Lai, Fanshi Li, Liang Li, Xiaochen Lian, Chao Liao, Liyang Liu, et al. Seedream 3.0 techni- cal report. arXiv preprint arXiv:2504.11346, 2025. 6
2025 arXiv
-
[13]
Imos: Intent-driven full-body motion synthesis for human- object interactions
Anindita Ghosh, Rishabh Dabral, Vladislav Golyanik, Christian Theobalt, and Philipp Slusallek. Imos: Intent-driven full-body motion synthesis for human- object interactions. In Eurographics, 2023. 2
2023
-
[14]
Hu- mans in 4d: Reconstructing and tracking humans with transformers
Shubham Goel, Georgios Pavlakos, Jathushan Ra- jasegaran, Angjoo Kanazawa, and Jitendra Malik. Hu- mans in 4d: Reconstructing and tracking humans with transformers. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision , pages 14783–14794, 2023. 5
2023
-
[15]
Talk-act: Enhance textural-awareness for 2d speaking avatar reenactment with diffusion model
Jiazhi Guan, Quanwei Yang, Kaisiyuan Wang, Hang Zhou, Shengyi He, Zhiliang Xu, Haocheng Feng, Er- rui Ding, Jingdong Wang, Hongtao Xie, et al. Talk-act: Enhance textural-awareness for 2d speaking avatar reenactment with diffusion model. In SIGGRAPH Asia 2024 Conference Papers ...
2024
-
[16]
Mohamed Hassan, Partha Ghosh, Joachim Tesch, Dimitrios Tzionas, and Michael J. Black. Populat- ing 3D scenes by learning human-scene interaction. In Proceedings IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2021. 2
2021
-
[17]
Synthesizing physical character-scene interactions
Mohamed Hassan, Yunrong Guo, Tingwu Wang, Michael Black, Sanja Fidler, and Xue Bin Peng. Synthesizing physical character-scene interactions. In ACM SIGGRAPH 2023 Conference Proceedings , pages 1–9, 2023. 2
2023
-
[18]
Id- animator: Zero-shot identity-preserving human video generation
Xuanhua He, Quande Liu, Shengju Qian, Xin Wang, Tao Hu, Ke Cao, Keyu Yan, and Jie Zhang. Id- animator: Zero-shot identity-preserving human video generation. arXiv preprint arXiv:2404.15275, 2024. 3
2024 arXiv
-
[19]
Interactdiffusion: Inter- action control in text-to-image diffusion models
Jiun Tian Hoe, Xudong Jiang, Chee Seng Chan, Yap- Peng Tan, and Weipeng Hu. Interactdiffusion: Inter- action control in text-to-image diffusion models. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR) , pages 6180–6189, 2024. 2
2024
-
[20]
Hand-object interaction image gener- ation
Hezhen Hu, Weilun Wang, Wengang Zhou, and Houqiang Li. Hand-object interaction image gener- ation. In NeurIPS, 2022. 2
2022
-
[21]
Animate anyone: Consistent and controllable image-to-video synthesis for character animation
Li Hu. Animate anyone: Consistent and controllable image-to-video synthesis for character animation. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 8153– 8163, 2024. 2, 3, 5
2024
-
[22]
Animate anyone 2: High-fidelity character image animation with environment affor- dance
Li Hu, Guangyuan Wang, Zhen Shen, Xin Gao, Dechao Meng, Lian Zhuo, Peng Zhang, Bang Zhang, and Liefeng Bo. Animate anyone 2: High-fidelity character image animation with environment affor- dance. arXiv preprint arXiv:2502.06145, 2025. 2, 3
2025 arXiv
-
[23]
Hunyuancus- tom: A multimodal-driven architecture for customized video generation
Teng Hu, Zhentao Yu, Zhengguang Zhou, Sen Liang, Yuan Zhou, Qin Lin, and Qinglin Lu. Hunyuancus- tom: A multimodal-driven architecture for customized video generation. arXiv preprint arXiv:2505.04512 ,
-
[24]
Amo sampler: Enhancing text rendering with overshooting
Xixi Hu, Keyang Xu, Bo Liu, Qiang Liu, and Hongliang Fei. Amo sampler: Enhancing text rendering with overshooting. arXiv preprint arXiv:2411.19415, 2024. 6
2024 arXiv
-
[25]
Personahoi: Effortlessly improving personalized face with human-object interaction gen- eration
Xinting Hu, Haoran Wang, Jan Eric Lenssen, and Bernt Schiele. Personahoi: Effortlessly improving personalized face with human-object interaction gen- eration. arXiv preprint arXiv:2501.05823, 2025. 2
2025 arXiv
-
[26]
Videomage: Multi-subject and motion cus- tomization of text-to-video diffusion models
Chi-Pin Huang, Yen-Siang Wu, Hung-Kai Chung, Kai-Po Chang, Fu-En Yang, and Yu-Chiang Frank Wang. Videomage: Multi-subject and motion cus- tomization of text-to-video diffusion models. arXiv preprint arXiv:2503.21781, 2025. 3
2025 arXiv
-
[27]
Hoigpt: Learning long se- quence hand-object interaction with language models
Mingzhen Huang, Fu-Jen Chu, Bugra Tekin, Kevin J Liang, Haoyu Ma, Weiyao Wang, Xingyu Chen, Pierre Gleize, Hongfei Xue, Siwei Lyu, Kris Kitani, Matt Feiszli, and Hao Tang. Hoigpt: Learning long se- quence hand-object interaction with language models. In IEEE Conference on Comp...
2025
-
[28]
Conceptmaster: Multi- concept video customization on diffusion transformer models without test-time tuning
Yuzhou Huang, Ziyang Yuan, Quande Liu, Qiulin Wang, Xintao Wang, Ruimao Zhang, Pengfei Wan, Di Zhang, and Kun Gai. Conceptmaster: Multi- concept video customization on diffusion transformer models without test-time tuning. arXiv preprint arXiv:2501.04698, 2025. 3
2025 arXiv
-
[29]
Vbench: Comprehensive benchmark suite for video generative models
Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianx- ing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive benchmark suite for video generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and P...
2024
-
[30]
Make-your- anchor: A diffusion-based 2d avatar generation frame- work
Ziyao Huang, Fan Tang, Yong Zhang, Xiaodong Cun, Juan Cao, Jintao Li, and Tong-Yee Lee. Make-your- anchor: A diffusion-based 2d avatar generation frame- work. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6997–7006, 2024. 2, 3
2024
-
[31]
Videobooth: Diffusion-based video generation with image prompts
Yuming Jiang, Tianxing Wu, Shuai Yang, Chenyang Si, Dahua Lin, Yu Qiao, Chen Change Loy, and Ziwei Liu. Videobooth: Diffusion-based video generation with image prompts. 2023. 3
2023
-
[32]
Vace: All-in- one video creation and editing
Zeyinzi Jiang, Zhen Han, Chaojie Mao, Jingfeng Zhang, Yulin Pan, and Yu Liu. Vace: All-in- one video creation and editing. arXiv preprint arXiv:2503.07598, 2025. 3, 6, 7
2025 arXiv
-
[33]
Dreampose: Fashion image-to-video synthesis via stable diffu- sion
Johanna Karras, Aleksander Holynski, Ting-Chun Wang, and Ira Kemelmacher-Shlizerman. Dreampose: Fashion image-to-video synthesis via stable diffu- sion. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 22623–22633. IEEE,
2023
-
[34]
Subject- driven video generation via disentangled identity and motion
Daneul Kim, Jingxu Zhang, Wonjoon Jin, Sunghyun Cho, Qi Dai, Jaesik Park, and Chong Luo. Subject- driven video generation via disentangled identity and motion. arXiv preprint arXiv:2504.17816, 2025. 3
2025 arXiv
-
[35]
Auto- encoding variational bayes, 2013
Diederik P Kingma, Max Welling, et al. Auto- encoding variational bayes, 2013. 3
2013
-
[36]
Dispose: Disentangling pose guidance for controllable human image animation
Hongxiang Li, Yaowei Li, Yuhang Yang, Junjie Cao, Zhihong Zhu, Xuxin Cheng, and Long Chen. Dispose: Disentangling pose guidance for controllable human image animation. arXiv preprint arXiv:2412.09349 ,
-
[37]
Object motion guided human motion synthesis
Jiaman Li, Jiajun Wu, and C Karen Liu. Object motion guided human motion synthesis. ACM Trans. Graph., 42(6), 2023. 2
2023
-
[38]
Movie weaver: Tuning-free multi-concept video personalization with anchored prompts
Feng Liang, Haoyu Ma, Zecheng He, Tingbo Hou, Ji Hou, Kunpeng Li, Xiaoliang Dai, Felix Juefei-Xu, Samaneh Azadi, Animesh Sinha, et al. Movie weaver: Tuning-free multi-concept video personalization with anchored prompts. arXiv preprint arXiv:2502.07802,
-
[39]
Omnihuman-1: Rethinking the scaling-up of one-stage conditioned human animation models
Gaojie Lin, Jianwen Jiang, Jiaqi Yang, Zerong Zheng, and Chao Liang. Omnihuman-1: Rethinking the scaling-up of one-stage conditioned human animation models. arXiv preprint arXiv:2502.01061, 2025. 3
2025 arXiv
-
[40]
Flow match- ing for generative modeling
Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow match- ing for generative modeling. arXiv preprint arXiv:2210.02747, 2022. 6
2022 arXiv
-
[41]
Hoigen- 1m: A large-scale dataset for human-object interaction video generation
Kun Liu, Qi Liu, Xinchen Liu, Jie Li, Yongdong Zhang, Jiebo Luo, Xiaodong He, and Wu Liu. Hoigen- 1m: A large-scale dataset for human-object interaction video generation. In CVPR, 2025. 2
2025
-
[42]
Phan- tom: Subject-consistent video generation via cross- modal alignment
Lijie Liu, Tianxaing Ma, Bingchuan Li, Zhuowei Chen, Jiawei Liu, Qian He, and Xinglong Wu. Phan- tom: Subject-consistent video generation via cross- modal alignment. arXiv preprint arXiv:2502.11079 ,
-
[43]
Grounding dino: Marry- ing dino with grounded pre-training for open-set ob- ject detection
Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jian- wei Yang, Hang Su, et al. Grounding dino: Marry- ing dino with grounded pre-training for open-set ob- ject detection. In European Conference on Computer Vision, pages 38–55. Sp...
2024
-
[44]
Easyhoi: Unleashing the power of large models for reconstructing hand-object interac- tions in the wild
Yumeng Liu, Xiaoxiao Long, Zemin Yang, Yuan Liu, Marc Habermann, Christian Theobalt, Yuexin Ma, and Wenping Wang. Easyhoi: Unleashing the power of large models for reconstructing hand-object interac- tions in the wild. arXiv preprint arXiv:2411.14280 ,
-
[45]
Dreamactor- m1: Holistic, expressive and robust human image animation with hybrid guidance
Yuxuan Luo, Zhengkun Rong, Lizhen Wang, Longhao Zhang, Tianshu Hu, and Yongming Zhu. Dreamactor- m1: Holistic, expressive and robust human image animation with hybrid guidance. arXiv preprint arXiv:2504.01724, 2025. 2, 3
2025 arXiv
-
[46]
Follow your pose: Pose-guided text-to-video generation using pose-free videos
Yue Ma, Yingqing He, Xiaodong Cun, Xintao Wang, Siran Chen, Xiu Li, and Qifeng Chen. Follow your pose: Pose-guided text-to-video generation using pose-free videos. In Proceedings of the AAAI Con- ference on Artificial Intelligence , pages 4117–4125,
-
[47]
Mimo: Controllable character video synthesis with spatial decomposed modeling
Yifang Men, Yuan Yao, Miaomiao Cui, and Liefeng Bo. Mimo: Controllable character video synthesis with spatial decomposed modeling. arXiv preprint arXiv:2409.16160, 2024. 2, 3
2024 arXiv
-
[48]
Maxime Oquab, Timoth ´ee Darcet, Theo Moutakanni, Huy V . V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Russell Howes, Po-Yao Huang, Hu Xu, Vasu Sharma, Shang-Wen Li, Wojciech Galuba, Mike Rabbat, Mido Assran, Nicol...
2023
-
[49]
Georgios Pavlakos, Vasileios Choutas, Nima Ghor- bani, Timo Bolkart, Ahmed A. A. Osman, Dimitrios Tzionas, and Michael J. Black. Expressive body cap- ture: 3d hands, face, and body from a single image. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)...
2019
-
[50]
Reconstructing hands in 3D with transformers
Georgios Pavlakos, Dandan Shan, Ilija Radosavovic, Angjoo Kanazawa, David Fouhey, and Jitendra Ma- lik. Reconstructing hands in 3D with transformers. In CVPR, 2024. 5
2024
-
[51]
Scalable diffu- sion models with transformers
William Peebles and Saining Xie. Scalable diffu- sion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vi- sion, pages 4195–4205, 2023. 2, 3
2023
-
[52]
Controlnext: Pow- erful and efficient control for image and video gener- ation
Bohao Peng, Jian Wang, Yuechen Zhang, Wenbo Li, Ming-Chang Yang, and Jiaya Jia. Controlnext: Pow- erful and efficient control for image and video gener- ation. arXiv preprint arXiv:2408.06070, 2024. 2, 3
2024 arXiv
-
[53]
Hoi-diff: Text- driven synthesis of 3d human-object interactions using diffusion models
Xiaogang Peng, Yiming Xie, Zizhao Wu, Varun Jam- pani, Deqing Sun, and Huaizu Jiang. Hoi-diff: Text- driven synthesis of 3d human-object interactions using diffusion models. arXiv preprint arXiv:2312.06553 ,
-
[54]
Learning transferable visual models from natural language supervision
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sas- try, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, ...
2021
-
[55]
Sam 2: Segment anything in images and videos
Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman R ¨adle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.00714 , 2024. 5
2024 arXiv
-
[56]
Seaweed-7b: Cost-effective training of video generation foundation model
Team Seawead, Ceyuan Yang, Zhijie Lin, Yang Zhao, Shanchuan Lin, Zhibei Ma, Haoyuan Guo, Hao Chen, Lu Qi, Sen Wang, et al. Seaweed-7b: Cost-effective training of video generation foundation model. arXiv preprint arXiv:2504.08685, 2025. 3, 6
2025 arXiv
-
[57]
Seed1.5-vl technical report
ByteDance Seed Team. Seed1.5-vl technical report. arXiv preprint arXiv:2505.07062, 2025. 5
2025 arXiv
-
[58]
Stableanimator: High-quality identity-preserving human image anima- tion
Shuyuan Tu, Zhen Xing, Xintong Han, Zhi-Qi Cheng, Qi Dai, Chong Luo, and Zuxuan Wu. Stableanimator: High-quality identity-preserving human image anima- tion. arXiv preprint arXiv:2411.17697, 2024. 2, 3
2024 arXiv
-
[59]
Wan: Open and advanced large-scale video generative models
Team Wan, Ang Wang, Baole Ai, Bin Wen, Chao- jie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haim- ing Zhao, Jianxiao Yang, Jianyuan Zeng, Jiayu Wang, Jingfeng Zhang, Jingren Zhou, Jinkai Wang, Jixuan Chen, Kai Zhu, Kang Zhao, Keyu Yan, Lianghua Huang, Mengyang Feng, Ningyi Zhang, P...
2025 arXiv
-
[60]
Faceverse: a fine-grained and detail-controllable 3d face morphable model from a hybrid dataset
Lizhen Wang, Zhiyuan Chen, Tao Yu, Chenguang Ma, Liang Li, and Yebin Liu. Faceverse: a fine-grained and detail-controllable 3d face morphable model from a hybrid dataset. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 20333–20342...
2022
-
[61]
Disco: Disentan- gled control for realistic human dance generation
Tan Wang, Linjie Li, Kevin Lin, Yuanhao Zhai, Chung-Ching Lin, Zhengyuan Yang, Hanwang Zhang, Zicheng Liu, and Lijuan Wang. Disco: Disentan- gled control for realistic human dance generation. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognitio...
2024
-
[62]
Unianimate: Taming unified video diffusion models for consistent human image anima- tion
Xiang Wang, Shiwei Zhang, Changxin Gao, Jiayu Wang, Xiaoqiang Zhou, Yingya Zhang, Luxin Yan, and Nong Sang. Unianimate: Taming unified video diffusion models for consistent human image anima- tion. arXiv preprint arXiv:2406.01188, 2024. 2, 3, 6, 7
2024 arXiv
-
[63]
Dreamvideo: Composing your dream videos with customized subject and motion
Yujie Wei, Shiwei Zhang, Zhiwu Qing, Hangjie Yuan, Zhiheng Liu, Yu Liu, Yingya Zhang, Jingren Zhou, and Hongming Shan. Dreamvideo: Composing your dream videos with customized subject and motion. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recogni...
2024
-
[64]
InterDiff: Generating 3d human-object in- teractions with physics-informed diffusion
Sirui Xu, Zhengyuan Li, Yu-Xiong Wang, and Liang- Yan Gui. InterDiff: Generating 3d human-object in- teractions with physics-informed diffusion. In ICCV,
-
[65]
Intermimic: Towards universal whole- body control for physics-based human-object interac- tions
Sirui Xu, Hung Yu Ling, Yu-Xiong Wang, and Liangyan Gui. Intermimic: Towards universal whole- body control for physics-based human-object interac- tions. In CVPR, 2025. 2
2025
-
[66]
Anchorcrafter: Ani- mate cyber-anchors selling your products via human- object interacting video generation
Ziyi Xu, Ziyao Huang, Juan Cao, Yong Zhang, Xi- aodong Cun, Qing Shuai, Yuchen Wang, Linchao Bao, Jintao Li, and Fan Tang. Anchorcrafter: Ani- mate cyber-anchors selling your products via human- object interacting video generation. arXiv preprint arXiv:2411.17383, 2024. 2, 6, 7
2024 arXiv
-
[67]
Magicanimate: Temporally con- sistent human image animation using diffusion model
Zhongcong Xu, Jianfeng Zhang, Jun Hao Liew, Han- shu Yan, Jia-Wei Liu, Chenxu Zhang, Jiashi Feng, and Mike Zheng Shou. Magicanimate: Temporally con- sistent human image animation using diffusion model. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern ...
2024
-
[68]
Hoi-swap: Swapping objects in videos with hand-object interaction awareness
Zihui Xue, Mi Luo, Chen Changan, and Kristen Grau- man. Hoi-swap: Swapping objects in videos with hand-object interaction awareness. arXiv preprint arXiv:2406.07754, 2024. 2
2024 arXiv
-
[69]
Diffusion-guided reconstruction of ev- eryday hand-object interaction clips
Yufei Ye, Abhinav Hebbar, Poorvi Gupta, and Shub- ham Tulsiani. Diffusion-guided reconstruction of ev- eryday hand-object interaction clips. In ICCV, 2023. 2
2023
-
[70]
Affordance diffusion: Synthesiz- ing hand-object interactions
Yufei Ye, Xueting Li, Abhinav Gupta, Shalini De Mello, Stan Birchfield, Jiaming Song, Shubham Tul- siani, and Sifei Liu. Affordance diffusion: Synthesiz- ing hand-object interactions. In CVPR, 2023. 2
2023
-
[71]
Identity-preserving text-to-video genera- tion by frequency decomposition
Shenghai Yuan, Jinfa Huang, Xianyi He, Yun- yuan Ge, Yujun Shi, Liuhan Chen, Jiebo Luo, and Li Yuan. Identity-preserving text-to-video genera- tion by frequency decomposition. arXiv preprint arXiv:2411.17440, 2024. 3
2024 arXiv
-
[72]
GraspXL: Generating grasp- ing motions for diverse objects at scale
Hui Zhang, Sammy Christen, Zicong Fan, Otmar Hilliges, and Jie Song. GraspXL: Generating grasp- ing motions for diverse objects at scale. In European Conference on Computer Vision (ECCV), 2024. 2
2024
-
[73]
Scaling in-the-wild training for diffusion-based illu- mination harmonization and editing by imposing con- sistent light transport
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Scaling in-the-wild training for diffusion-based illu- mination harmonization and editing by imposing con- sistent light transport. In The Thirteenth International Conference on Learning Representations, 2025. 6
2025
-
[74]
Hoidiffusion: Gen- erating realistic 3d hand-object interaction data
Mengqi Zhang, Yang Fu, Zheng Ding, Sifei Liu, Zhuowen Tu, and Xiaolong Wang. Hoidiffusion: Gen- erating realistic 3d hand-object interaction data. arXiv preprint arXiv:2403.12011, 2024. 2
2024 arXiv
-
[75]
Place: Proximity learning of articula- tion and contact in 3d environments
Siwei Zhang, Yan Zhang, Qianli Ma, Michael J Black, and Siyu Tang. Place: Proximity learning of articula- tion and contact in 3d environments. In 2020 Inter- national Conference on 3D Vision (3DV), pages 642–
2020
-
[76]
Generating 3d peo- ple in scenes without people
Yan Zhang, Mohamed Hassan, Heiko Neumann, Michael J Black, and Siyu Tang. Generating 3d peo- ple in scenes without people. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6194–6204, 2020. 2
2020
-
[77]
Mim- icmotion: High-quality human motion video gener- ation with confidence-aware pose guidance
Yuang Zhang, Jiaxi Gu, Li-Wen Wang, Han Wang, Junqi Cheng, Yuefeng Zhu, and Fangyuan Zou. Mim- icmotion: High-quality human motion video gener- ation with confidence-aware pose guidance. arXiv preprint arXiv:2406.19680, 2024. 2, 3
2024 arXiv
-
[78]
Taste-rob: Advancing video generation of task-oriented hand- object interaction for generalizable robotic manipula- tion
Hongxiang Zhao, Xingchen Liu, Mutian Xu, Yiming Hao, Weikai Chen, and Xiaoguang Han. Taste-rob: Advancing video generation of task-oriented hand- object interaction for generalizable robotic manipula- tion. In CVPR, 2025. 2
2025
-
[79]
Concat-id: Towards universal identity-preserving video synthesis
Yong Zhong, Zhuoyi Yang, Jiayan Teng, Xiaotao Gu, and Chongxuan Li. Concat-id: Towards universal identity-preserving video synthesis. arXiv preprint arXiv:2503.14151, 2025. 3
2025 arXiv
-
[80]
Dormant: Defending against pose-driven human image animation
Jiachen Zhou, Mingsi Wang, Tianlin Li, Guozhu Meng, and Kai Chen. Dormant: Defending against pose-driven human image animation. arXiv preprint arXiv:2409.14424, 2024. 8
2024 arXiv
-
[81]
Champ: Controllable and consistent human image animation with 3d parametric guidance
Shenhao Zhu, Junming Leo Chen, Zuozhuo Dai, Zi- long Dong, Yinghui Xu, Xun Cao, Yao Yao, Hao Zhu, and Siyu Zhu. Champ: Controllable and consistent human image animation with 3d parametric guidance. In European Conference on Computer Vision , pages 145–162. Springer, 2024. 2, 3
2024
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.