Pith. sign in

REVIEW 4 major objections 5 minor 8 cited by

DreamActor-H1: High-Fidelity Human-Product Demonstration Video Generation via Motion-designed Diffusion Transformers

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Pith's one-line read DreamActor-H1 claims it can generate a realistic human-product demonstration video from a single human photo and a single product photo, preserving both identities.

desk verdict A useful task formulation and an honest architecture, but the comparative evaluation is too weak to support the claimed edge over SOTA, and the motion template assumption is under-tested. read the letter →

arxiv 2506.10568 v2 pith:UBBHENJ6 submitted 2025-06-12 cs.CV cs.AI

classification cs.CVcs.AI
keywords human-productdemonstrationvideogenerationdiffusiontransformeridentitypreservationhuman-objectinteractionmotionguidancemaskedcross-attentione-commerce
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a diffusion-transformer video generator can take two reference images, one of a person and one of a product, and synthesize a short demonstration video in which the person's face and the product's logo, texture, and contour all remain recognizable, and in which hand motions line up with the product. This matters because e-commerce and marketing currently need studio shoots or multi-view product captures to produce such demos, while a two-image pipeline would make personalized advertising videos cheap and accessible. The claimed mechanism combines reference attention with a masked cross-attention that injects a fixed product latent, plus motion guidance built from a 3D body mesh template and a product bounding box, with structured text describing category-level semantics. If correct, the framework offers a practical template-based route to identity-preserving human-product video generation.

What carries the argument

The load-bearing mechanism is the masked object attention: a VAE-encoded product latent, held fixed rather than self-updating, is concatenated with video tokens, processed by self-attention, scaled by a product-region mask, and added back as a residual, which prevents the network from washing out fine product detail as the diffusion transformer deepens. Around it sit two supporting mechanisms: reference attention for human-product appearance injection, and motion guidance that concatenates a colored 3D body mesh template (SMPL-X) plus the product bounding box with the input noise, with inference-time template matching that picks and resizes motions from a pool covering products of roughly 1 to 40 centimeters.

What would settle it

Take a product with a non-standard shape, such as an umbrella or a transparent bottle, and a person image with no table in front of them, run the inference pipeline, and check whether the hand and product bounding boxes overlap and move coherently across frames: if the vision-language model misestimates size or the matched template cannot fit the scene, the generated frames will show the hand passing through the product or the product floating, contradicting the motion-realism claim.

Watch

Extended reading notes

Core claim

The central claim is that DreamActor-H1 simultaneously preserves human identity and fine-grained product identity while producing natural demonstration motions, something the authors argue existing pose-guided, human-object-interaction, and multi-subject customization methods cannot do from the same input format. The method encodes the human reference and product reference through a variational autoencoder; reference attention injects appearance, and an additional masked object attention injects the product latent as a residual update restricted to product-containing regions, so logos and textures survive the diffusion process. Motion comes from a 3D body mesh template and a product bounding box, with an automatic template-matching stage that selects and scales a predefined motion according to product size, body orientation, and aspect ratio. Structured dictionary-style text generated by a vision-language model adds category-level semantics. In quantitative comparisons and a user study, the full model reports the best CLIP-I, DINO-I, FaceSim-Arc, and win rates against the compared baselines.

Load-bearing premise

The load-bearing assumption is that a predefined pool of motion templates plus a rule-based match using product size, body orientation, and aspect ratio yields natural hand-product interactions for any given input pair; when that match fails, the video's gestures will not align with the product.

Editorial extensions

If this is right

  • An e-commerce seller could generate a personalized 65-frame, 720 by 1280 demonstration video from two images, with no multi-view product capture or depth maps.
  • Product details that normally break in video generation, such as logos, text, and textures on packaging, are claimed to survive because the masked object attention injects a fixed product latent rather than relying only on text prompts.
  • The automatic motion-template selection means the same framework can adapt to products of different sizes and aspect ratios within the 1 to 40 centimeter range without requiring the user to supply a pose sequence.
  • The structured text encoding is asserted to improve 3D consistency during small product rotations, so the product does not morph when the demonstrator turns it in hand.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The authors' own limitation note says the template pool can produce unnatural interactions when the input scene does not match the template, such as a table-pickup template requiring a table in front of the person; this suggests replacing rule-based template matching with a learned contact-aware motion retargeting model as the natural next step.
  • The masked object attention idea could transfer beyond products to any two-subject video generation task, such as a person interacting with a specific pet, instrument, or piece of furniture, whenever one subject's fine detail must be preserved.
  • A testable extension is to measure identity preservation under exactly the failure mode the authors name: fast product rotation, where VAE temporal compression can make text and textures flicker.
  • Because product-size estimation is delegated to a vision-language model, non-standard shapes are a known failure point; replacing that estimate with a direct metric estimate from the image or a user-supplied physical size would provide a cheap ablation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. DreamActor-H1 proposes a Diffusion Transformer framework that takes a human reference image and a product reference image and generates a demonstration video. Appearance is injected through reference attention plus a masked object attention that adds fixed VAE-encoded product latents as residual updates; motion is guided by SMPL-X body templates and product bounding boxes, with a pre-defined template pool and rule-based template matching at inference; VLM-generated structured text captions supply category-level semantics. The model is initialized from Seaweed-7B and trained on a hybrid dataset of roughly 15,000 collected demonstration videos plus supplementary live-stream and Seaweed data. The paper reports quantitative and user-study comparisons with AnchorCrafter, Phantom, VACE, and UniAnimate-DiT, together with ablations of the object-attention and text modules.

Significance. If the claims held, this would be a practically useful contribution for e-commerce demonstration generation, because it addresses joint human/product identity preservation and motion guidance from only two input images. The architectural ideas, especially the masked object cross-attention and the template-plus-bounding-box motion guidance, are clearly motivated, and the authors are candid about limitations in Section 5. The paper is an empirical systems paper rather than a derivation paper, so the main burden is experimental evidence. That evidence is currently too weak to establish the central claims of outperforming state-of-the-art methods and generating realistic demonstration motions; the framework is plausible, but the evaluation is not yet conclusive.

major comments (4)
  1. [§4.2, Tables 1 and 2] The comparison with state-of-the-art methods is not sufficiently controlled to support the abstract's claim of outperforming SOTA. AnchorCrafter is evaluated on only 3 videos, as the authors state; Phantom and VACE run at different resolutions and on different base models; and UniAnimate-DiT is given the authors' first frames and pose sequences as inputs, so it is not a fair comparison for motion realism or end-to-end demonstration generation. All metrics in Table 1 are reported as single means without error bars, confidence intervals, or significance tests, and the user study in Table 2 reports only 'good' or 'same' rates with unequal sample sizes across methods. These issues are load-bearing for the headline claim and should be addressed with matched evaluation protocols, more baselines under identical input conditions, and statistical reporting.
  2. [§3.3 and §5] The motion realism claim rests on the inference-time template pool and rule-based matching algorithm described in §3.3, but the paper provides no evaluation of template-match success, VLM size-error rates, or coverage of arbitrary input pairs. The authors explicitly state in Section 5 that the system 'may produce unnatural interactions due to relying on pre-defined motions unrelated to specific products' and that the VLM 'may inaccurately judge product sizes, especially for non-standard shapes.' Because the test set is not described as including such difficult cases, the central motion contribution is validated only under favorable conditions. The paper should report failure rates of template matching on a diverse test set, break down results by product shape/size, and analyze cases where the VLM size prediction is incorrect.
  3. [§4.2] There is no direct metric for interaction correctness, despite 'realistic demonstration motions' and 'intuitive alignment of hand gestures with product placements' being central claims. The only motion-related metric is VBench 'motion smoothness,' which is near-ceiling for all methods and does not measure hand-object contact, spatial alignment, occlusion, or physical plausibility. The user study asks about human identity, product identity, and overall video quality, but does not isolate motion realism or interaction correctness. A dedicated human evaluation or automated hand-object alignment metric on a common set of inputs is needed to substantiate the motion contribution.
  4. [§4.3, Table 1] The ablation study reports single-run mean scores without variance or significance testing, so the improvements of 'Ours' over 'Ours baseline' and 'Ours w/o text' cannot be distinguished from noise. In addition, Section 4.3 refers to 'Tab. 4,' but no Table 4 is present in the manuscript; either the missing table must be added or the cross-reference corrected. The ablations are important because they support the two main architectural contributions, masked object attention and text guidance, and they deserve the same statistical rigor as the main comparison.
minor comments (5)
  1. [§3.1, §3.2] The abbreviation 'V AE' appears with a space in several places and should be written as 'VAE' for consistency.
  2. [Table 1 caption] The caption contains the typo 'we us our first frames' and should read 'we use our first frames.'
  3. [§4.2] The prompt description contains the typo 'graps' and should read 'grasps.'
  4. [Reference [56]] The author listing 'Team Seawead' appears to be a typo for 'Team Seaweed.'
  5. [§3.5, §4.2] The test set used for Tables 1 and 2 is not described: the number of test videos, product categories, and selection protocol are missing, which makes it difficult to assess sampling bias or generalizability.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: DreamActor-H1 is an empirical image-to-video pipeline; the motion-template limitation is acknowledged in the paper and does not reduce a claimed derivation to its own inputs.

full rationale

DreamActor-H1 is a trained image-to-video generation system rather than a formal derivation, so the classic circularity patterns (result equal to fit by construction, uniqueness theorem imported from the authors, ansatz smuggled in via self-citation) do not apply. The appearance-preservation claim is evaluated with external metrics (CLIP-I, DINO-I, ArcFace) against reference images, and the motion branch uses a pre-defined template pool selected by VLM/rule-based matching. The authors explicitly flag this in Sec. 5: the system 'may produce unnatural interactions due to relying on pre-defined motions unrelated to specific products' and 'the VLM may inaccurately judge product sizes, especially for non-standard shapes.' These are stated limitations on generalizability, not circular steps. The only mild self-citation is the use of reference attention 'following [39,45]' in Sec. 3.2, where [45] (DreamActor-M1) and [39] (OmniHuman-1) share some authors with this paper; however, this is a design choice rather than a load-bearing uniqueness argument, and the paper's claimed contribution is the additional masked object attention and structured text conditioning. No fitted parameter is renamed as a prediction, and no evaluation metric reduces to the training target by construction. The system is therefore self-contained with respect to external benchmarks, and the acknowledged limitations should be weighed as correctness risk rather than circularity.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central claim rests on several domain assumptions rather than new mathematical axioms. The most fragile is the motion template pool and matching rules, which the paper itself flags as limited. No new physical or conceptual entities are introduced.

free parameters (2)
  • Classifier-free guidance scale = 2.5
    Chosen by hand to balance identity fidelity and diversity; reported in Section 4.1.
  • Region-specific loss weights for faces, hands, and products = not reported
    Mentioned in Section 3.5 as part of the flow matching objective, but exact values are not provided.
assumptions (5)
  • domain assumption SMPL-X and FaceVerse models capture the range of human demonstration motions sufficiently for training and inference.
    Invoked in Section 3.3 to fit 3D body and face parameters to both training videos and inference human images.
  • domain assumption GroundingDINO and SAM2 provide accurate product detection and segmentation for generating motion guidance and for evaluation masks.
    Used throughout Section 3.3 for bounding box extraction and in Section 4.2 for evaluation mask creation.
  • domain assumption The vision-language model Seed1.5-VL correctly estimates product size, category, material, and text for structured text guidance and box scaling.
    Relied on in Sections 3.3 and 3.4; the paper admits in Section 5 that size estimates can be inaccurate for non-standard shapes.
  • domain assumption The pre-defined motion template pool, built from the training set, covers the space of demonstration motions and product sizes (1 to 40 centimeters).
    Stated in Section 3.3 as the source of all inference motions; the paper's limitation section notes that mismatched templates can cause unnatural interactions.
  • domain assumption The pretrained Seaweed-7B model supplies a strong video diffusion prior that transfers to the human-product demonstration task.
    Section 3.1 describes initializing from Seaweed-7B; the paper relies on this base model to provide general video generation capabilities.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DreamActor-H1: High-Fidelity Human-Product Demonstration Video Generation via Motion-designed Diffusion Transformers." pith.science (2026). https://pith.science/paper/UBBHENJ6

@misc{pith2026250610568,
  author       = {Pith},
  title        = {Pith review of: DreamActor-H1: High-Fidelity Human-Product Demonstration Video Generation via Motion-designed Diffusion Transformers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UBBHENJ6}},
  note         = {Machine review of arXiv:2506.10568}
}
read the original abstract

In e-commerce and digital marketing, generating high-fidelity human-product demonstration videos is important for effective product presentation. However, most existing frameworks either fail to preserve the identities of both humans and products or lack an understanding of human-product spatial relationships, leading to unrealistic representations and unnatural interactions. To address these challenges, we propose a Diffusion Transformer (DiT)-based framework. Our method simultaneously preserves human identities and product-specific details, such as logos and textures, by injecting paired human-product reference information and utilizing an additional masked cross-attention mechanism. We employ a 3D body mesh template and product bounding boxes to provide precise motion guidance, enabling intuitive alignment of hand gestures with product placements. Additionally, structured text encoding is used to incorporate category-level semantics, enhancing 3D consistency during small rotational changes across frames. Trained on a hybrid dataset with extensive data augmentation strategies, our approach outperforms state-of-the-art techniques in maintaining the identity integrity of both humans and products and generating realistic demonstration motions. Project page: https://lizhenwangt.github.io/DreamActor-H1/.

Figures

Figures reproduced from arXiv: 2506.10568 by the authors.

Figure 1
Figure 1. DreamActor-H1 can generate high-fidelity and photo-realistic human-product demonstration videos from human and product [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The pipeline of DreamActor-H1 leverages a DiT architecture, starting with dataset preparation where a VLM describes product [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. During inference, our framework retrieves optimal mo [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Comparisons with AnchorCrafter [66], Phantom [42], VACE [32] and UniAnimate-DiT※ [62]. Note that we only generate 3 videos for AnchorCrafter, and UniAnimate-DiT uses our first frames and pose sequences as inputs. Methods CLIP-I↑ DINO-I↑ FaceSim-Arc↑ Motion Smoothness↑ …
Figure 5
Figure 5. Figure 5: Ablation studies with “Ours baseline” (w/o object atten [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Our video results generated from human images and product images. [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Our results generated from more human images and product images. [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 8 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. AgentHOI: Multi-Agent Reasoning for Human-Object-Interaction Video Generation via Implicit Representation Alignment

    cs.CV 2026-07 conditional novelty 6.0 of 10

    AgentHOI generates human-object interaction videos from text plus one human image and one object image, using multi-agent action planning and implicit text-to-motion feature alignment inside a video diffusion model.

  2. StreamHOI: Interaction-aware Temporal Memory Adaptation for Streaming HOI Video Generation

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A streaming image-to-video model that assigns block-specific sink/local memory and learned RoPE distance scaling preserves long human-object interactions under bounded latency.

  3. HOMIE: Human-object Centric Video Personalization via Multimodal Intelligent Enhancement

    cs.CV 2026-07 conditional novelty 6.0 of 10

    HOMIE unifies inter- and intra-subject video personalization by injecting MLLM-derived relational features into DiT self-attention (GMG) and tagging tokens with modality/reference embeddings (MRE), reporting SOTA on a...

  4. MVHOI: Bridge Multi-view Condition to Complex Human-Object Interaction Video Reenactment via 3D Foundation Model

    cs.CV 2026-03 conditional novelty 6.0 of 10

    Using a 3D foundation model to produce viewpoint-aware anchors plus multi-view reference textures enables realistic human-object-interaction reenactment with large out-of-plane rotations.

  5. HairWeaver: Few-Shot Photorealistic Hair Motion Synthesis with Sim-to-Real Guided Video Diffusion

    cs.CV 2026-02 conditional novelty 6.0 of 10

    HairWeaver animates a single human photo with physically plausible hair motion by transferring simulated CG hair dynamics into a frozen video diffusion model via two lightweight LoRA adapters.

  6. HOComp: Interaction-Aware Human-Object Composition

    cs.CV 2025-07 conditional novelty 6.0 of 10

    A diffusion-transformer method that composes a foreground object into a human image with MLLM-chosen interaction regions, pose keypoint supervision, and appearance/background consistency losses, plus a new paired dataset.

  7. Real-Time Human-Centric World Modeling for Upper-Body Human-Object Interaction

    cs.CV 2026-07 conditional novelty 5.0 of 10

    A distilled video model jointly controls multi-scale upper-body motion latents and two discrete contact states to generate real-time human–object interaction at 25 FPS.

  8. DreamSwapV: Mask-guided Subject Swapping for Any Customized Video Editing

    cs.CV 2025-08 unverdicted novelty 5.0 of 10

    DreamSwapV performs mask-guided, subject-agnostic subject swapping in videos using multiple conditions, an adaptive mask strategy, and a new benchmark.

Reference graph

Works this paper leans on

81 extracted references · 43 canonical work pages · cited by 8 Pith papers

  1. [1]

    Stable video diffusion: Scaling latent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023. 3

  2. [2]

    Magicpose: Realistic human poses and facial expressions retar- geting with identity-aware diffusion

    Di Chang, Yichun Shi, Quankai Gao, Jessica Fu, Hongyi Xu, Guoxian Song, Qing Yan, Yizhe Zhu, Xiao Yang, and Mohammad Soleymani. Magicpose: Realistic human poses and facial expressions retar- geting with identity-aware diffusion. arXiv preprint arXiv:2311.12052, 2023. 2, 3

  3. [3]

    Virtualmodel: Generating object-id-retentive human-object interac- tion image by diffusion model for e-commerce mar- keting

    Binghui Chen, Chongyang Zhong, Wangmeng Xi- ang, Yifeng Geng, and Xuansong Xie. Virtualmodel: Generating object-id-retentive human-object interac- tion image by diffusion model for e-commerce mar- keting. arXiv preprint arXiv:2405.09985, 2024. 2

  4. [4]

    Demamba: Ai- generated video detection on million-scale genvideo benchmark

    Haoxing Chen, Yan Hong, Zizheng Huang, Zhuoer Xu, Zhangxuan Gu, Yaohui Li, Jun Lan, Huijia Zhu, Jianfu Zhang, Weiqiang Wang, et al. Demamba: Ai- generated video detection on million-scale genvideo benchmark. arXiv preprint arXiv:2405.19707, 2024. 8

  5. [5]

    Multi-subject open-set per- sonalization in video generation

    Tsai-Shien Chen, Aliaksandr Siarohin, Willi Mena- pace, Yuwei Fang, Kwot Sin Lee, Ivan Skorokhodov, Kfir Aberman, Jun-Yan Zhu, Ming-Hsuan Yang, and Sergey Tulyakov. Multi-subject open-set per- sonalization in video generation. arXiv preprint arXiv:2501.06187, 2025. 3

  6. [6]

    Arcface: Additive angular margin loss for deep face recognition

    Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Arcface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4690–4699, 2019. 6

  7. [7]

    Cinema: Coherent multi-subject video generation via mllm- based guidance

    Yufan Deng, Xun Guo, Yizhi Wang, Jacob Zhiyuan Fang, Angtian Wang, Shenghai Yuan, Yiding Yang, Bo Liu, Haibin Huang, and Chongyang Ma. Cinema: Coherent multi-subject video generation via mllm- based guidance. arXiv preprint arXiv:2503.10391 ,

  8. [8]

    Cg-hoi: Contact- guided 3d human-object interaction generation

    Christian Diller and Angela Dai. Cg-hoi: Contact- guided 3d human-object interaction generation. In Proc. Computer Vision and Pattern Recognition (CVPR), IEEE, 2024. 2

Show all 81 references
  1. [9]

    Scaling rectified flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international confer- ence on machi...

  2. [10]

    Re-hold: Video hand object interaction reenactment via adaptive layout-instructed diffusion model

    Yingying Fan, Quanwei Yang, Kaisiyuan Wang, Hang Zhou, Yingying Li, Haocheng Feng, Errui Ding, Yu Wu, and Jingdong Wang. Re-hold: Video hand object interaction reenactment via adaptive layout-instructed diffusion model. CVPR, 2025. 2

  3. [11]

    Humandit: Pose-guided diffusion transformer for long-form human motion video generation

    Qijun Gan, Yi Ren, Chen Zhang, Zhenhui Ye, Pan Xie, Xiang Yin, Zehuan Yuan, Bingyue Peng, and Jianke Zhu. Humandit: Pose-guided diffusion transformer for long-form human motion video generation. arXiv preprint arXiv:2502.04847, 2025. 2, 3

  4. [12]

    Seedream 3.0 techni- cal report

    Yu Gao, Lixue Gong, Qiushan Guo, Xiaoxia Hou, Zhichao Lai, Fanshi Li, Liang Li, Xiaochen Lian, Chao Liao, Liyang Liu, et al. Seedream 3.0 techni- cal report. arXiv preprint arXiv:2504.11346, 2025. 6

  5. [13]

    Imos: Intent-driven full-body motion synthesis for human- object interactions

    Anindita Ghosh, Rishabh Dabral, Vladislav Golyanik, Christian Theobalt, and Philipp Slusallek. Imos: Intent-driven full-body motion synthesis for human- object interactions. In Eurographics, 2023. 2

  6. [14]

    Hu- mans in 4d: Reconstructing and tracking humans with transformers

    Shubham Goel, Georgios Pavlakos, Jathushan Ra- jasegaran, Angjoo Kanazawa, and Jitendra Malik. Hu- mans in 4d: Reconstructing and tracking humans with transformers. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision , pages 14783–14794, 2023. 5

  7. [15]

    Talk-act: Enhance textural-awareness for 2d speaking avatar reenactment with diffusion model

    Jiazhi Guan, Quanwei Yang, Kaisiyuan Wang, Hang Zhou, Shengyi He, Zhiliang Xu, Haocheng Feng, Er- rui Ding, Jingdong Wang, Hongtao Xie, et al. Talk-act: Enhance textural-awareness for 2d speaking avatar reenactment with diffusion model. In SIGGRAPH Asia 2024 Conference Papers ...

  8. [16]

    Mohamed Hassan, Partha Ghosh, Joachim Tesch, Dimitrios Tzionas, and Michael J. Black. Populat- ing 3D scenes by learning human-scene interaction. In Proceedings IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2021. 2

  9. [17]

    Synthesizing physical character-scene interactions

    Mohamed Hassan, Yunrong Guo, Tingwu Wang, Michael Black, Sanja Fidler, and Xue Bin Peng. Synthesizing physical character-scene interactions. In ACM SIGGRAPH 2023 Conference Proceedings , pages 1–9, 2023. 2

  10. [18]

    Id- animator: Zero-shot identity-preserving human video generation

    Xuanhua He, Quande Liu, Shengju Qian, Xin Wang, Tao Hu, Ke Cao, Keyu Yan, and Jie Zhang. Id- animator: Zero-shot identity-preserving human video generation. arXiv preprint arXiv:2404.15275, 2024. 3

  11. [19]

    Interactdiffusion: Inter- action control in text-to-image diffusion models

    Jiun Tian Hoe, Xudong Jiang, Chee Seng Chan, Yap- Peng Tan, and Weipeng Hu. Interactdiffusion: Inter- action control in text-to-image diffusion models. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR) , pages 6180–6189, 2024. 2

  12. [20]

    Hand-object interaction image gener- ation

    Hezhen Hu, Weilun Wang, Wengang Zhou, and Houqiang Li. Hand-object interaction image gener- ation. In NeurIPS, 2022. 2

  13. [21]

    Animate anyone: Consistent and controllable image-to-video synthesis for character animation

    Li Hu. Animate anyone: Consistent and controllable image-to-video synthesis for character animation. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 8153– 8163, 2024. 2, 3, 5

  14. [22]

    Animate anyone 2: High-fidelity character image animation with environment affor- dance

    Li Hu, Guangyuan Wang, Zhen Shen, Xin Gao, Dechao Meng, Lian Zhuo, Peng Zhang, Bang Zhang, and Liefeng Bo. Animate anyone 2: High-fidelity character image animation with environment affor- dance. arXiv preprint arXiv:2502.06145, 2025. 2, 3

  15. [23]

    Hunyuancus- tom: A multimodal-driven architecture for customized video generation

    Teng Hu, Zhentao Yu, Zhengguang Zhou, Sen Liang, Yuan Zhou, Qin Lin, and Qinglin Lu. Hunyuancus- tom: A multimodal-driven architecture for customized video generation. arXiv preprint arXiv:2505.04512 ,

  16. [24]

    Amo sampler: Enhancing text rendering with overshooting

    Xixi Hu, Keyang Xu, Bo Liu, Qiang Liu, and Hongliang Fei. Amo sampler: Enhancing text rendering with overshooting. arXiv preprint arXiv:2411.19415, 2024. 6

  17. [25]

    Personahoi: Effortlessly improving personalized face with human-object interaction gen- eration

    Xinting Hu, Haoran Wang, Jan Eric Lenssen, and Bernt Schiele. Personahoi: Effortlessly improving personalized face with human-object interaction gen- eration. arXiv preprint arXiv:2501.05823, 2025. 2

  18. [26]

    Videomage: Multi-subject and motion cus- tomization of text-to-video diffusion models

    Chi-Pin Huang, Yen-Siang Wu, Hung-Kai Chung, Kai-Po Chang, Fu-En Yang, and Yu-Chiang Frank Wang. Videomage: Multi-subject and motion cus- tomization of text-to-video diffusion models. arXiv preprint arXiv:2503.21781, 2025. 3

  19. [27]

    Hoigpt: Learning long se- quence hand-object interaction with language models

    Mingzhen Huang, Fu-Jen Chu, Bugra Tekin, Kevin J Liang, Haoyu Ma, Weiyao Wang, Xingyu Chen, Pierre Gleize, Hongfei Xue, Siwei Lyu, Kris Kitani, Matt Feiszli, and Hao Tang. Hoigpt: Learning long se- quence hand-object interaction with language models. In IEEE Conference on Comp...

  20. [28]

    Conceptmaster: Multi- concept video customization on diffusion transformer models without test-time tuning

    Yuzhou Huang, Ziyang Yuan, Quande Liu, Qiulin Wang, Xintao Wang, Ruimao Zhang, Pengfei Wan, Di Zhang, and Kun Gai. Conceptmaster: Multi- concept video customization on diffusion transformer models without test-time tuning. arXiv preprint arXiv:2501.04698, 2025. 3

  21. [29]

    Vbench: Comprehensive benchmark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianx- ing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive benchmark suite for video generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and P...

  22. [30]

    Make-your- anchor: A diffusion-based 2d avatar generation frame- work

    Ziyao Huang, Fan Tang, Yong Zhang, Xiaodong Cun, Juan Cao, Jintao Li, and Tong-Yee Lee. Make-your- anchor: A diffusion-based 2d avatar generation frame- work. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6997–7006, 2024. 2, 3

  23. [31]

    Videobooth: Diffusion-based video generation with image prompts

    Yuming Jiang, Tianxing Wu, Shuai Yang, Chenyang Si, Dahua Lin, Yu Qiao, Chen Change Loy, and Ziwei Liu. Videobooth: Diffusion-based video generation with image prompts. 2023. 3

  24. [32]

    Vace: All-in- one video creation and editing

    Zeyinzi Jiang, Zhen Han, Chaojie Mao, Jingfeng Zhang, Yulin Pan, and Yu Liu. Vace: All-in- one video creation and editing. arXiv preprint arXiv:2503.07598, 2025. 3, 6, 7

  25. [33]

    Dreampose: Fashion image-to-video synthesis via stable diffu- sion

    Johanna Karras, Aleksander Holynski, Ting-Chun Wang, and Ira Kemelmacher-Shlizerman. Dreampose: Fashion image-to-video synthesis via stable diffu- sion. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 22623–22633. IEEE,

  26. [34]

    Subject- driven video generation via disentangled identity and motion

    Daneul Kim, Jingxu Zhang, Wonjoon Jin, Sunghyun Cho, Qi Dai, Jaesik Park, and Chong Luo. Subject- driven video generation via disentangled identity and motion. arXiv preprint arXiv:2504.17816, 2025. 3

  27. [35]

    Auto- encoding variational bayes, 2013

    Diederik P Kingma, Max Welling, et al. Auto- encoding variational bayes, 2013. 3

  28. [36]

    Dispose: Disentangling pose guidance for controllable human image animation

    Hongxiang Li, Yaowei Li, Yuhang Yang, Junjie Cao, Zhihong Zhu, Xuxin Cheng, and Long Chen. Dispose: Disentangling pose guidance for controllable human image animation. arXiv preprint arXiv:2412.09349 ,

  29. [37]

    Object motion guided human motion synthesis

    Jiaman Li, Jiajun Wu, and C Karen Liu. Object motion guided human motion synthesis. ACM Trans. Graph., 42(6), 2023. 2

  30. [38]

    Movie weaver: Tuning-free multi-concept video personalization with anchored prompts

    Feng Liang, Haoyu Ma, Zecheng He, Tingbo Hou, Ji Hou, Kunpeng Li, Xiaoliang Dai, Felix Juefei-Xu, Samaneh Azadi, Animesh Sinha, et al. Movie weaver: Tuning-free multi-concept video personalization with anchored prompts. arXiv preprint arXiv:2502.07802,

  31. [39]

    Omnihuman-1: Rethinking the scaling-up of one-stage conditioned human animation models

    Gaojie Lin, Jianwen Jiang, Jiaqi Yang, Zerong Zheng, and Chao Liang. Omnihuman-1: Rethinking the scaling-up of one-stage conditioned human animation models. arXiv preprint arXiv:2502.01061, 2025. 3

  32. [40]

    Flow match- ing for generative modeling

    Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow match- ing for generative modeling. arXiv preprint arXiv:2210.02747, 2022. 6

  33. [41]

    Hoigen- 1m: A large-scale dataset for human-object interaction video generation

    Kun Liu, Qi Liu, Xinchen Liu, Jie Li, Yongdong Zhang, Jiebo Luo, Xiaodong He, and Wu Liu. Hoigen- 1m: A large-scale dataset for human-object interaction video generation. In CVPR, 2025. 2

  34. [42]

    Phan- tom: Subject-consistent video generation via cross- modal alignment

    Lijie Liu, Tianxaing Ma, Bingchuan Li, Zhuowei Chen, Jiawei Liu, Qian He, and Xinglong Wu. Phan- tom: Subject-consistent video generation via cross- modal alignment. arXiv preprint arXiv:2502.11079 ,

  35. [43]

    Grounding dino: Marry- ing dino with grounded pre-training for open-set ob- ject detection

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jian- wei Yang, Hang Su, et al. Grounding dino: Marry- ing dino with grounded pre-training for open-set ob- ject detection. In European Conference on Computer Vision, pages 38–55. Sp...

  36. [44]

    Easyhoi: Unleashing the power of large models for reconstructing hand-object interac- tions in the wild

    Yumeng Liu, Xiaoxiao Long, Zemin Yang, Yuan Liu, Marc Habermann, Christian Theobalt, Yuexin Ma, and Wenping Wang. Easyhoi: Unleashing the power of large models for reconstructing hand-object interac- tions in the wild. arXiv preprint arXiv:2411.14280 ,

  37. [45]

    Dreamactor- m1: Holistic, expressive and robust human image animation with hybrid guidance

    Yuxuan Luo, Zhengkun Rong, Lizhen Wang, Longhao Zhang, Tianshu Hu, and Yongming Zhu. Dreamactor- m1: Holistic, expressive and robust human image animation with hybrid guidance. arXiv preprint arXiv:2504.01724, 2025. 2, 3

  38. [46]

    Follow your pose: Pose-guided text-to-video generation using pose-free videos

    Yue Ma, Yingqing He, Xiaodong Cun, Xintao Wang, Siran Chen, Xiu Li, and Qifeng Chen. Follow your pose: Pose-guided text-to-video generation using pose-free videos. In Proceedings of the AAAI Con- ference on Artificial Intelligence , pages 4117–4125,

  39. [47]

    Mimo: Controllable character video synthesis with spatial decomposed modeling

    Yifang Men, Yuan Yao, Miaomiao Cui, and Liefeng Bo. Mimo: Controllable character video synthesis with spatial decomposed modeling. arXiv preprint arXiv:2409.16160, 2024. 2, 3

  40. [48]

    Maxime Oquab, Timoth ´ee Darcet, Theo Moutakanni, Huy V . V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Russell Howes, Po-Yao Huang, Hu Xu, Vasu Sharma, Shang-Wen Li, Wojciech Galuba, Mike Rabbat, Mido Assran, Nicol...

  41. [49]

    Georgios Pavlakos, Vasileios Choutas, Nima Ghor- bani, Timo Bolkart, Ahmed A. A. Osman, Dimitrios Tzionas, and Michael J. Black. Expressive body cap- ture: 3d hands, face, and body from a single image. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)...

  42. [50]

    Reconstructing hands in 3D with transformers

    Georgios Pavlakos, Dandan Shan, Ilija Radosavovic, Angjoo Kanazawa, David Fouhey, and Jitendra Ma- lik. Reconstructing hands in 3D with transformers. In CVPR, 2024. 5

  43. [51]

    Scalable diffu- sion models with transformers

    William Peebles and Saining Xie. Scalable diffu- sion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vi- sion, pages 4195–4205, 2023. 2, 3

  44. [52]

    Controlnext: Pow- erful and efficient control for image and video gener- ation

    Bohao Peng, Jian Wang, Yuechen Zhang, Wenbo Li, Ming-Chang Yang, and Jiaya Jia. Controlnext: Pow- erful and efficient control for image and video gener- ation. arXiv preprint arXiv:2408.06070, 2024. 2, 3

  45. [53]

    Hoi-diff: Text- driven synthesis of 3d human-object interactions using diffusion models

    Xiaogang Peng, Yiming Xie, Zizhao Wu, Varun Jam- pani, Deqing Sun, and Huaizu Jiang. Hoi-diff: Text- driven synthesis of 3d human-object interactions using diffusion models. arXiv preprint arXiv:2312.06553 ,

  46. [54]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sas- try, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, ...

  47. [55]

    Sam 2: Segment anything in images and videos

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman R ¨adle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.00714 , 2024. 5

  48. [56]

    Seaweed-7b: Cost-effective training of video generation foundation model

    Team Seawead, Ceyuan Yang, Zhijie Lin, Yang Zhao, Shanchuan Lin, Zhibei Ma, Haoyuan Guo, Hao Chen, Lu Qi, Sen Wang, et al. Seaweed-7b: Cost-effective training of video generation foundation model. arXiv preprint arXiv:2504.08685, 2025. 3, 6

  49. [57]

    Seed1.5-vl technical report

    ByteDance Seed Team. Seed1.5-vl technical report. arXiv preprint arXiv:2505.07062, 2025. 5

  50. [58]

    Stableanimator: High-quality identity-preserving human image anima- tion

    Shuyuan Tu, Zhen Xing, Xintong Han, Zhi-Qi Cheng, Qi Dai, Chong Luo, and Zuxuan Wu. Stableanimator: High-quality identity-preserving human image anima- tion. arXiv preprint arXiv:2411.17697, 2024. 2, 3

  51. [59]

    Wan: Open and advanced large-scale video generative models

    Team Wan, Ang Wang, Baole Ai, Bin Wen, Chao- jie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haim- ing Zhao, Jianxiao Yang, Jianyuan Zeng, Jiayu Wang, Jingfeng Zhang, Jingren Zhou, Jinkai Wang, Jixuan Chen, Kai Zhu, Kang Zhao, Keyu Yan, Lianghua Huang, Mengyang Feng, Ningyi Zhang, P...

  52. [60]

    Faceverse: a fine-grained and detail-controllable 3d face morphable model from a hybrid dataset

    Lizhen Wang, Zhiyuan Chen, Tao Yu, Chenguang Ma, Liang Li, and Yebin Liu. Faceverse: a fine-grained and detail-controllable 3d face morphable model from a hybrid dataset. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 20333–20342...

  53. [61]

    Disco: Disentan- gled control for realistic human dance generation

    Tan Wang, Linjie Li, Kevin Lin, Yuanhao Zhai, Chung-Ching Lin, Zhengyuan Yang, Hanwang Zhang, Zicheng Liu, and Lijuan Wang. Disco: Disentan- gled control for realistic human dance generation. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognitio...

  54. [62]

    Unianimate: Taming unified video diffusion models for consistent human image anima- tion

    Xiang Wang, Shiwei Zhang, Changxin Gao, Jiayu Wang, Xiaoqiang Zhou, Yingya Zhang, Luxin Yan, and Nong Sang. Unianimate: Taming unified video diffusion models for consistent human image anima- tion. arXiv preprint arXiv:2406.01188, 2024. 2, 3, 6, 7

  55. [63]

    Dreamvideo: Composing your dream videos with customized subject and motion

    Yujie Wei, Shiwei Zhang, Zhiwu Qing, Hangjie Yuan, Zhiheng Liu, Yu Liu, Yingya Zhang, Jingren Zhou, and Hongming Shan. Dreamvideo: Composing your dream videos with customized subject and motion. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recogni...

  56. [64]

    InterDiff: Generating 3d human-object in- teractions with physics-informed diffusion

    Sirui Xu, Zhengyuan Li, Yu-Xiong Wang, and Liang- Yan Gui. InterDiff: Generating 3d human-object in- teractions with physics-informed diffusion. In ICCV,

  57. [65]

    Intermimic: Towards universal whole- body control for physics-based human-object interac- tions

    Sirui Xu, Hung Yu Ling, Yu-Xiong Wang, and Liangyan Gui. Intermimic: Towards universal whole- body control for physics-based human-object interac- tions. In CVPR, 2025. 2

  58. [66]

    Anchorcrafter: Ani- mate cyber-anchors selling your products via human- object interacting video generation

    Ziyi Xu, Ziyao Huang, Juan Cao, Yong Zhang, Xi- aodong Cun, Qing Shuai, Yuchen Wang, Linchao Bao, Jintao Li, and Fan Tang. Anchorcrafter: Ani- mate cyber-anchors selling your products via human- object interacting video generation. arXiv preprint arXiv:2411.17383, 2024. 2, 6, 7

  59. [67]

    Magicanimate: Temporally con- sistent human image animation using diffusion model

    Zhongcong Xu, Jianfeng Zhang, Jun Hao Liew, Han- shu Yan, Jia-Wei Liu, Chenxu Zhang, Jiashi Feng, and Mike Zheng Shou. Magicanimate: Temporally con- sistent human image animation using diffusion model. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern ...

  60. [68]

    Hoi-swap: Swapping objects in videos with hand-object interaction awareness

    Zihui Xue, Mi Luo, Chen Changan, and Kristen Grau- man. Hoi-swap: Swapping objects in videos with hand-object interaction awareness. arXiv preprint arXiv:2406.07754, 2024. 2

  61. [69]

    Diffusion-guided reconstruction of ev- eryday hand-object interaction clips

    Yufei Ye, Abhinav Hebbar, Poorvi Gupta, and Shub- ham Tulsiani. Diffusion-guided reconstruction of ev- eryday hand-object interaction clips. In ICCV, 2023. 2

  62. [70]

    Affordance diffusion: Synthesiz- ing hand-object interactions

    Yufei Ye, Xueting Li, Abhinav Gupta, Shalini De Mello, Stan Birchfield, Jiaming Song, Shubham Tul- siani, and Sifei Liu. Affordance diffusion: Synthesiz- ing hand-object interactions. In CVPR, 2023. 2

  63. [71]

    Identity-preserving text-to-video genera- tion by frequency decomposition

    Shenghai Yuan, Jinfa Huang, Xianyi He, Yun- yuan Ge, Yujun Shi, Liuhan Chen, Jiebo Luo, and Li Yuan. Identity-preserving text-to-video genera- tion by frequency decomposition. arXiv preprint arXiv:2411.17440, 2024. 3

  64. [72]

    GraspXL: Generating grasp- ing motions for diverse objects at scale

    Hui Zhang, Sammy Christen, Zicong Fan, Otmar Hilliges, and Jie Song. GraspXL: Generating grasp- ing motions for diverse objects at scale. In European Conference on Computer Vision (ECCV), 2024. 2

  65. [73]

    Scaling in-the-wild training for diffusion-based illu- mination harmonization and editing by imposing con- sistent light transport

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Scaling in-the-wild training for diffusion-based illu- mination harmonization and editing by imposing con- sistent light transport. In The Thirteenth International Conference on Learning Representations, 2025. 6

  66. [74]

    Hoidiffusion: Gen- erating realistic 3d hand-object interaction data

    Mengqi Zhang, Yang Fu, Zheng Ding, Sifei Liu, Zhuowen Tu, and Xiaolong Wang. Hoidiffusion: Gen- erating realistic 3d hand-object interaction data. arXiv preprint arXiv:2403.12011, 2024. 2

  67. [75]

    Place: Proximity learning of articula- tion and contact in 3d environments

    Siwei Zhang, Yan Zhang, Qianli Ma, Michael J Black, and Siyu Tang. Place: Proximity learning of articula- tion and contact in 3d environments. In 2020 Inter- national Conference on 3D Vision (3DV), pages 642–

  68. [76]

    Generating 3d peo- ple in scenes without people

    Yan Zhang, Mohamed Hassan, Heiko Neumann, Michael J Black, and Siyu Tang. Generating 3d peo- ple in scenes without people. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6194–6204, 2020. 2

  69. [77]

    Mim- icmotion: High-quality human motion video gener- ation with confidence-aware pose guidance

    Yuang Zhang, Jiaxi Gu, Li-Wen Wang, Han Wang, Junqi Cheng, Yuefeng Zhu, and Fangyuan Zou. Mim- icmotion: High-quality human motion video gener- ation with confidence-aware pose guidance. arXiv preprint arXiv:2406.19680, 2024. 2, 3

  70. [78]

    Taste-rob: Advancing video generation of task-oriented hand- object interaction for generalizable robotic manipula- tion

    Hongxiang Zhao, Xingchen Liu, Mutian Xu, Yiming Hao, Weikai Chen, and Xiaoguang Han. Taste-rob: Advancing video generation of task-oriented hand- object interaction for generalizable robotic manipula- tion. In CVPR, 2025. 2

  71. [79]

    Concat-id: Towards universal identity-preserving video synthesis

    Yong Zhong, Zhuoyi Yang, Jiayan Teng, Xiaotao Gu, and Chongxuan Li. Concat-id: Towards universal identity-preserving video synthesis. arXiv preprint arXiv:2503.14151, 2025. 3

  72. [80]

    Dormant: Defending against pose-driven human image animation

    Jiachen Zhou, Mingsi Wang, Tianlin Li, Guozhu Meng, and Kai Chen. Dormant: Defending against pose-driven human image animation. arXiv preprint arXiv:2409.14424, 2024. 8

  73. [81]

    Champ: Controllable and consistent human image animation with 3d parametric guidance

    Shenhao Zhu, Junming Leo Chen, Zuozhuo Dai, Zi- long Dong, Yinghui Xu, Xun Cao, Yao Yao, Hao Zhu, and Siyu Zhu. Champ: Controllable and consistent human image animation with 3d parametric guidance. In European Conference on Computer Vision , pages 145–162. Springer, 2024. 2, 3

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.