Pith. sign in

REVIEW 4 major objections 6 minor 36 references

VFX Creator: Animated Visual Effect Generation with Controllable Diffusion Transformer

T0 review · 4 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that visual effects can be generated as image animation from a still image, a text prompt, an instance mask, and start/end timestamps, and supports the claim with a new dataset and a diffusion-transformer framework.

desk verdict New VFX dataset is the real contribution; the temporal control metric is likely circular and the SOTA claim needs better evidence. read the letter →

arxiv 2502.05979 v4 pith:D67XIEJ6 submitted 2025-02-09 cs.CV

Xinyu Liu , Ailing Zeng , Wei Xue , Harry Yang , Wenhan Luo , Qifeng Liu , Yike Guo This is my paper · ORCID
classification cs.CV
keywords visualeffectsgenerationcontrollablevideodiffusiontransformerspatialcontroltemporallow-rankadaptationVFXdatasetimageanimation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that visual effects can be turned into an image-animation problem: instead of hand-authoring an effect, a user supplies a still image, a text prompt like 'explode it,' a mask marking the target object, and start/end timestamps, and a video diffusion transformer generates the effect on that object and within that time window. To make this learnable, the authors build Open-VFX, a dataset of 675 videos covering 15 effect categories with text, masks, and timestamps, and they design VFX Creator, which adapts a frozen video transformer with low-rank updates plus two control modules. On their test set the system beats three open-source baselines on FID-VID, FVD, and dynamic degree, and reports a temporal IoU around 0.84 on selected effects, with human raters preferring it over or tying it with the commercial platforms that supplied the source videos. If the results hold, VFX production could move from expensive per-shot manual work to promptable, instance- and rhythm-controllable generation.

What carries the argument

The load-bearing machinery is a pair of control adapters bolted onto a frozen video diffusion transformer. For space, a mask-control branch runs in parallel with the main network and injects per-frame instance masks through a zero-initialized convolution, so the effect is confined to the user-selected object. For time, a timestamp encoder maps normalized start/end frames into token embeddings that are concatenated with the text-prompt tokens and fed to the transformer's cross-attention, so the requested rhythm is coupled to the prompt. Both adapters are trained as low-rank updates, which keeps the number of new parameters small and lets the model learn from tens of videos per effect; the paper's ablation shows the text-space timestamp injection beats timestep-embedding injection.

What would settle it

Take a random sample of Open-VFX videos, have people mark the true start/end frames and affected regions by hand, and recompute the paper's temporal IoU and timestamp errors against those manual labels; a large drop would show the automatic labels inflated the reported accuracy.

Watch

Extended reading notes

Core claim

VFX Creator recasts animated visual effects as an image-animation task. The central claim is that a video diffusion transformer, lightly adapted, can take a static reference image, a text prompt naming one of 15 effects, a mask selecting the instance to affect, and normalized start/end timestamps, and output a video in which the named effect occurs on the chosen instance within the requested interval. The paper argues the design is data-efficient: with a frozen base network, trainable low-rank adapters, a mask-conditioned control branch, and timestamp tokens fused into the text embeddings, tens of videos per effect suffice. On the Open-VFX test set the system reports FID-VID 29.92 and FVD 752.95, better than three open-source baselines; temporal IoU averages 0.84 across three effects; and a user study places it above or on par with the commercial platforms whose videos were used as pseudo ground truth.

Load-bearing premise

The whole evaluation leans on automatically produced labels for when each effect starts and ends and which pixels it covers; if those labels are biased, the training target and the timing metric share the same bias and the reported temporal accuracy could be flattering.

Editorial extensions

If this is right

  • A user can point at an instance in a single image, type an effect such as 'melt it' or 'squish it,' and give rough start/end times, and the model will produce a video that honors both the region and the timing.
  • Because training needs only tens of videos per effect, new effect types can be added without collecting large datasets; the 10-shot runs already give balanced quality, and 40-shot lowers FVD on some effects.
  • The new temporal metric (frame-level error, second-level error, and temporal IoU) gives a standard way to compare how precisely a generator obeys requested effect timing.
  • Training one unified model across all 15 effects lowers quality compared with per-effect training; the paper attributes this to effects confusing each other and suggests mixture-of-experts as a future fix.
  • Open-VFX gives the community a benchmark with masks and timestamps for 15 VFX categories, which the paper positions as a foundation for later VFX-generation work.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, we would test whether the timestamp-token mechanism can sync effects to an external clock, such as music beats or shot cuts; the paper does not report that experiment.
  • Beyond the paper, the mask-control branch looks portable to other video diffusion transformers, but the paper only demonstrates it on its chosen baseline; transfer is our speculation.
  • Beyond the paper, because the source videos come from two commercial generators, Open-VFX may carry their stylistic biases; collecting effects from real film footage would reveal whether the model generalizes beyond synthetic VFX aesthetics.
  • Beyond the paper, the temporal metric could be stressed by sampling start/end pairs over the full range instead of the restricted ranges used in evaluation, which would expose whether extreme timings degrade control.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces Open-VFX, a dataset of 675 VFX videos collected from Pika and PixVerse across 15 effect categories, annotated with text prompts, SAM2-derived instance masks, and Co-Tracker-derived start/end timestamps. It also proposes VFX Creator, a controllable video diffusion framework built on CogVideoX, adding a spatial mask-control branch (ControlNet-style) and two temporal-conditioning strategies (timestamp tokens in text space or temporal masks in timestep space) via LoRA adapters. Experiments report FID-VID, FVD, Dynamic Degree, temporal error metrics (Ef, Es, TIoU), qualitative comparisons, and user studies, claiming state-of-the-art spatial and temporal controllability on the Open-VFX test set.

Significance. If the claims are substantiated, the Open-VFX dataset would be a useful new resource for controllable VFX generation, and the LoRA-based, few-shot adaptation approach is practically appealing because it lowers the data barrier for customizing video diffusion models. The paper also introduces a new task formulation (VFX generation as image animation with spatial/temporal control) and proposes a dedicated temporal-control metric. However, the current experimental evidence does not yet establish the headline claims: the evaluation design has several load-bearing weaknesses, including possible circularity in the temporal metric, uncontrolled baselines, and very small test sets without statistical support.

major comments (4)
  1. [§5.2, Eq. (4)-(5)] The temporal-control metrics are not fully defined, and their validity is questionable. TIoU is introduced in the text but no equation or computation procedure is provided. More importantly, the manuscript never states how the predicted start/end timestamps (t_hat_start,i, t_hat_end,i) are obtained from generated videos. If the same Co-Tracker displacement pipeline used to create ground-truth labels in §3.3 is applied to generated videos, then the ground truth and predictions share the same motion-onset/offset bias, so the metric may reward reproducing Co-Tracker's biases rather than perceptually correct timing. The paper reports no manual verification, inter-annotator agreement, or human sanity check for these labels. Because Table 2 covers only 3 of 15 effects, the headline temporal-controllability claim is not supported by the evidence as presented.
  2. [§5.3, Table 1] The comparison protocol is not controlled. VFX Creator is fine-tuned on the Open-VFX training split, while DynamiCrafter, CogVideoX, and LTX-Video are evaluated off-the-shelf without any fine-tuning on the dataset. The reported FID-VID/FVD/Dynamic Degree gains may therefore reflect task-specific adaptation rather than an architectural advantage of the proposed method. Additionally, the 'pseudo ground-truth Pika or PixVerse' baseline comes from the same commercial platforms used to source the training videos, so it is not a clean external baseline for generalization. The authors should either fine-tune the open-source baselines under the same protocol or clearly frame the comparison as 'zero-shot baselines vs. fine-tuned method,' and ideally evaluate on independently produced VFX videos.
  3. [§5.1, Table 3] The quantitative evaluation rests on very small test sets (5 videos per effect, per §5.1) and no error bars, confidence intervals, or significance tests are reported. The shot-number ablation in Table 3 is also internally inconsistent with the claim that increasing the number of shots generally improves performance: for 'Ta-da it,' the 10-shot FVD (2861.18) is much worse than the 1-shot value (1432.40), and for 'Explode it,' the 10-shot FVD (2829.00) is worse than 1-shot (2667.72). These non-monotonic results and the lack of variance estimates mean the data-efficiency claim is not established.
  4. [§5.4, §5.5, Fig. 6 and Fig. 8] Spatial controllability is only demonstrated qualitatively; there is no quantitative metric for whether the effect occurs on the specified instance and not elsewhere. The reader cannot judge from still frames alone whether mask guidance is precise. The authors should add a spatial-accuracy metric, such as mask IoU between the region where the effect is applied in the generated video and the target mask, computed over the test set. Similarly, the temporal-control evaluation in Table 2 should be extended beyond three effects to all categories to support the general claim.
minor comments (6)
  1. [§3.3] The description of Co-Tracker-based timestamp labeling is underspecified: 'motion is deemed to begin when the coordinates start to shift' needs a concrete displacement threshold and a policy for handling tracker noise and multi-point aggregation.
  2. [§5.4 and Fig. 7] There are typos: 'Defalte it' should be 'Deflate it' in §5.4, and 'VF Creator' in Fig. 7 should be 'VFX Creator'.
  3. [Appendix A] The appendix heading 'DEFINATIONS' should be 'DEFINITIONS'.
  4. [§5.2] In Eqs. (4) and (5), the variables t_start,i and t_end,i are said to be 'normalized' but no normalization range is specified; also, the relationship between frame-level error and second-level error via FPS is stated but not derived with explicit units.
  5. [§4.2.1] The construction of the 'temporal mask' used in Strategy I is not formalized; the authors should state how the binary moving/static mask is derived from the start and end timestamps and how it is aligned with the training frames after random shifting.
  6. [Introduction] The paper claims the 'first high-quality VFX video dataset' but does not discuss existing related datasets (e.g., video object segmentation or effect datasets); a brief comparison would help position the contribution.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the derivation chain is self-contained and the temporal-control metric, while sharing auto-generated labels with training, does not reduce by construction to a fitted input.

full rationale

The paper's central claims are empirical: VFX Creator is trained on Open-VFX and evaluated against open-source baselines (DynamiCrafter, LTX-Video, CogVideoX) using standard metrics (FID-VID, FVD, Dynamic Degree) plus a user study. Those evaluations are external to the fitted parameters, so they do not reduce to the paper's inputs. The only plausible circularity concern is the temporal-control evaluation (Section 5.2, Eqs. 4-5): ground-truth start/end timestamps are produced by Co-Tracker (Section 3.3), and the same Co-Tracker/SAM2 annotations are used as training conditions. However, the paper never states that predicted segments are obtained by running Co-Tracker on generated videos; the metric is defined as a difference between predicted and ground-truth segments without specifying the predictor. Without that specification, one cannot exhibit the equation-level equivalence required for a circularity finding. If the implementation did reuse Co-Tracker as the predictor, this would be a label-bias/validity weakness, not a derivation that is equivalent to its input by construction. No load-bearing self-citation or imported uniqueness theorem appears. The limitation section candidly discusses dataset breadth and unified-training quality, further indicating the claims are not forced by the paper's own definitions.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central claims rest on the reliability of automated annotations, the representativeness of scraped commercial videos, and the validity of standard video quality metrics for the VFX domain. No new physical or conceptual entities are postulated. Hyperparameters such as LoRA rank (128) and 3000 training steps are standard choices, not fitted constants.

assumptions (4)
  • domain assumption Co-Tracker point displacements accurately indicate the onset and offset of visual effects motion.
    Section 3.3 relies on Co-Tracker for timestamp labels; if it tracks unrelated motion, timestamps are wrong.
  • domain assumption SAM2 mask sequences correctly isolate the animated instance throughout the effect.
    Section 3.3 and Section 4.2.2 use SAM2 masks for spatial conditions and evaluation; inaccurate masks would weaken spatial control claims.
  • domain assumption Pika and PixVerse videos are representative of the VFX generation task and legally usable for redistribution.
    The dataset is sourced from these commercial platforms and used as ground truth in Section 5.3.
  • domain assumption FID-VID and FVD are valid measures of VFX video quality for this comparison.
    Section 5.2 uses these metrics without validating their sensitivity to effect-specific artifacts.

how reviews work

0 comments
Cite this review

Pith. "Pith review of VFX Creator: Animated Visual Effect Generation with Controllable Diffusion Transformer." pith.science (2026). https://pith.science/paper/D67XIEJ6

@misc{pith2026250205979,
  author       = {Pith},
  title        = {Pith review of: VFX Creator: Animated Visual Effect Generation with Controllable Diffusion Transformer},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/D67XIEJ6}},
  note         = {Machine review of arXiv:2502.05979}
}
read the original abstract

Crafting magic and illusions is one of the most thrilling aspects of filmmaking, with visual effects (VFX) serving as the powerhouse behind unforgettable cinematic experiences. While recent advances in generative artificial intelligence have driven progress in generic image and video synthesis, the domain of controllable VFX generation remains relatively underexplored. In this work, we propose a novel paradigm for animated VFX generation as image animation, where dynamic effects are generated from user-friendly textual descriptions and static reference images. Our work makes two primary contributions: (i) Open-VFX, the first high-quality VFX video dataset spanning 15 diverse effect categories, annotated with textual descriptions, instance segmentation masks for spatial conditioning, and start-end timestamps for temporal control. (ii) VFX Creator, a simple yet effective controllable VFX generation framework based on a Video Diffusion Transformer. The model incorporates a spatial and temporal controllable LoRA adapter, requiring minimal training videos. Specifically, a plug-and-play mask control module enables instance-level spatial manipulation, while tokenized start-end motion timestamps embedded in the diffusion process, alongside the text encoder, allow precise temporal control over effect timing and pace. Extensive experiments on the Open-VFX test set demonstrate the superiority of the proposed system in generating realistic and dynamic effects, achieving state-of-the-art performance and generalization ability in both spatial and temporal controllability. Furthermore, we introduce a specialized metric to evaluate the precision of temporal control. By bridging traditional VFX techniques with generative approaches, VFX Creator unlocks new possibilities for efficient and high-quality video effect generation, making advanced VFX accessible to a broader audience.

Figures

Figures reproduced from arXiv: 2502.05979 by the authors.

Figure 1
Figure 1. VFX Creator is an efficient framework based on a Video Diffusion Transformer, enabling spatial and temporal control for visual effect (VFX) video generation. With minimal training data, a plug-and-play mask control module allows precise instance-level manipulation, while the integration of tokenized start-end motion timestamps with text space provides fine-grained temporal control over the VFX rhythm. Crafting magic… view at source ↗
Figure 2
Figure 2. Overview of our proposed Open-VFX Dataset. (a) demonstrates diverse input inference images in the dataset, including humans, animals, objects, and [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. More examples of our Open-VFX dataset, including 10 VFXs and diverse reference images. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Pipeline of VFX Creator. We introduce two novel modules: (a) Spa￾tial Controlled LoRA Adapter. This module integrates a mask-conditioned ControlNet with LoRA, injecting mask sequences into the model to enable instance-level spatial manipulation. (b) Temporal Controlled…
Figure 5
Figure 5. Figure 5: Qualitative comparisons of VFX video generation on two different visual effects between our method, DynamiCrafter, LTX-Video, CogVideoX-5B, and [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Qualitative results of spatial controllable VFX video generation of our method on two different visual effects. Users can precisely specify the animated [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: User Study. Our VF Creator demonstrates superior human prefer￾ence compared to other methods 6 LIMITATION Despite the introduction of our pioneering visual effects video dataset, Open-VFX, alongside the development of VFX Creator for spatial-temporal controllable effec…
Figure 8
Figure 8. Figure 8: More spatial controlled VFX generation results of our method on two different visual effects. [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 9
Figure 9. Figure 9: More qualitative comparison results of VFX video generation on two different visual effects between our method, DynamiCrafter, LTX-Video, CogVideoX [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: More temporal controlled VFX generation results of our method. [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

36 extracted references · 13 canonical work pages

  1. [1]

    Adobe. n.d.. From pyrotechnics to prosthetics: A guide to special effects in movies. https://www.adobe.com/creativecloud/video/discover/a-guide-to-special- effects-in-movies.html Accessed: 2023-01-17. Yogesh Balaji, Martin Renqiang Min, Bing Bai, Rama Chellappa, and Hans Peter Graf

    work page 2023

  2. [4]

    arXiv preprint arXiv:2310.19512 (2023)

    Videocrafter1: Open diffusion models for high-quality video generation. arXiv preprint arXiv:2310.19512 (2023). Zixiang Chen, Yihe Deng, Yue Wu, Quanquan Gu, and Yuanzhi Li

    arXiv 2023

  3. [5]

    Comparison results of mix and single training for visual effect generation. Metric Method Cake-ify Crumble Crush Decapitate Deflate Dissolve Explode Eye-pop harley Inflate Levitate Melt Squish Ta-da Venom FID-VID↓ Single 54.48 65.11 46.71 43.76 103.90 76.14 50.97 34.87 94.62 86.14 35.12 63.37 44.35 54.73 117.90 Mix 67.22 65.06 44.52 44.52 111.28 87.00 84....

    work page 1946

  4. [6]

    In SIGGRAPH Asia 2024 Confer- ence Papers

    Camera Settings as Tokens: Modeling Photography on Latent Diffusion Models. In SIGGRAPH Asia 2024 Confer- ence Papers. 1–11. Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai

    work page 2024

  5. [7]

    arXiv preprint arXiv:2307.04725 (2023)

    Animatediff: Animate your personalized text- to-image diffusion models without specific tuning. arXiv preprint arXiv:2307.04725 (2023). Yoav HaCohen, Nisan Chiprut, Benny Brazowski, Daniel Shalem, Dudu Moshe, Eitan Richardson, Eran Levin, Guy Shiran, Nir Zabari, Ori Gordon, et al

    arXiv 2023

  6. [8]

    arXiv preprint arXiv:2501.00103 (2024)

    LTX-Video: Realtime Video Latent Diffusion. arXiv preprint arXiv:2501.00103 (2024). Jonathan Ho, Ajay Jain, and Pieter Abbeel

    arXiv 2024

  7. [11]

    CoTracker3: Simpler and Better Point Tracking by Pseudo-Labelling Real Videos. In Proc. arXiv:2410.11831. Diederik P Kingma

    arXiv

  8. [13]

    https://kling.kuaishou.com/z Accessed: 2025-01-19

    Keling. https://kling.kuaishou.com/z Accessed: 2025-01-19. Guojun Lei, Chi Wang, Hong Li, Rong Zhang, Yikai Wang, and Weiwei Xu

    work page 2025

Show all 36 references
  1. [14]

    Animateanything: Consistent and controllable animation for video generation.arXiv preprint arXiv:2411.10836 (2024). Zhimin Li, Jianwei Zhang, Qin Lin, Jiangfeng Xiong, Yanxin Long, Xinchi Deng, Ying- fang Zhang, Xingchao Liu, Minbin Huang, Zedong Xiao, Dayou Chen, Jiajun He, J...

    2024 arXiv

  2. [15]

    Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Trans- former with Fine-Grained Chinese Understanding. arXiv:2405.08748 [cs.CV] https://arxiv.org/abs/2405.08748 Bin Lin, Yunyang Ge, Xinhua Cheng, Zongjian Li, Bin Zhu, Shaodong Wang, Xianyi He, Yang Ye, Shenghai Yuan, Liuhan...

    arXiv

  3. [16]

    arXiv preprint arXiv:2412.00131 (2024)

    Open-sora plan: Open-source large video generation model. arXiv preprint arXiv:2412.00131 (2024). I Loshchilov

    2024 arXiv

  4. [18]

    arXiv preprint arXiv:2312.04966 (2023)

    Customizing motion in text-to-video diffusion models. arXiv preprint arXiv:2312.04966 (2023). Niranjan D Narvekar and Lina J Karam

    2023 arXiv

  5. [20]

    arXiv preprint arXiv:2112.10741 (2021)

    Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:2112.10741 (2021). Wenqi Ouyang, Yi Dong, Lei Yang, Jianlou Si, and Xingang Pan

    2021 arXiv

  6. [21]

    In SIGGRAPH Asia 2024 Conference Papers

    I2VEdit: First- Frame-Guided Video Editing via Image-to-Video Diffusion Models. In SIGGRAPH Asia 2024 Conference Papers . 1–11. William Peebles and Saining Xie

    2024

  7. [22]

    https://www.pexels.com/ Accessed: 2024-01-19

    Free Stock Photos. https://www.pexels.com/ Accessed: 2024-01-19. Pika

    2024

  8. [23]

    https://pika.art/ Accessed: 2025-01-11

    Pika: A platform for creative AI art . https://pika.art/ Accessed: 2025-01-11. Pixverse

    2025

  9. [24]

    https://app

    Pixverse: AI-powered Image and Video Editing Platform . https://app. pixverse.ai/ Accessed: 2025-01-11. Adam Polyak, Amit Zohar, Andrew Brown, Andros Tjandra, Animesh Sinha, Ann Lee, Apoorv Vyas, Bowen Shi, Chih-Yao Ma, Ching-Yao Chuang, et al

    2025

  10. [25]

    arXiv preprint arXiv:2410.13720 (2024)

    Movie gen: A cast of media foundation models. arXiv preprint arXiv:2410.13720 (2024). Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu

    2024 arXiv

  11. [26]

    Exploring the limits of transfer learning with a unified text-to-text transformer.Journal of machine learning research 21, 140 (2020), 1–67. Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura G...

    2020

  12. [27]

    arXiv:2408.00714 [cs.CV] https://arxiv.org/abs/2408.00714 Yixuan Ren, Yang Zhou, Jimei Yang, Jing Shi, Difan Liu, Feng Liu, Mingi Kwon, and Abhinav Shrivastava

    SAM 2: Segment Anything in Images and Videos. arXiv:2408.00714 [cs.CV] https://arxiv.org/abs/2408.00714 Yixuan Ren, Yang Zhou, Jimei Yang, Jing Shi, Difan Liu, Feng Liu, Mingi Kwon, and Abhinav Shrivastava

    arXiv

  13. [28]

    arXiv preprint arXiv:2402.14780 (2024)

    Customize-a-video: One-shot motion customization of text-to-video diffusion models. arXiv preprint arXiv:2402.14780 (2024). Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer

    2024 arXiv

  14. [29]

    arXiv:2112.10752 [cs.CV] https://arxiv.org/abs/2112.10752 Ruizhi Shao, Youxin Pang, Zerong Zheng, Jingxiang Sun, and Yebin Liu

    High-Resolution Image Synthesis with Latent Diffusion Models. arXiv:2112.10752 [cs.CV] https://arxiv.org/abs/2112.10752 Ruizhi Shao, Youxin Pang, Zerong Zheng, Jingxiang Sun, and Yebin Liu

    arXiv

  15. [30]

    arXiv preprint arXiv:2405.17405 (2024)

    Hu- man4DiT: Free-view Human Video Generation with 4D Diffusion Transformer. arXiv preprint arXiv:2405.17405 (2024). Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli

    2024 arXiv

  16. [31]

    arXiv preprint arXiv:2011.13456 (2020)

    Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456 (2020). Genmo Team

    2020 arXiv

  17. [33]

    In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Dreamvideo: Composing your dream videos with customized subject and motion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 6537–6549. Jianzong Wu, Xiangtai Li, Yanhong Zeng, Jiangning Zhang, Qianyu Zhou, Yining Li, Yunhai Tong, and Kai C...

    2024 arXiv

  18. [34]

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala

    Cogvideox: Text- to-video diffusion models with an expert transformer.arXiv preprint arXiv:2408.06072 (2024). Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. 2023a. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conferen...

    2024 arXiv

  19. [35]

    Ta-da it

    Visual Effect Types and Corresponding Definitions in the Open VFX Dataset. Types of VFX Definition Cake-ify it Transform the subject into hyper-realistic prop cakes. Crumble it Break apart the subjects into fragments. Crush it Apply a hydraulic press to flatten the subject. De...

    2018

  20. [2011]

    IEEE Transactions on Image Processing 20, 9 (2011), 2678–2683

    A no-reference image blur metric based on the cumulative probability of blur detection (CPBD). IEEE Transactions on Image Processing 20, 9 (2011), 2678–2683. Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen

    2011

  21. [2013]

    arXiv preprint arXiv:1312.6114 (2013)

    Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013). Kuaishou

    2013 arXiv

  22. [2017]

    arXiv preprint arXiv:1711.05101 (2017)

    Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017). Xin Ma, Yaohui Wang, Gengyun Jia, Xinyuan Chen, Ziwei Liu, Yuan-Fang Li, Cunjian Chen, and Yu Qiao. 2024b. Latte: Latent diffusion transformer for video generation. arXiv preprint arXiv:2401.03048 (...

    2017 arXiv

  23. [2018]

    arXiv preprint arXiv:1812.01717 (2018)

    Towards accurate generative models of video: A new metric & challenges. arXiv preprint arXiv:1812.01717 (2018). Tan Wang, Linjie Li, Kevin Lin, Yuanhao Zhai, Chung-Ching Lin, Zhengyuan Yang, Hanwang Zhang, Zicheng Liu, and Lijuan Wang. 2024a. Disco: Disentangled control for re...

    2018 arXiv

  24. [2020]

    Advances in neural information processing systems 33 (2020), 6840–6851

    Denoising diffusion probabilistic models. Advances in neural information processing systems 33 (2020), 6840–6851. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    2020

  25. [2021]

    arXiv preprint arXiv:2106.09685 (2021)

    Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685 (2021). Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al

    2021 arXiv

  26. [2022]

    arXiv:2208.02813 [cs.LG] https://arxiv.org/abs/2208.02813 I-Sheng Fang, Yue-Hua Han, and Jun-Cheng Chen

    Towards Understanding Mixture of Experts in Deep Learning. arXiv:2208.02813 [cs.LG] https://arxiv.org/abs/2208.02813 I-Sheng Fang, Yue-Hua Han, and Jun-Cheng Chen

    arXiv

  27. [2023]

    arXiv preprint arXiv:2311.15127 (2023)

    Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127 (2023). Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Ng, Ricky Wang, and A...

    2023 arXiv

  28. [2024]

    Video generation models as world simulators. (2024). https: //openai.com/research/video-generation-models-as-world-simulators Anastasia Chabanova

    2024

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.