Pith. sign in

REVIEW 4 major objections 6 minor 42 references

FastInit: Fast Noise Initialization for Temporally Consistent Video Generation

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A learned single-pass noise predictor replaces costly iterative refinement in video diffusion, matching temporal-consistency gains at roughly baseline speed.

desk verdict First learned noise-prediction network for video, with a real 5x speedup; the main problem is a missing loss function that makes the training claim unverifiable. read the letter →

arxiv 2506.16119 v1 pith:JY4ZYG2R submitted 2025-06-19 cs.CV cs.AI

Chengyu Bai , Yuming Li , Zhongyu Zhao , Jintao Chen , Peidong Jia , Qi She , Ming Lu , Shanghang Zhang This is my paper · ORCID
classification cs.CVcs.AI
keywords videogenerationdiffusionmodelsnoiseinitializationtemporalconsistencyTuckerdecompositiontext-to-videoinference-timeefficiencypredictionnetwork
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FastInit claims that the temporal-consistency gains obtained by iteratively refining the initial noise in video diffusion models can be distilled into a learned single forward pass. The paper introduces a Video Noise Prediction Network that maps a random Gaussian latent and a text prompt to a refined latent, trained on a new Prompt-Noise Dataset of roughly 150,000 prompt–noise–refined-noise triples per backbone. If the claim holds, video diffusion models can accept the predicted refined noise at inference time and match or exceed the quality of iterative refinement while running at nearly the same speed as the unmodified sampler, removing the roughly five-times overhead. The paper reports consistent gains in temporal-consistency and quality metrics over standard Gaussian initialization on two text-to-video backbones and two public benchmarks.

What carries the argument

The load-bearing object is the differentiable Tucker decomposition of the 4-D noise latent, which separates it into a compact core tensor $G$ and mode-specific orthonormal bases $U^{(c)}, U^{(t)}, U^{(h)}, U^{(w)}$ capturing channel, temporal, height, and width structure. The Tucker-Based Noise Filter uses softmax-gated per-mode reweighting and a learnable core mask to suppress high-rank stochastic components while preserving the low-frequency spatio-temporal content; the Global Contextual Residual Module restores details lost by the low-rank projection and injects the text prompt as an additive conditioning signal. Together these two branches define the single-pass mapping that the paper argues substitutes for iterative noise refinement.

What would settle it

Run FastInit on a third publicly available video diffusion backbone whose latents were not part of the training dataset, using the same prompts and sampling steps, and compare the generated videos to standard Gaussian initialization and to iterative-refinement-initialized videos on a temporal-consistency metric such as UMT-FVD or Temporal Flickering. If the FastInit-initialized videos are not consistently better than the standard baseline, or show artifacts on that backbone, the model-agnostic plug-and-play claim would be refuted.

Watch

Extended reading notes

Core claim

The paper's central claim is that the useful signal in the initial latent is its low-frequency structure, and that this structure can be predicted rather than searched. Formally, FastInit learns the mapping $\Phi:(z_{\mathrm{rand}}, p) \mapsto z_{\mathrm{refined}}$, where $z_{\mathrm{rand}} \sim \mathcal{N}(0,1)$ is the random $C \times T \times H \times W$ latent and $p$ is the prompt embedding. The network's Tucker-Based Noise Filter approximates the latent by a Tucker decomposition $z \approx G \times_1 U^{(c)} \times_2 U^{(t)} \times_3 U^{(h)} \times_4 U^{(w)}$, reweights the orthonormal factor matrices with per-mode MLP gates, and applies a learnable core mask; a Global Contextual Residual Module built on a lightweight transformer with interleaved 3-D convolution and self-attention adds back fine detail and prompt-aware semantics, blended by a learned scalar $\beta$. Trained on the Prompt-Noise Dataset, the network claims to replace iterative low-frequency refinement with one prediction step, so that an off-the-shelf diffusion sampler produces temporally consistent videos.

Load-bearing premise

FastInit is said to be model-agnostic, but VNPNet is trained only on refined noise produced by two specific video diffusion backbones, so the claim that it improves arbitrary video diffusion models rests on an untested assumption of cross-model transfer.

Editorial extensions

If this is right

  • Inference cost returns to near baseline: in the paper's timing experiments, FastInit runs in about 42 seconds versus about 211 seconds for the five-iteration refinement baseline.
  • FastInit can be dropped into an existing video diffusion model at inference time without retraining or changing the backbone weights.
  • The learned predictor improves temporal-consistency metrics and visual quality over standard Gaussian initialization on both tested backbones, and it matches or beats the iterative refinement baseline on most reported scores.
  • The released Prompt-Noise Dataset provides a reusable resource for future noise-initialization methods.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The model-agnostic claim is only supported on two backbones; a natural test is whether the trained predictor transfers to a video diffusion transformer or a different latent video model, since the mapping may overfit to the latent statistics of the two training backbones.
  • Because VNPNet distills the iterative optimizer's output into a feed-forward network, FastInit can be viewed as amortized inference-time scaling; extending this idea, one could train the predictor against a differentiable temporal-consistency metric instead of distilling a fixed iterative baseline, potentially improving on the teacher.
  • The Tucker-based factorization may also be useful outside video diffusion: the same structure-preserving denoising of latent noise could be adapted to image noise prediction or to other generative models that sample from a high-dimensional Gaussian prior.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes FastInit, a method that replaces FreeInit's iterative noise refinement during video-diffusion inference with a single forward pass through a Video Noise Prediction Network (VNPNet). VNPNet takes random Gaussian noise and a text embedding as input and outputs a refined noise latent, which is then fed into an off-the-shelf text-to-video diffusion sampler. The refined noise is produced by a Tucker-based noise filter plus a global contextual residual module (Eq. (2)). The network is trained on a newly constructed Prompt-Noise Dataset (PNData) whose labels are FreeInit-refined noise latents from AnimateDiff and ModelScopeT2V. Experiments report quality metrics on ChronoMagic-Bench and VBench for these two backbones, together with an inference-time table showing roughly 5x speedup over FreeInit (42.47s vs 210.96s for 25 steps). The paper claims consistent improvements in visual quality and temporal consistency, model-agnostic plug-and-play applicability, and no need to retrain the diffusion backbone.

Significance. If the central claim holds, FastInit would be a practically valuable inference-time module: it removes the dominant computational overhead of FreeInit while retaining most of its temporal-consistency benefit. The efficiency result in Table 6 is concrete and supports the main speed claim. The architectural elements (differentiable Tucker decomposition with learnable reweighting, plus a UniFormer-based residual module) are well motivated, and the construction of a 150k-pair noise dataset is a useful resource. However, the paper's independent contribution is best understood as a learned distillation of FreeInit rather than a new noise-refinement principle, and the current manuscript does not provide the training objective, error bars, or cross-model evidence needed to substantiate the broader claims. The manuscript is not yet reproducible as written.

major comments (4)
  1. [4.1 and Eq. (2)] The training objective for VNPNet is never specified. Eq. (2) defines the predicted refined noise, but no loss function (e.g., mean squared error between the prediction and the FreeInit-refined label in PNData) is given anywhere in Sections 3, 4, or the appendices. Without a loss, the claim that VNPNet 'learns the mapping' in Eq. (1) is not verifiable and the reported numbers cannot be reproduced. Please provide the loss equation and training details, and also reconcile the '100 iterations' with batch size 144 (which processes only 14,400 samples versus roughly 150,000 training pairs) and the 8-A100 statement in Section 4.1 with the 4-A100 statement in Appendix A.3.
  2. [3.1 and Table 4] Because VNPNet is trained on FreeInit's refined noise labels (Section 3.1, PNData), any temporal-consistency improvement over Standard DDIM is at least partly inherited from the teacher. The paper should frame FastInit explicitly as a distillation of FreeInit and compare it against the teacher on the same prompts. Table 4 does not support the claim that FastInit 'achieves higher scores than FreeInit itself across multiple metrics' in a uniform sense: for AnimateDiff, FastInit has lower Subject Consistency (96.36 vs 96.45) and lower Temporal Flickering (98.73 vs 98.74) than FreeInit Iter5. Paired per-prompt differences with confidence intervals are needed before such a claim can be made.
  3. [Tables 1, 3, and 5] No error bars or significance tests are reported for any quality metric, and several differences are small enough to be within noise (e.g., ModelScope UMTScore 3.06 vs 3.01, AnimateDiff Motion Smoothness 98.23 vs 98.04). Some results go in the opposite direction: Table 3 shows FastInit below FreeInit on ModelScope Motion Smoothness (96.43 vs 96.67), and Table 5 shows the Tucker variant's UMT-FVD (233.84) is worse than the AnimateDiff baseline (232.45) and the SVD variant (233.63). The claims of 'consistent improvements' and 'best or comparable across all metrics' are therefore overstated. Please report variances across seeds and test prompts, or soften the claims to match the evidence.
  4. [1, 3.2, and 4.3] The model-agnostic and plug-and-play claims are not tested. VNPNet is trained only on AnimateDiff and ModelScopeT2V, and all experiments evaluate only those same two backbones. A central advertised advantage is generalization across models, so at least one held-out video diffusion backbone should be tested (e.g., VideoCrafter or an open DiT-based model) to show that the learned noise mapping is not overfit to the two training backbones. If such an experiment is not feasible, the model-agnostic claim should be removed or substantially qualified.
minor comments (6)
  1. [Eq. (4)] The notation '||U(i)||ℓ2 indicates element-wise row normalization' is unclear; please define precisely what is normalized (rows of the factor matrix, their ℓ2 norms, or the entries).
  2. [Eq. (2) and surrounding text] The symbols 'bzrefined' and 'bzrand' appear in the text after Eq. (2) and should be typeset consistently as 'z_refined' and 'z_rand'.
  3. [3.4 and Fig. 3] The module is called 'Global Contextual Residual Module' in Section 3.2 and the caption of Fig. 3, but 'Global Contextual Refinement Module' in Section 3.4; please unify the name.
  4. [5 (Ablation Studies)] The sentence 'the Tucker variant demonstrates the best or comparable performance across all metrics' is contradicted by Table 5, where Tucker has the worst UMT-FVD (233.84) among the compared variants except MLP; please revise the wording.
  5. [Table 4] The row 'FreeInit Iter1' for AnimateDiff appears to reproduce the Standard DDIM row from Table 1, but this is not stated; clarify whether FreeInit Iter1 is equivalent to the standard sampler.
  6. [Related Work] 'WAN' should be spelled 'Wan' for consistency with the reference list, and there are a few other small typos (e.g., 'denosing' in Fig. 2 caption).

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: FastInit is a disclosed distillation of FreeInit with an independent speed contribution.

full rationale

The derivation chain is PNData -> VNPNet -> refined noise -> temporally consistent videos. PNData pairs random Gaussian noise with noise optimized by FreeInit [32], so the consistency signal in the training labels originates from an external, published algorithm rather than FastInit itself. This is a teacher-student setup, and the paper explicitly acknowledges that the model is trained using data produced by FreeInit (Section 4.3). The claimed improvement over standard DDIM is inherited from FreeInit if the network fits its targets, but that is not a definitional equivalence: FastInit is evaluated on external benchmarks (ChronoMagic-Bench, VBench) against FreeInit and IV-Mixed Sampler, and the independent contribution is the one-forward-pass approximation (Eq. (1), Eq. (2)) that removes the 5x iterative overhead. No load-bearing argument rests on self-citation: FreeInit and GoldenNoise share no authors with the present paper, and no uniqueness theorem is imported. The absence of an explicit loss function in Section 4.1 is an incompleteness, not circularity, because no equation can be shown to reduce to the training targets by construction. The training-steps versus dataset-size discrepancy and GPU-count mismatch also affect reproducibility but do not establish a circular derivation. Therefore no circular step forces the central result from its inputs.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The reported gains rest on assumptions the paper imports rather than proves: FreeInit-refined noise is a valid training target, low-frequency noise components carry temporal structure, and the learned mapping transfers beyond the two backbones used to build PNData. The only explicit constants introduced by the authors are the Tucker ranks, the residual weight β, and the number of FreeInit iterations used to create labels; all are chosen by hand or learned and none are given error bars.

free parameters (4)
  • Tucker decomposition ranks [Rc, Rt, Rh, Rw] = [4, 8, 32, 32]
    Fixed for all experiments after preliminary grid search (Appendix A.1); controls 7:1 compression and the trade-off between detail preservation and filtering.
  • Residual weight β = learned, value not reported
    Learned scalar in Eq. (2) that balances the Tucker-filtered noise and the UniFormer global residual; its value is not reported.
  • FreeInit iterations for PNData labels = 5
    Target refined noise is generated by 5 FreeInit iterations (Section 3.1, Table 2); this choice sets the ceiling for what the student network can imitate.
  • Training duration = 100 iterations
    VNPNet is trained for 100 iterations on 8 A100 GPUs (Section 4.1); with ~150k pairs per backbone this is likely 100 epochs, but the text is ambiguous.
assumptions (5)
  • domain assumption Low-frequency components of the initial noise determine temporal consistency in video diffusion.
    Adopted from FreeInit in Section 1 and used to justify the Tucker filter; the paper provides no independent derivation or measurement.
  • domain assumption FreeInit's refined noise is a good supervisory signal for temporally consistent noise.
    PNData labels are FreeInit outputs after 5 iterations (Section 3.1); the paper does not show that these targets are metric-optimal or that a network trained on them can exceed the teacher reliably.
  • domain assumption Tucker modes of a Gaussian noise tensor carry meaningful spatial and temporal structure.
    Section 3.3 interprets U^t as motion and U^h/U^w as layout, but the input is random noise and no evidence links Tucker factors of noise to scene semantics.
  • domain assumption VBench and ChronoMagic-Bench metrics measure the temporal consistency the method targets.
    Section 4.3 uses these benchmark scores as ground truth without user studies, confidence intervals, or calibration against human judgment.
  • standard math Tucker decomposition and reconstruction are differentiable and stable for per-sample noise tensors.
    The TBNF relies on differentiable Tucker factorization (Eq. (3)-(6)); standard tensor algebra is assumed, and no citation for the decomposition algorithm is given.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FastInit: Fast Noise Initialization for Temporally Consistent Video Generation." pith.science (2026). https://pith.science/paper/JY4ZYG2R

@misc{pith2026250616119,
  author       = {Pith},
  title        = {Pith review of: FastInit: Fast Noise Initialization for Temporally Consistent Video Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JY4ZYG2R}},
  note         = {Machine review of arXiv:2506.16119}
}
read the original abstract

Video generation has made significant strides with the development of diffusion models; however, achieving high temporal consistency remains a challenging task. Recently, FreeInit identified a training-inference gap and introduced a method to iteratively refine the initial noise during inference. However, iterative refinement significantly increases the computational cost associated with video generation. In this paper, we introduce FastInit, a fast noise initialization method that eliminates the need for iterative refinement. FastInit learns a Video Noise Prediction Network (VNPNet) that takes random noise and a text prompt as input, generating refined noise in a single forward pass. Therefore, FastInit greatly enhances the efficiency of video generation while achieving high temporal consistency across frames. To train the VNPNet, we create a large-scale dataset consisting of pairs of text prompts, random noise, and refined noise. Extensive experiments with various text-to-video models show that our method consistently improves the quality and temporal consistency of the generated videos. FastInit not only provides a substantial improvement in video generation but also offers a practical solution that can be applied directly during inference. The code and dataset will be released.

Figures

Figures reproduced from arXiv: 2506.16119 by the authors.

Figure 1
Figure 1. Example video generation results with noises initialized by the proposed FastInit. We propose a method called FastInit, which is a fast noise-initialization scheme that eliminates the need for iterative refinement. Given a random noise and a text prompt, FastInit generates a refined noise in a single forward pass. The resulting videos show significantly improved temporal consistency and visual fidelity, and the meth… view at source ↗
Figure 2
Figure 2. The illustration of various pipelines. Standard diffusion begins with random noise but produces low quality. Refine initial noise improves quality through slow iterative denoising. Complex Sampler uses an image-diffusion backbone for iterative enhancement at denosing step, resulting in low speed and high memory consumption. FastInit employs a prediction network to generate refined noise directly. and visual quality … view at source ↗
Figure 3
Figure 3. The illustration of FastInit. (a) illustrates the overall architecture of VNPNet, showcasing its two main components: the Tucker-Based Noise Filter (TBNF) and the Global Contextual Residual Module (GCRM). In part (b), the TBNF acts as a structure-preserving denoiser by highlighting the principal spatio-temporal components while reducing high-rank stochastic noise. (c) demonstrates that during inference, NPNet can be… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 16 canonical work pages

  1. [1]

    A noise is worth diffusion guidance

    Donghoon Ahn, Jiwon Kang, Sanghyun Lee, Jaewon Min, Minjae Kim, Wooseok Jang, Hyoung- won Cho, Sayak Paul, SeonHwa Kim, Eunju Cha, et al. A noise is worth diffusion guidance. arXiv preprint arXiv:2412.03895, 2024

    arXiv 2024

  2. [2]

    Frozen in time: A joint video and image encoder for end-to-end retrieval

    Max Bain, Arsha Nagrani, Gül Varol, and Andrew Zisserman. Frozen in time: A joint video and image encoder for end-to-end retrieval. InIEEE International Conference on Computer Vision, 2021

    2021

  3. [3]

    Vidu: a highly consistent, dynamic and skilled text-to-video generator with diffusion models.arXiv preprint arXiv:2405.04233, 2024

    Fan Bao, Chendong Xiang, Gang Yue, Guande He, Hongzhou Zhu, Kaiwen Zheng, Min Zhao, Shilong Liu, Yaole Wang, and Jun Zhu. Vidu: a highly consistent, dynamic and skilled text-to-video generator with diffusion models.arXiv preprint arXiv:2405.04233, 2024

    arXiv 2024

  4. [4]

    Stable video diffusion: Scaling latent video diffusion models to large datasets.arXiv preprint arXiv:2311.15127, 2023

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Do- minik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets.arXiv preprint arXiv:2311.15127, 2023

    arXiv 2023

  5. [5]

    Attend-and-excite: Attention-based semantic guidance for text-to-image diffusion models.ACM transactions on Graphics (TOG), 42(4):1–10, 2023

    Hila Chefer, Yuval Alaluf, Yael Vinker, Lior Wolf, and Daniel Cohen-Or. Attend-and-excite: Attention-based semantic guidance for text-to-image diffusion models.ACM transactions on Graphics (TOG), 42(4):1–10, 2023

    2023

  6. [6]

    Videocrafter2: Overcoming data limitations for high-quality video diffusion models

    Haoxin Chen, Yong Zhang, Xiaodong Cun, Menghan Xia, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter2: Overcoming data limitations for high-quality video diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7310–7320, 2024

    2024

  7. [7]

    Preserve your own correlation: A noise prior for video diffusion models

    Songwei Ge, Seungjun Nah, Guilin Liu, Tyler Poon, Andrew Tao, Bryan Catanzaro, David Jacobs, Jia-Bin Huang, Ming-Yu Liu, and Yogesh Balaji. Preserve your own correlation: A noise prior for video diffusion models. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 22930–22941, 2023

    work page 2023

  8. [8]

    Factorizing text-to-video generation by explicit image conditioning

    Rohit Girdhar, Mannat Singh, Andrew Brown, Quentin Duval, Samaneh Azadi, Sai Saketh Rambhatla, Akbar Shah, Xi Yin, Devi Parikh, and Ishan Misra. Factorizing text-to-video generation by explicit image conditioning. InEuropean Conference on Computer Vision, pages 205–224. Springer, 2024

    2024

Show all 42 references
  1. [9]

    Reuse and diffuse: Iterative denoising for text-to-video generation.arXiv preprint arXiv:2309.03549, 2023

    Jiaxi Gu, Shicong Wang, Haoyu Zhao, Tianyi Lu, Xing Zhang, Zuxuan Wu, Songcen Xu, Wei Zhang, Yu-Gang Jiang, and Hang Xu. Reuse and diffuse: Iterative denoising for text-to-video generation.arXiv preprint arXiv:2309.03549, 2023

    2023 arXiv

  2. [10]

    I4vgen: Image as stepping stone for text-to-video generation.arXiv preprint arXiv:2406.02230, 2024

    Xiefan Guo, Jinlin Liu, Miaomiao Cui, and Di Huang. I4vgen: Image as stepping stone for text-to-video generation.arXiv preprint arXiv:2406.02230, 2024

    2024 arXiv

  3. [11]

    Initno: Boosting text-to-image diffusion models via initial noise optimization

    Xiefan Guo, Jinlin Liu, Miaomiao Cui, Jiankai Li, Hongyu Yang, and Di Huang. Initno: Boosting text-to-image diffusion models via initial noise optimization. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9380–9389, 2024

    2024

  4. [12]

    Animatediff: Animate your personalized text-to-image diffusion models without specific tuning.arXiv preprint arXiv:2307.04725, 2023

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text-to-image diffusion models without specific tuning.arXiv preprint arXiv:2307.04725, 2023

    2023 arXiv

  5. [13]

    Kingma, Ben Poole, Mohammad Norouzi, David J

    Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P. Kingma, Ben Poole, Mohammad Norouzi, David J. Fleet, and Tim Salimans. Imagen video: High definition video generation with diffusion models, 2022

    2022

  6. [14]

    Cogvideo: Large-scale pretraining for text-to-video generation via transformers.arXiv preprint arXiv:2205.15868, 2022

    Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang. Cogvideo: Large-scale pretraining for text-to-video generation via transformers.arXiv preprint arXiv:2205.15868, 2022

    2022 arXiv

  7. [15]

    Hunyuanvideo: A systematic framework for large video generative models, 2025

    Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, Kathrina Wu, Qin Lin, Junkun Yuan, Yanxin Long, Aladdin Wang, Andong Wang, Changlin Li, Duojun Huang, Fang Yang, Hao Tan, Hongmei Wang, Jacob Song, Jiawang Bai, J...

    2025

  8. [16]

    Training-free guidance in text-to-video generation via multimodal planning and structured noise initialization

    Jialu Li, Shoubin Yu, Han Lin, Jaemin Cho, Jaehong Yoon, and Mohit Bansal. Training-free guidance in text-to-video generation via multimodal planning and structured noise initialization. arXiv preprint arXiv:2504.08641, 2025

    2025 arXiv

  9. [17]

    Uniformer: Unified transformer for efficient spatiotemporal representation learning, 2022

    Kunchang Li, Yali Wang, Peng Gao, Guanglu Song, Yu Liu, Hongsheng Li, and Yu Qiao. Uniformer: Unified transformer for efficient spatiotemporal representation learning, 2022

    2022

  10. [18]

    All seeds are not equal: Enhancing compositional text-to-image generation with reliable random seeds, 2025

    Shuangqi Li, Hieu Le, Jingyi Xu, and Mathieu Salzmann. All seeds are not equal: Enhancing compositional text-to-image generation with reliable random seeds, 2025

    2025

  11. [19]

    Tuning-free noise rectification for high fidelity image-to-video generation.arXiv preprint arXiv:2403.02827, 2024

    Weijie Li, Litong Gong, Yiran Zhu, Fanda Fan, Biao Wang, Tiezheng Ge, and Bo Zheng. Tuning-free noise rectification for high fidelity image-to-video generation.arXiv preprint arXiv:2403.02827, 2024

    2024 arXiv

  12. [20]

    Freelong: Training-free long video generation with spectralblend temporal attention.arXiv preprint arXiv:2407.19918, 2024

    Yu Lu, Yuanzhi Liang, Linchao Zhu, and Yi Yang. Freelong: Training-free long video generation with spectralblend temporal attention.arXiv preprint arXiv:2407.19918, 2024

    2024 arXiv

  13. [21]

    Inference-time scaling for diffusion models beyond scaling denoising steps.arXiv preprint arXiv:2501.09732, 2025

    Nanye Ma, Shangyuan Tong, Haolin Jia, Hexiang Hu, Yu-Chuan Su, Mingda Zhang, Xuan Yang, Yandong Li, Tommi Jaakkola, Xuhui Jia, et al. Inference-time scaling for diffusion models beyond scaling denoising steps.arXiv preprint arXiv:2501.09732, 2025

    2025 arXiv

  14. [22]

    Sdedit: Guided image synthesis and editing with stochastic differential equations, 2022

    Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations, 2022

    2022

  15. [23]

    Freenoise: Tuning-free longer video diffusion via noise rescheduling, 2023

    Haonan Qiu, Menghan Xia, Yong Zhang, Yingqing He, Xintao Wang, Ying Shan, and Ziwei Liu. Freenoise: Tuning-free longer video diffusion via noise rescheduling, 2023

    2023

  16. [24]

    Iv-mixed sampler: Lever- aging image diffusion models for enhanced video synthesis.arXiv preprint arXiv:2410.04171, 2024

    Shitong Shao, Zikai Zhou, Lichen Bai, Haoyi Xiong, and Zeke Xie. Iv-mixed sampler: Lever- aging image diffusion models for enhanced video synthesis.arXiv preprint arXiv:2410.04171, 2024

    2024 arXiv

  17. [25]

    Make-a-video: Text-to-video generation without text-video data, 2022

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, Devi Parikh, Sonal Gupta, and Yaniv Taigman. Make-a-video: Text-to-video generation without text-video data, 2022

    2022

  18. [26]

    Scaling llm test-time compute optimally can be more effective than scaling model parameters, 2024

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling llm test-time compute optimally can be more effective than scaling model parameters, 2024

    2024

  19. [27]

    End-to-end diffusion latent optimization improves classifier guidance

    Bram Wallace, Akash Gokul, Stefano Ermon, and Nikhil Naik. End-to-end diffusion latent optimization improves classifier guidance. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 7280–7290, 2023

    2023

  20. [28]

    Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025

    Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, et al. Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025

    2025 arXiv

  21. [29]

    Modelscope text-to-video technical report, 2023

    Jiuniu Wang, Hangjie Yuan, Dayou Chen, Yingya Zhang, Xiang Wang, and Shiwei Zhang. Modelscope text-to-video technical report, 2023

    2023

  22. [30]

    The silent prompt: Initial noise as implicit guidance for goal-driven image generation.arXiv preprint arXiv:2412.05101, 2024

    Ruoyu Wang, Huayang Huang, Ye Zhu, Olga Russakovsky, and Yu Wu. The silent prompt: Initial noise as implicit guidance for goal-driven image generation.arXiv preprint arXiv:2412.05101, 2024

    2024 arXiv

  23. [31]

    Cono: Consistency noise injection for tuning-free long video diffusion.arXiv preprint arXiv:2406.05082, 2024

    Xingrui Wang, Xin Li, and Zhibo Chen. Cono: Consistency noise injection for tuning-free long video diffusion.arXiv preprint arXiv:2406.05082, 2024

    2024 arXiv

  24. [32]

    Freeinit: Bridging initialization gap in video diffusion models

    Tianxing Wu, Chenyang Si, Yuming Jiang, Ziqi Huang, and Ziwei Liu. Freeinit: Bridging initialization gap in video diffusion models. InEuropean Conference on Computer Vision, pages 378–394. Springer, 2024. 11

    2024

  25. [33]

    Msr-vtt: A large video description dataset for bridging video and language

    Jun Xu, Tao Mei, Ting Yao, and Yong Rui. Msr-vtt: A large video description dataset for bridging video and language. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 5288–5296, 2016

    2016

  26. [34]

    Good seed makes a good crop: Discovering secret seeds in text-to-image diffusion models, 2025

    Katherine Xu, Lingzhi Zhang, and Jianbo Shi. Good seed makes a good crop: Discovering secret seeds in text-to-image diffusion models, 2025

    2025

  27. [35]

    Noise calibration: Plug-and-play content-preserving video enhancement using pre-trained video diffusion models

    Qinyu Yang, Haoxin Chen, Yong Zhang, Menghan Xia, Xiaodong Cun, Zhixun Su, and Ying Shan. Noise calibration: Plug-and-play content-preserving video enhancement using pre-trained video diffusion models. InEuropean Conference on Computer Vision, pages 307–326. Springer, 2024

    2024

  28. [36]

    Cogvideox: Text-to-video diffusion models with an expert transformer, 2025

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, Da Yin, Yuxuan Zhang, Weihan Wang, Yean Cheng, Bin Xu, Xiaotao Gu, Yuxiao Dong, and Jie Tang. Cogvideox: Text-to-video diffusion models with an ex...

    2025

  29. [37]

    Chronomagic-bench: A benchmark for metamorphic evaluation of text-to-time-lapse video generation, 2024

    Shenghai Yuan, Jinfa Huang, Yongqi Xu, Yaoyang Liu, Shaofeng Zhang, Yujun Shi, Ruijie Zhu, Xinhua Cheng, Jiebo Luo, and Li Yuan. Chronomagic-bench: A benchmark for metamorphic evaluation of text-to-time-lapse video generation, 2024

    2024

  30. [38]

    Riflex: A free lunch for length extrapolation in video diffusion transformers.arXiv preprint arXiv:2502.15894, 2025

    Min Zhao, Guande He, Yixiao Chen, Hongzhou Zhu, Chongxuan Li, and Jun Zhu. Riflex: A free lunch for length extrapolation in video diffusion transformers.arXiv preprint arXiv:2502.15894, 2025

    2025 arXiv

  31. [39]

    Vbench-2.0: Advancing video generation benchmark suite for intrinsic faithfulness, 2025

    Dian Zheng, Ziqi Huang, Hongbo Liu, Kai Zou, Yinan He, Fan Zhang, Yuanhan Zhang, Jingwen He, Wei-Shi Zheng, Yu Qiao, and Ziwei Liu. Vbench-2.0: Advancing video generation benchmark suite for intrinsic faithfulness, 2025

    2025

  32. [40]

    Open-sora: Democratizing efficient video production for all

    Zangwei Zheng, Xiangyu Peng, Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all. arXiv preprint arXiv:2412.20404, 2024

    2024 arXiv

  33. [41]

    Magicvideo: Efficient video generation with latent diffusion models, 2023

    Daquan Zhou, Weimin Wang, Hanshu Yan, Weiwei Lv, Yizhe Zhu, and Jiashi Feng. Magicvideo: Efficient video generation with latent diffusion models, 2023

    2023

  34. [42]

    Golden noise for diffusion models: A learning framework.arXiv preprint arXiv:2411.09502, 2024

    Zikai Zhou, Shitong Shao, Lichen Bai, Zhiqiang Xu, Bo Han, and Zeke Xie. Golden noise for diffusion models: A learning framework.arXiv preprint arXiv:2411.09502, 2024. 12 A Technical Appendices and Supplementary Material A.1 Implementation Details The Tucker-Based Noise Filter...

    2024 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.