{"id":"d875f5ae-3e8d-4ee0-9e0f-8ec82bcf97e2","arxiv_id":"2502.03496","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"FreqPrior refines the initial noise of video diffusion models with a frequency filter designed so the refined noise keeps a nearly standard Gaussian distribution, improving VBench quality and semantic scores and cutting inference time by roughly 23%.","lead":"This paper introduces FreqPrior, a way to initialize the noise for text-to-video diffusion models by filtering it in the frequency domain while keeping the noise distribution almost exactly standard Gaussian, fixing a variance-loss problem in the prior FreeInit method. It reports the best VBench scores on three open video models and about 23% less inference time than FreeInit.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The theoretical guarantee in Eq. (10) rests on Assumption 1 that z_noise is standard Gaussian, which the deployed partial-sampling pipeline does not satisfy; the covariance-error table therefore does not establish that the actual refined noise is nearly Gaussian.","rationale":"The paper makes a genuine contribution: it identifies a variance-decay mechanism in FreeInit and constructs a filtering scheme whose covariance error is analytically driven to machine zero under its assumptions. The algebra in Appendices B and C is internally consistent, and Table 1's near-zero errors for FreqPrior are consistent with Q=0 for Hermitian-symmetric low-pass filters. The strongest claim, however, is not merely that the covariance is close to I but that the refined noise closely approximates N(0,I), and that inference requires Gaussianity of the refined noise. Gaussianity is inherited only from Assumption 1. In the deployed pipeline, z_noise is a weighted combination of a DDIM-generated latent z_t and the initial noise epsilon; z_t is a deterministic nonlinear function of the previous refined noise and is correlated with epsilon. At t=321 the z_t weight is small, so the approximation may be benign, but the paper provides no measurement of how much non-Gaussianity survives the refinement. The reader's conditional verdict already captures this: the experiments are plausible and consistent across three models, the theory is internally consistent, but the theory is not tightly coupled to the algorithm as run. I would not change the verdict; the concrete test above would settle whether the Assumption 1 violation is negligible or whether the theoretical claim needs to be weakened to a heuristic justification.","tokens_in":21715,"tokens_out":20579,"duration_ms":190322,"concrete_test":"Run FreqPrior's actual partial-sampling pipeline with the released code (same scheduler, t=321, two refinement iterations) on a public text-to-video model, and collect z_noise samples from Eq. (1) as well as the final refined noise z. Compute a distributional distance, e.g. energy distance or MMD with a Gaussian kernel, between z_noise and N(0,I) and between the refined z and N(0,I), using random low-dimensional projections for tractability. Compare with the same distance for FreeInit's refined noise and for a true Gaussian prior. If the refined z distance is statistically indistinguishable from a Gaussian baseline, the Assumption 1 violation is negligible; if it is comparable to or larger than FreeInit's deviation, then covariance error alone does not transfer the theoretical guarantee to the deployed algorithm.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.3 (Assumption 1) assumes z_noise ~ N(0,I), and Appendix B.2 uses Gaussianity to conclude that the refined noise z is Gaussian with covariance I - 2cos^2(theta)/(1+cos^2(theta)) Q^2. In Algorithm 1, however, z_noise = sqrt(alpha_bar_T/alpha_bar_t) z_t + sqrt(1 - alpha_bar_T/alpha_bar_t) epsilon, where z_t is the DDIM partial-sampling output and is correlated with epsilon through the previous refinement iterations. For t=321 and a standard linear schedule, sqrt(alpha_bar_T/alpha_bar_t) is small, roughly 0.06-0.1, so the violation may be mild, but it is not zero and z_t is not independent of the added Gaussian noise. Consequently the linear transformation in Eqs. (34)-(37) maps a non-Gaussian input through the filter, and the resulting z is not exactly Gaussian; its covariance alone no longer characterizes the distance to N(0,I). Table 1 and Eq. (10) are computed under the assumption and therefore do not by themselves prove that the deployed FreqPrior prior is nearly Gaussian. The empirical VBench gains remain plausible, but the paper's central theoretical claim is conditional on an assumption that the actual algorithm violates.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces FreqPrior, a noise-initialization strategy for text-to-video diffusion models. The method iteratively refines initial Gaussian noise by (i) partially denoising with DDIM to an intermediate timestep, (ii) mixing that latent with fresh Gaussian noise in a prescribed ratio, (iii) filtering in the 3D Fourier domain with a low-pass mask M and a complementary high-pass mask (1-M^2)^{0.5}, and (iv) post-processing real/imaginary parts to form the next prior. The authors derive the distribution of the refined noise under Assumption 1 (that the mixed latent z_noise is standard Gaussian), prove a bound on its covariance error relative to FreeInit, and report improved VBench scores on VideoCrafter, ModelScope, and AnimateDiff with about 23% inference-time savings.","tokens_in":21945,"tokens_out":7154,"duration_ms":71845,"significance":"If the theoretical guarantee held for the actual deployed pipeline, this would be a valuable contribution: a principled frequency-filtering noise prior that preserves low-frequency semantics while enriching high-frequency detail and stays close to a standard Gaussian. The algebraic derivations in Appendices B and C are self-contained, the covariance-error bound in Eq. (10) is a clean result, and the numerical table is consistent with the formulas. The practical partial-sampling strategy is well motivated. However, the central theoretical claim is conditional on an assumption that the algorithm itself violates, and the empirical hyperparameters were tuned on the same benchmark used for the headline scores. The method remains plausible, but the paper's main justification for 'closely approximates a standard Gaussian distribution' is not yet established for the actual algorithm.","major_comments":[{"comment":"Assumption 1 states that z_noise follows N(0,I), but the actual pipeline computes z_noise = sqrt(alpha_bar_T/alpha_bar_t) * z_t + sqrt(1 - alpha_bar_T/alpha_bar_t) * epsilon, where z_t is the DDIM partial-sampling output. The latent z_t is a deterministic function of the previous refined noise (and hence of previous epsilons), not an independent standard Gaussian, and it is generally correlated with epsilon through the refinement loop. All distributional results, including Eq. (53), Table 1, and the bound Eq. (10), are derived under this Gaussianity assumption; for a non-Gaussian z_noise, covariance error alone does not characterize the distance to N(0,I). The authors should either (i) prove a quantitative bound on the non-Gaussianity of z_noise (e.g., in total variation distance) as a function of sqrt(alpha_bar_T/alpha_bar_t), or (ii) restrict the theoretical claim to the idealized full-diffusion setting and explicitly present the partial-sampling variant as an empirically validated heuristic.","section":"Sec. 3.3, Eq. (1), Algorithm 1"},{"comment":"The hyperparameters cos theta (0.7 for VideoCrafter, 0.8 for ModelScope and AnimateDiff) and t (321) are selected via ablations on VBench, and the final scores in Table 2 are measured on the same VBench benchmark. This makes the empirical comparison partially circular: the reported gains may contain an optimistic-bias component from tuning on the evaluation set. The paper should either reserve a separate validation split of VBench prompts for hyperparameter selection, or report results for a prespecified configuration and present the ablations solely as sensitivity analyses.","section":"Sec. 4.1, Table 3, Figure 6, Table 2"}],"minor_comments":[{"comment":"The name 'ModelScopoe' should be 'ModelScope'.","section":"Related Work"},{"comment":"The caption contains a duplicated 'and' in 'sampling process, diffusion process, and and noise refinement'.","section":"Figure 2 caption"},{"comment":"The word 'open-soruce' should be 'open-source'.","section":"Sec. 4.1"},{"comment":"The word 'accross' should be 'across'.","section":"Figure 6"},{"comment":"The statement that at timestep t around 400 the latent 'has roughly taken shape and resembles the clean latent z_0' is an empirical claim that should be supported by evidence or a reference.","section":"Sec. 3.1"},{"comment":"The worst-case bound in Eq. (10) guarantees a reduction of at least 50% in covariance error, while Table 1 reports FreqPrior errors on the order of 1e-26 versus FreeInit errors of order 1-10, i.e., a reduction of essentially 100%; the paper should clarify whether the near-zero values follow from the specific filter choices and whether a much tighter bound is available.","section":"Eq. (10) and Table 1"}],"recommendation":"major_revision","confidential_remarks":"The violation of Assumption 1 is fundamental to the theoretical contribution, not a mere presentation issue; the authors should either repair the theory or explicitly scope it to the idealized setting. The VBench tuning concern also matters for the empirical claims. If both are addressed, the paper could be acceptable; in its current form the central guarantee does not transfer to the deployed algorithm."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, quick take: FreqPrior is worth a look. The sum-of-squares filter (1-M^2)^0.5 instead of the classical (1-M), the two-noise real/imaginary post-processing, and the covariance-error bound in Eq. (10) are new relative to FreeInit and the other cited noise-prior work. The authors actually derive the distribution of their refined noise rather than hand-wave, and the algebra in Appendices B and C checks out under their explicit Assumption 1. The numerical covariance table is consistent with the formulas. On the empirical side, the gains over Gaussian noise and FreeInit are modest but consistent across three open video diffusion models, and the ~23% inference-time saving from partial sampling is real and useful.\n\nNow the soft spots. The paper's theoretical claim—that FreqPrior's noise stays within a factor of 1/2 of FreeInit's covariance error and is thereby 'closely Gaussian'—rests on Assumption 1: z_noise ~ N(0,I). But in the actual Algorithm 1, z_noise = sqrt(alpha_bar_T/alpha_bar_t) z_t + sqrt(1 - alpha_bar_T/alpha_bar_t) epsilon, where z_t is the DDIM partial-sampling output and is correlated with epsilon through previous refinement iterations. That's not a standard Gaussian vector independent of epsilon. The stress-test note is right that the coefficient on z_t is small (around 0.06-0.1), so the violation may be mild in practice, but the paper doesn't quantify it. The covariance-error table is computed under the assumption, so it does not by itself establish that the deployed prior is nearly Gaussian. This is a gap between theory and the actual algorithm, and the authors should acknowledge it or fix it.\n\nAlso, the VBench headline numbers are point estimates from a single run with hyperparameters (cos theta, timestep t) selected on the same benchmark. No error bars, no per-dimension breakdown. The ablations show the choice is not super sensitive, which helps, but it would be good to see variance across seeds.\n\nNone of this is fatal. The empirical improvement is plausible—better low-frequency retention with less variance decay is a reasonable mechanism—and the speed-up is a clear practical win. The caveats are about the strength of the theoretical guarantee, not about whether the method works. The paper is transparent; it states Assumption 1 explicitly and has a limitations section.\n\nWho is this for? Anyone working on noise priors or inference-time tweaks for video diffusion models. I'd send it to a serious referee. The right outcome is probably conditional acceptance: require the authors to address the assumption gap (empirically or with a relaxation) and to add error bars.\n\nRecommendation: engage with it, but read the theory with the deployment mismatch in mind.","headline":"A genuinely new noise-refinement idea with a real speed-up and consistent empirical gains, but the central Gaussianity guarantee only holds under an assumption the deployed pipeline violates.","tokens_in":22522,"tokens_out":2724,"would_cite":true,"duration_ms":25148,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FreqPrior is a frequency-domain noise-refinement method that keeps the video-diffusion prior near a standard Gaussian and, the paper reports, achieves the highest VBench total scores on three video diffusion models.","keywords":["video diffusion models","noise prior initialization","frequency-domain filtering","Fourier transform","variance decay","text-to-video generation","DDIM sampling","Gaussian noise prior"],"falsifier":"Measure the empirical distribution of $z_{noise}$ at the chosen intermediate timestep on a real model, for example by collecting hundreds of latents and running a normality test that includes third and fourth moments. If $z_{noise}$ is clearly non-Gaussian while the covariance error stays tiny, then the theoretical guarantee in Eq. (10) does not strictly apply to the deployed pipeline, and the observed quality gains would need another explanation.","tokens_in":21491,"feed_emoji":"🎬","tokens_out":10998,"duration_ms":81457,"temperature":0.7,"pith_summary":"FreqPrior is a new noise-initialization strategy for text-to-video diffusion models. The paper argues that the initial noise should stay close to a standard Gaussian distribution, and that FreeInit's frequency filtering does not: it decays the noise variance, producing overly smooth videos with weak motion and lost detail. FreqPrior refines the noise with a frequency filter whose low-pass and high-pass terms obey $m^2 + (1-m^2) = 1$ rather than $m + (1-m) = 1$, so variance is preserved while low frequencies are retained and high frequencies enriched. The paper proves this refined prior has covariance error at most $\\frac{\\cos^2\\theta}{1+\\cos^2\\theta}$ times FreeInit's, at least a 50% reduction, and reports the highest VBench total scores on VideoCrafter, ModelScope, and AnimateDiff, with about 23% less inference time than FreeInit thanks to partial sampling.","feed_headline":"FreqPrior noise beats FreeInit on three video models","feed_subtitle":"Refined noise stays near Gaussian, raising VBench scores while cutting inference time by about 23%.","key_machinery":"The load-bearing object is the noise-refinement operator built from three steps: two correlated mixtures $x_1,x_2$ of the diffused latent $z_{noise}$ with fresh Gaussians $\\eta_1,\\eta_2$; a Fourier-transform filtering stage that retains low frequencies through a low-pass mask $M$ and enriches high frequencies through $(1-M^2)^{1/2}$ instead of the classical $1-M$; and a post-processing step that combines real and imaginary parts of two filtered transforms symmetrically. The square-law filter choice preserves Gaussianity because $m u + (1-m^2)^{1/2}v$ is standard Gaussian when $u,v$ are independent standard Gaussians, and the real/imaginary combination cancels cross terms. The proof uses the DFT identities $AB=BA=0$ and $A^2+B^2=NI$ to obtain the closed-form covariances and the inequality in Eq. (10).","core_discovery":"The paper claims that a good noise prior for pretrained video diffusion models is one that remains essentially standard Gaussian after refinement. It identifies variance decay in FreeInit's refined noise, whose covariance matrix $\\Sigma_{FreeInit}=P^2+(I-P)^2$ is strictly smaller than $I$, and shows that this decay blurs frames and reduces motion. FreqPrior instead produces refined noise with covariance $I - \\frac{2\\cos^2\\theta}{1+\\cos^2\\theta}Q^2$, and the paper proves $\\|I-\\Sigma_{FreqPrior}\\|_F \\le \\frac{\\cos^2\\theta}{1+\\cos^2\\theta} \\|I-\\Sigma_{FreeInit}\\|_F$, i.e. at least a 50% lower covariance error, with numerical covariance errors around $10^{-26}$. The same low-frequency retention that preserves semantic fidelity is kept, so the method improves both quality and semantic scores on VBench while partial sampling cuts inference time by roughly 23%.","pith_inferences":["The covariance-error guarantee is a second-order statement; if the actual $z_{noise}$ in the pipeline is non-Gaussian, because it is built from a partially denoised latent, the approximation could still hold in practice, but higher-order statistics would be needed to confirm the theory transfers.","The same filter design could be tested on image diffusion models, where low-frequency leakage has also been reported, as a cheap extension beyond video generation.","A testable prediction of the paper's diagnosis is that deliberately decaying the variance of the initial Gaussian should reproduce FreeInit-style blur and motion loss, while increasing variance should degrade quality in the opposite direction."],"forward_implications":["FreqPrior can be applied to existing pretrained text-to-video diffusion models without fine-tuning, and it improves VBench total scores over Gaussian noise and FreeInit on VideoCrafter, ModelScope, and AnimateDiff.","The partial-sampling prior search, which perturbs a latent at intermediate timestep $t=321$, reduces prior-refinement cost to about 23% less inference time than FreeInit while keeping generation quality.","The square-law high-pass filter provides a general recipe for manipulating noise in the frequency domain without making its distribution deviate from standard Gaussian, a recipe that should transfer to other diffusion-model noise-editing tasks.","Because the refined prior remains near-Gaussian, downstream denoising sees a prior compatible with the model's training distribution, unlike frame-correlated priors that crash pretrained models."],"supporting_citations":[{"why":"FreeInit is the frequency-filtering baseline whose variance decay and blurred outputs FreqPrior diagnoses and improves upon.","marker":"Wu et al., 2024"},{"why":"DDIM is the sampler used both for the partial sampling prior search and for video generation.","marker":"Song et al., 2021a"},{"why":"Supplies the diffusion process and alpha-bar scheduler notation behind Eq. (1).","marker":"Ho et al., 2020"},{"why":"PYoCo's mixed and progressive noise priors are the frame-correlated baselines whose covariance errors are measured.","marker":"Ge et al., 2023"},{"why":"VBench is the benchmark providing the quality, semantic, and total scores used in the main comparison.","marker":"Huang et al., 2024b"},{"why":"Documents low-frequency signal leakage in diffusion noise, motivating the frequency-domain approach.","marker":"Lin et al., 2024"}],"fun_headline_variants":["FreqPrior beats FreeInit: better videos, 23% faster","Gaussian-close noise prior tops VBench, cuts inference time","Frequency-filtered noise stays Gaussian, outperforms FreeInit","FreqPrior: noise refinement that preserves Gaussian, wins on quality","Video diffusion upgrades: FreqPrior noise, 23% less compute"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Assumption 1 in Section 3.3 states that $z_{noise}$ after the diffusion process is standard Gaussian, $N(0,I)$, and in the deployed algorithm it is formed from a partially denoised DDIM latent $z_t$ plus injected noise, so the Gaussian-derived covariance formulas and the 50% bound inherit that approximation.","fun_headline_variants_meta":{"raw":{"variants":["FreqPrior beats FreeInit: better videos, 23% faster","Gaussian-close noise prior tops VBench, cuts inference time","Frequency-filtered noise stays Gaussian, outperforms FreeInit","FreqPrior: noise refinement that preserves Gaussian, wins on quality","Video diffusion upgrades: FreqPrior noise, 23% less compute"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000642,"raw_usage":{"total_tokens":2959,"prompt_tokens":957,"completion_tokens":2002,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":573,"completion_tokens_details":{"reasoning_tokens":1911}},"tokens_in":573,"tokens_out":2002,"duration_ms":19222,"temperature":1.0,"reasoning_tokens":1911,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T10:31:17.359937+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the empirical distribution of $z_{noise}$ at the chosen intermediate timestep on a real model, for example by collecting hundreds of latents and running a normality test that includes third and fourth moments. If $z_{noise}$ is clearly non-Gaussian while the covariance error stays tiny, then the theoretical guarantee in Eq. (10) does not strictly apply to the deployed pipeline, and the observed quality gains would need another explanation.","supporting_citations":[],"review_version":1}