{"id":"c18fa930-b519-4753-b719-89f447afc4ae","arxiv_id":"2505.21070","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"DualParal parallelizes both temporal frames and DiT layers using block-wise denoising, KV caching, and coordinated noise initialization, generating 1,025-frame videos up to 6.54x faster than baselines.","lead":"This paper presents DualParal, a way to generate long videos on multiple GPUs by splitting both the frame sequence and the model layers. On eight RTX 4090s it reports up to 6.54x lower latency and 1.48x lower memory for 1,025-frame videos compared with prior distributed methods.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Quality evidence stops at 257 frames; the 1025-frame 'artifact-free' claim is unsupported, so the headline efficiency gains may be for outputs that fail the promised quality.","rationale":"I read the paper in good faith. The parallelization design is coherent, and the efficiency tables are plausible; the quality scores at 129 and 257 frames suggest that desynchronized block-wise denoising does not immediately collapse. The reader's weakest assumption is therefore not refuted, but it is also not the sharpest problem: the short-horizon evidence partially supports it. The truly load-bearing gap is the absence of long-horizon quality measurement. The paper's abstract makes two promises—efficiency and artifact-free infinite-length generation—but only the first is measured at the claimed length of 1,025 frames. This is an evidential hole rather than an internal inconsistency, so it strengthens the reader's CONDITIONAL verdict without moving it. The bubble-ratio arithmetic error in Appendix A.3 (the warm-up and cool-down sum is N^2, not N^2-N-1, and the plotted 5.2% example matches the incorrect value) is real but secondary: the asymptotic conclusion that bubble ratio approaches zero for long videos still holds. My concern is the long-horizon manifestation of the desynchronized-denoising assumption, compounded by potentially stale cached KV features, so I partially agree with the reader's identification of the weakest assumption.","tokens_in":17558,"tokens_out":6513,"duration_ms":85452,"concrete_test":"Run the Table 3 VBench protocol on 513- and 1,025-frame generations with at least 3 seeds per prompt for DualParal (with and without feature cache), FIFO, Video-Infinity, and DeepSpeed-Ulysses, and report per-seed mean plus or minus standard deviation. Accept the central claim only if DualParal's 1,025-frame overall score remains within the 129-frame margin to the best baseline; if it drops below FIFO or Video-Infinity at 1,025 frames, the artifact-free minute-length claim fails. Optionally, masking the feature cache in a separate run would isolate whether stale KV reuse is the source of any long-horizon quality loss.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that DualParal efficiently produces 1,025-frame videos with quality comparable to distributed baselines. The efficiency evidence is direct, but the quality evidence is not: Table 3 evaluates VBench only at 129 and 257 frames, with no error bars or seeds, and no VBench or human evaluation is reported at 513 or 1,025 frames. The 1,025-frame regime is precisely where the method's core assumption—denoising desynchronized temporal blocks with a pretrained model—is most strained. The feature cache of Section 3.2 reuses KV features computed one denoising step earlier, so those keys and values correspond to a noisier version of the context block than the current timestamp calls for; over long horizons this staleness can accumulate. The coordinated initialization of Section 3.3 is validated only visually on short clips (Figure 6), not on minute-length generations. 'Infinitely long' is asserted in the abstract but never tested; the longest generated video is 1,025 frames and its quality is unmeasured. If 1,025-frame outputs show temporal drift, repetitive artifacts, or incoherent motion, the 6.54x latency and 1.48x memory advantages are achieved on outputs that do not satisfy the paper's own 'artifact-free' promise, which would falsify the central claim as stated.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DualParal, a distributed inference strategy for DiT-based video diffusion models that combines temporal sequence parallelism with pipeline parallelism over model layers. The method uses a block-wise denoising FIFO queue so that different temporal blocks can be denoised at different, asynchronous noise levels, and augments this with a feature cache that reuses KV features from previously processed blocks and a coordinated noise initialization strategy. Experiments on Wan2.1 report up to 6.54x lower latency and 1.48x lower peak memory at 1,025 frames on 8xRTX 4090 GPUs relative to Ring Attention, with VBench-based quality comparisons reported at 129 and 257 frames. The abstract and conclusion additionally claim fast, artifact-free, and infinitely long video generation.","tokens_in":17805,"tokens_out":10942,"duration_ms":112919,"significance":"If fully validated, DualParal would be a practically useful contribution to long-video generation on commodity multi-GPU setups, and the paper has several real strengths: it directly measures latency and memory against four existing methods, includes ablations of the queue, device pipeline, and feature cache, and provides a project page and code repository. The core idea of resolving the synchronization conflict between sequence and pipeline parallelism through block-wise denoising is reasonable and extends prior work such as FIFO-Diffusion and Video-Infinity. However, the headline quality claim for 1,025-frame videos is not supported by the reported experiments, and there are mathematical and experimental-consistency issues that need to be addressed before the central claims can be accepted.","major_comments":[{"comment":"The efficiency claim is made for 513 and 1,025 frames in Table 2, but the quality evaluation in Table 3 only covers 129 and 257 frames. No VBench scores, human evaluation, or error bars are reported for 513 or 1,025 frames, which is precisely the regime where the desynchronized block-wise denoising and the stale feature cache are most likely to cause temporal drift or artifacts. Since the abstract promises 'artifact-free and infinitely long' generation, the missing quality evidence at the headline lengths is load-bearing. Please add quantitative quality evaluation at 513 and 1,025 frames, ideally with multiple seeds and variance estimates, or substantially soften the abstract and conclusion claims.","section":"§4.2 / Table 3"},{"comment":"The feature cache reuses Self-Attention KV features from a block at an earlier, noisier denoising state. When denoising block B'_i, the context block B_{i+1} was processed in the previous iteration as part of B'_{i+1}, so the cached K and V correspond to the previous noise level, not the current one. Over long generations this staleness can accumulate and affect temporal coherence. The paper does not analyze this effect or provide an ablation comparing cached KV against freshly recomputed KV at long video lengths. Such an experiment is needed to support the claim that the cache preserves quality.","section":"§3.2"},{"comment":"Equation (3) is based on an algebraic error. The text states that the bubble size equals 1 + 2 + ... + (N-1) + 1 + 2 + ... + N, which evaluates to N^2, not N^2 - N - 1. Consequently, for N=4 the formula gives 11 instead of 16, and the stated 5.2% bubble ratio in Figure 3 does not follow from the written derivation. The asymptotic conclusion that the bubble ratio tends to zero as the number of blocks grows is unaffected, but the quantitative analysis in Section 3.4 and the proof in Appendix A.3 should be corrected.","section":"§3.4 / Appendix A.3"},{"comment":"The quality comparison in Table 3 includes DeepSpeed-Ulysses, but the text and Appendix A.5 state that DeepSpeed-Ulysses cannot run on 8xRTX 4090 GPUs because the number of attention heads is not divisible by 8. The manuscript does not state on how many GPUs DeepSpeed-Ulysses was run for the quality experiments. If it used a different device count than DualParal and the other baselines, the comparison in Table 3 is not properly controlled. Please specify the hardware configuration for each method in the quality evaluation, or match the device counts.","section":"§4.2 / Appendix A.5"}],"minor_comments":[{"comment":"There is a typo in 'Baslines'; it should be 'Baselines'.","section":"§1"},{"comment":"The ordering of noise levels in the queue is described inconsistently: the text says blocks are arranged with 'progressively decreasing noise levels, ranging from 1 to T', while the Figure 1 caption says 'noise levels increasing from tail to head'. Please clarify the naming of indices and the direction of increasing noise.","section":"§3.1 / Figure 1"},{"comment":"The two key observations about noise initialization are supported only by qualitative examples. A quantitative comparison of the four initialization schemes in Figure 2 would make the design choice more convincing.","section":"§3.3 / Figure 2"},{"comment":"At 257 frames, DualParal's overall VBench score is 78.17% versus 78.08% for Video-Infinity, a difference of 0.09 percentage points with no error bars or significance testing. The claim of achieving the 'highest overall score' should be tempered, and standard deviations or multiple-seed results should be reported.","section":"§4.2 / Table 3"},{"comment":"The text 'N onBubble Size' appears to be a spacing/formatting error in the denominator of Equation (3).","section":"§3.4"}],"recommendation":"major_revision","confidential_remarks":"For the editor: the authors have a clear overlap with Video-Infinity (same research group), and the paper compares against Video-Infinity extensively. This is not by itself improper, but I recommend ensuring the baseline settings are justified from the official implementation and that the relationship is transparent. The main risk is the gap between the efficiency evidence at 1,025 frames and the quality evidence at 129/257 frames; if the authors cannot supply quality measurements at the longer lengths, the abstract and conclusion should be revised to claim only efficiency gains at fixed block lengths, not 'artifact-free and infinitely long' generation. The bubble-ratio equation error is minor in impact but should be corrected."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version: DualParal is a real engineering win on the efficiency side, and the latency/memory tables are the part of the paper that should be believed. The quality story is thinner than the abstract suggests — the \"artifact-free, infinitely long\" phrasing is not backed by the experiments.\n\nWhat's new: the paper combines FIFO block-wise denoising with pipeline-parallel layer sharding, and adds two practical tricks — a KV feature cache that avoids re-sending the previous block, and a coordinated noise initialization that prevents repetitive-noise artifacts. The ablation in Table 4 shows each piece contributes. The comparison against FIFO and Video-Infinity is mostly fair (NumC=8 for all), and the efficiency numbers are internally consistent with the fixed-memory-block design. If I needed to generate minute-long 480p video on 8 consumer GPUs, this is the method I'd try.\n\nSoft spots, in rough order of importance:\n\n1. Quality is not measured where the method makes its strongest claim. Table 3 gives VBench only at 129 and 257 frames. The 1025-frame results, the headline of the paper, have no quality evaluation attached. That matters because the whole approach assumes a pretrained DiT can denoise desynchronized blocks without retraining; that assumption is most likely to fail at long horizons. The stress-test concern is on point: the feature cache reuses KV features computed one denoising step earlier at a different noise level, and staleness can accumulate. \"Infinitely long\" is asserted, not demonstrated.\n\n2. Bubble ratio arithmetic. Eq. (3) says the bubble size sums to N^2 - N - 1, but the sum 1+...+N-1 + 1+...+N is N^2. The 5.2% figure is therefore wrong; it would be 7.4% for N=4, T=50, Block num=4. Not a load-bearing flaw, but it is a visible error in the theory section.\n\n3. No error bars or seeds on the VBench numbers. Several between-method differences are within a point or two and could easily be noise.\n\n4. Reproducibility: the paper points to a repository but gives no commit hash or release artifact. For a systems paper, shipping code matters.\n\nOne more thing: the author overlap with Video-Infinity is worth noting but not disqualifying; the comparison looks honest, just potentially self-preferential.\n\nBottom line: this is a useful engineering paper that deserves a serious referee. The efficiency contribution is real. Before acceptance, the authors should produce quality numbers at 513/1025 frames, fix the bubble derivation, and release the code.","headline":"Solid efficiency contribution; quality claims need to catch up with the efficiency evidence before this can be taken at face value.","tokens_in":18354,"tokens_out":2820,"would_cite":true,"duration_ms":31546,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"One inference strategy generates minute-long videos 6.5 times faster across 8 GPUs.","keywords":["video diffusion","diffusion transformer","distributed inference","pipeline parallelism","sequence parallelism","long video generation","block-wise denoising","feature cache"],"falsifier":"Generate a 1,025-frame video with DualParal on the Wan2.1-1.3B model and inspect frames at block boundaries (every eight latent frames) for abrupt jumps in subject identity, background, or motion. If visible seams or duplicated content persist even with the coordinated noise initialization and NumC=8, the central assumption is false. A quantitative check would compare VBench subject-consistency and temporal-flickering scores of DualParal at 257 frames against the full-sequence baseline at 129 frames; a collapse on those two metrics would indicate the desynchronized schedule degrades temporal coherence.","tokens_in":17355,"feed_emoji":"🎞️","tokens_out":4310,"duration_ms":41245,"temperature":0.7,"pith_summary":"The paper tries to show that long video generation with diffusion transformers need not be serialized across devices. It proposes DualParal, a distributed inference strategy that splits both the frame sequence and the model layers across GPUs, using a block-wise denoising schedule in which different temporal blocks are denoised at different noise levels. A FIFO queue feeds blocks through a device pipeline in reverse order, so each GPU can compute and communicate asynchronously rather than waiting for global synchronization. The result, if correct, is that a standard pretrained video diffusion model can generate 1,025-frame videos with up to 6.54x lower latency and 1.48x lower memory than existing distributed methods on 8xRTX 4090 GPUs, with video quality comparable to existing baselines.","feed_headline":"DualParal generates minute-long videos 6.5x faster on 8 GPUs","feed_subtitle":"Block-wise denoising parallelizes frames and model layers, cutting memory by 1.5x at 1,025 frames.","key_machinery":"The central object is the FIFO block queue combined with a device pipeline. Blocks of frame latents are ordered with monotonically decreasing noise levels from tail to head; each diffusion step appends a new noisy block at the tail, pops a clean block at the head, and denoises the queue in reverse order across GPUs that each hold a consecutive subset of DiT layers. This block-wise denoising is what dissolves the conflict between sequence parallelism and pipeline parallelism, and the feature cache and coordinated noise initialization patch the two failure modes the schedule creates, namely redundant communication and repetitive-noise artifacts.","core_discovery":"DualParal establishes that the synchronization requirement of diffusion models, where all frames must share the same noise level at each step, can be replaced by a block-wise denoising schedule without retraining. In this schedule, blocks are queued from almost clean to fully noisy, and each denoising step processes them from tail to head through a model that is itself divided across GPUs. Because adjacent blocks are denoised at neighboring noise levels, outputs can flow asynchronously between devices, eliminating the gather-and-reserialize bottleneck that arises when sequence parallelism and pipeline parallelism are naively combined. Two additions make the scheme practical: a per-GPU key-value cache that reuses self-attention features of the already-processed neighboring block, and a coordinated noise initialization that draws each new block from the full noise space while shuffling to avoid reusing identical noise patterns in concatenated blocks.","pith_inferences":["If desynchronized block-wise denoising is as benign for other DiT video generators as it is for the model tested here, the same scheduling trick may transfer to other large transformer video models; a direct comparison would separate a general principle from model-specific tolerance.","The coordinated noise initialization suggests a retraining-free recipe, full noise pool, shuffled assignment, and exclusion of the last few latents, that could also improve other sliding-window long-video methods without their fusion steps.","The reported efficiency gain is a steady-state result: warm-up and cool-down bubbles dominate when the number of blocks is small, so the advantage is strongest for genuinely long generations and weaker for short clips.","Because the feature cache is applied only to self-attention, models with additional cross-frame interaction mechanisms would need re-measurement of both the cache benefit and the noise-initialization effect."],"forward_implications":["Latency and memory stop growing with total video length: the per-step workload depends on block size, not on how many blocks have been generated, so minute-long or longer videos become feasible on commodity 8-GPU machines.","Infinite-length generation is supported directly: new blocks can be appended indefinitely while the first clean block is popped and decoded continuously.","The method works on a large DiT-based video generator at 480p and 720p resolution without retraining or fine-tuning, transferring to the model's existing inference settings.","The bubble ratio approaches zero as the number of blocks grows, so GPU utilization improves for longer videos rather than degrading.","Compared with FIFO, the pipelined model split also reduces peak memory because model weights are distributed across devices rather than replicated on every device."],"supporting_citations":[{"why":"Supplies the DiT-based video diffusion model whose blocks are split across devices and whose inference settings are reused throughout the experiments.","marker":"[27]"},{"why":"Introduces the FIFO queue and boundary-context concatenation idea that DualParal adapts, and serves as a baseline for latency and memory comparison.","marker":"[12]"},{"why":"Provides the distributed long-video generation baseline whose context-concatenation approach is extended and partly replaced by the feature cache.","marker":"[25]"},{"why":"Ring Attention is the sequence-parallelism baseline whose communication and memory costs are compared in the paper's quantitative analysis.","marker":"[17]"},{"why":"DeepSpeed-Ulysses is the sequence-parallelism baseline used for all-to-all communication cost comparison and for memory comparison.","marker":"[11]"},{"why":"Supplies the pipeline-parallelism formulation and the bubble-ratio metric used to analyze DualParal's idle time.","marker":"[9]"},{"why":"Represents the prior noise-rescheduling approach that uses a subset of the noise space, against which the paper's complete-noise-space initialization is positioned.","marker":"[21]"},{"why":"Uses the full noise space with latent fusion for long video generation, serving as the contrast for the paper's zero-fusion coordinated noise initialization.","marker":"[29]"}],"fun_headline_variants":["DualParal speeds up minute-long video diffusion 6.5x","Block-wise denoising enables asynchronous GPU pipelines","Parallel frames and layers cut video latency 6.5x","1,025-frame videos with 6.5x speedup on 8 GPUs","DualParal: dual parallelism for long video generation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"A pretrained diffusion transformer can denoise different temporal blocks at different, unsynchronized noise levels and still produce a single temporally coherent video, without any retraining or fine-tuning.","fun_headline_variants_meta":{"raw":{"variants":["DualParal speeds up minute-long video diffusion 6.5x","Block-wise denoising enables asynchronous GPU pipelines","Parallel frames and layers cut video latency 6.5x","1,025-frame videos with 6.5x speedup on 8 GPUs","DualParal: dual parallelism for long video generation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000278,"raw_usage":{"total_tokens":1685,"prompt_tokens":1007,"completion_tokens":678,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":623,"completion_tokens_details":{"reasoning_tokens":587}},"tokens_in":623,"tokens_out":678,"duration_ms":6118,"temperature":1.0,"reasoning_tokens":587,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T13:37:00.054077+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Generate a 1,025-frame video with DualParal on the Wan2.1-1.3B model and inspect frames at block boundaries (every eight latent frames) for abrupt jumps in subject identity, background, or motion. If visible seams or duplicated content persist even with the coordinated noise initialization and NumC=8, the central assumption is false. A quantitative check would compare VBench subject-consistency and temporal-flickering scores of DualParal at 257 frames against the full-sequence baseline at 129 frames; a collapse on those two metrics would indicate the desynchronized schedule degrades temporal coherence.","supporting_citations":[{"cited_title":"Wan: Open and advanced large-scale video generative models, 2025","cited_arxiv_id":null,"evidence_quote":"Supplies the DiT-based video diffusion model whose blocks are split across devices and whose inference settings are reused throughout the experiments."},{"cited_title":"Fifo-diffusion: Generating infinite videos from text without training","cited_arxiv_id":null,"evidence_quote":"Introduces the FIFO queue and boundary-context concatenation idea that DualParal adapts, and serves as a baseline for latency and memory comparison."},{"cited_title":"Gpipe: Efficient training of giant neural networks using pipeline parallelism","cited_arxiv_id":null,"evidence_quote":"Supplies the pipeline-parallelism formulation and the bubble-ratio metric used to analyze DualParal's idle time."},{"cited_title":"Freenoise: Tuning-free longer video diffusion via noise rescheduling","cited_arxiv_id":null,"evidence_quote":"Represents the prior noise-rescheduling approach that uses a subset of the noise space, against which the paper's complete-noise-space initialization is positioned."}],"review_version":1}