REVIEW 9 cited by
Video-Infinity: Distributed Long Video Generation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Diffusion models have recently achieved remarkable results for video generation. Despite the encouraging performances, the generated videos are typically constrained to a small number of frames, resulting in clips lasting merely a few seconds. The primary challenges in producing longer videos include the substantial memory requirements and the extended processing time required on a single GPU. A straightforward solution would be to split the workload across multiple GPUs, which, however, leads to two issues: (1) ensuring all GPUs communicate effectively to share timing and context information, and (2) modifying existing video diffusion models, which are usually trained on short sequences, to create longer videos without additional training. To tackle these, in this paper we introduce Video-Infinity, a distributed inference pipeline that enables parallel processing across multiple GPUs for long-form video generation. Specifically, we propose two coherent mechanisms: Clip parallelism and Dual-scope attention. Clip parallelism optimizes the gathering and sharing of context information across GPUs which minimizes communication overhead, while Dual-scope attention modulates the temporal self-attention to balance local and global contexts efficiently across the devices. Together, the two mechanisms join forces to distribute the workload and enable the fast generation of long videos. Under an 8 x Nvidia 6000 Ada GPU (48G) setup, our method generates videos up to 2,300 frames in approximately 5 minutes, enabling long video generation at a speed 100 times faster than the prior methods.
Forward citations
Cited by 9 Pith papers
-
MAVIN: Multi-Shot Audio-Visual Generation with Customized Narrative Control
MAVIN proposes boundary-aware attention, ID-aware propagation, a multi-agent scripting pipeline, and the MAVINSet dataset as the first framework for multi-shot audio-visual generation with narrative control, claiming ...
-
TokensGen: Harnessing Condensed Tokens for Long Video Generation
TokensGen generates consistent long videos by representing each clip as condensed semantic tokens, generating all tokens jointly from text, and stitching clips with adaptive FIFO denoising.
-
Ouroboros-Diffusion: Exploring Consistent Content Generation in Tuning-free Long Video Diffusion
Ouroboros-Diffusion improves long video consistency by combining low-frequency tail noise, subject-aware cross-frame attention, and self-recurrent gradient guidance in a tuning-free FIFO diffusion queue.
-
Ingredients: Blending Custom Photos with Video Diffusion Transformers
Ingredients adds a mask-supervised identity router to a video diffusion transformer, enabling multi-person videos from a few reference photos without per-identity fine-tuning.
-
Is Your World Simulator a Good Story Presenter? A Consecutive Events-Based Benchmark for Future Long Video Generation
A new story-completion benchmark, StoryEval, shows that 11 current text-to-video models complete fewer than half of the consecutive events in short story prompts.
-
Long Video Diffusion Generation with Segmented Cross-Attention and Content-Rich Video Data Curation
Presto lets each temporal segment of a video attend to a matching sub-caption, enabling long, content-rich videos that beat prior open-source and commercial models on VBench semantic and dynamic scores.
-
Towards Precise Scaling Laws for Video Diffusion Transformers
Video diffusion transformers follow scaling laws, and tuning learning rate and batch size per model and data size makes those laws precise enough to predict loss and optimal model size at larger scales.
-
Minute-Long Videos with Dual Parallelisms
DualParal parallelizes both temporal frames and DiT layers using block-wise denoising, KV caching, and coordinated noise initialization, generating 1,025-frame videos up to 6.54x faster than baselines.
-
Video Diffusion Transformers are In-Context Learners
Concatenating multiple video clips into one input and fine-tuning a LoRA adapter lets a pretrained video diffusion transformer produce consistent multi-scene videos from a single prompt.
Discussion (0). Continue with ORCID to comment.