REVIEW 6 cited by
ViD-GPT: Introducing GPT-style Autoregressive Generation in Video Diffusion Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
With the advance of diffusion models, today's video generation has achieved impressive quality. But generating temporal consistent long videos is still challenging. A majority of video diffusion models (VDMs) generate long videos in an autoregressive manner, i.e., generating subsequent clips conditioned on last frames of previous clip. However, existing approaches all involve bidirectional computations, which restricts the receptive context of each autoregression step, and results in the model lacking long-term dependencies. Inspired from the huge success of large language models (LLMs) and following GPT (generative pre-trained transformer), we bring causal (i.e., unidirectional) generation into VDMs, and use past frames as prompt to generate future frames. For Causal Generation, we introduce causal temporal attention into VDM, which forces each generated frame to depend on its previous frames. For Frame as Prompt, we inject the conditional frames by concatenating them with noisy frames (frames to be generated) along the temporal axis. Consequently, we present Video Diffusion GPT (ViD-GPT). Based on the two key designs, in each autoregression step, it is able to acquire long-term context from prompting frames concatenated by all previously generated frames. Additionally, we bring the kv-cache mechanism to VDMs, which eliminates the redundant computation from overlapped frames, significantly boosting the inference speed. Extensive experiments demonstrate that our ViD-GPT achieves state-of-the-art performance both quantitatively and qualitatively on long video generation. Code will be available at https://github.com/Dawn-LX/Causal-VideoGen.
Forward citations
Cited by 6 Pith papers
-
StarPose: 3D Human Pose Estimation via Spatial-Temporal Autoregressive Diffusion
StarPose lifts 2D keypoints to 3D poses with an autoregressive diffusion process that conditions on historical pose predictions and physics-style constraints, reporting SOTA on Human3.6M and MPI-INF-3DHP.
-
LoViC: Efficient Long Video Generation with Context Compression
LoViC uses FlexFormer, a single-query-token Q-Former with interpolated rotary positional encoding, to compress long video-text context for efficient long-video generation.
-
Hunyuan-GameCraft: High-dynamic Interactive Game Video Generation with Hybrid History Condition
Hunyuan-GameCraft generates long, action-controlled game videos from a single image by unifying keyboard/mouse inputs into a continuous camera space and conditioning on mixed historical context.
-
Video World Models with Long-term Spatial Memory
An autoregressive video world model with a persistent static point-cloud spatial memory and sparse episodic keyframes improves revisit consistency over point-cloud-conditioned baselines.
-
Long-Context State-Space Video World Models
A hybrid state-space and local-attention architecture gives autoregressive video diffusion models long-term spatial memory with constant per-frame inference cost, demonstrated on Maze and Minecraft.
-
BIFE: Better Interaction, Fewer Errors for Minute-Long Video Generation
BlockVid generates minute-long videos with a semantic sparse KV cache, Block Forcing training, and chunk-level noise scheduling, reporting large gains on its own LV-Bench and on VBench.
Discussion (0). Continue with ORCID to comment.