Pith. sign in

REVIEW 23 cited by

Real-Time Video Generation with Pyramid Attention Broadcast

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2408.12588 v3 pith:SGQ2PBMG submitted 2024-08-22 cs.CV cs.DC

Xuanlei Zhao , Xiaolong Jin , Kai Wang , Yang You This is my paper · ORCID
classification cs.CVcs.DC
keywords attentionbroadcastgenerationpyramidreal-timevideomethodachieving
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

We present Pyramid Attention Broadcast (PAB), a real-time, high quality and training-free approach for DiT-based video generation. Our method is founded on the observation that attention difference in the diffusion process exhibits a U-shaped pattern, indicating significant redundancy. We mitigate this by broadcasting attention outputs to subsequent steps in a pyramid style. It applies different broadcast strategies to each attention based on their variance for best efficiency. We further introduce broadcast sequence parallel for more efficient distributed inference. PAB demonstrates up to 10.5x speedup across three models compared to baselines, achieving real-time generation for up to 720p videos. We anticipate that our simple yet effective method will serve as a robust baseline and facilitate future research and application for video generation.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 23 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. JAGG: Jacobian-Aggregated Group Gradient for Efficient GRPO Training of Diffusion Models

    cs.LG 2026-07 conditional novelty 7.0 of 10

    JAGG replaces per-step gradient backpropagation in diffusion GRPO with two endpoint backward passes joined by timestep-weighted interpolation, giving ~2x backward-pass savings at modest quality cost.

  2. MXAttention: Data-Free Optimal Scaling and Pre-Normalization Quantization for MXFP4 Attention

    cs.LG 2026-07 conditional novelty 6.5 of 10

    A closed-form MXFP4 scale boundary Qmax=7.25 plus pre-normalization softmax quantization recovers nearly all FP16 video quality lost by naive MXFP4 attention.

  3. Token Radius Attention for Efficient Video Generation

    cs.CV 2026-08 conditional novelty 6.0 of 10

    Video diffusion transformers can run ~1.5-2x faster with competitive quality by converting each query's attention entropy into a spatially decayed retention radius instead of dense attention.

  4. EchoCache: Energy-Guided Cross-Modal Caching for Efficient Audio-Driven Video Generation

    cs.CV 2026-08 conditional novelty 6.0 of 10

    Audio time-frequency energy guides which video latents get recomputed during diffusion denoising, yielding up to 2.46x faster audio-driven video generation with competitive quality.

  5. CachedSearch: Training-Free Cached Exploration for Test-Time Search in Video Diffusion

    cs.AI 2026-07 conditional novelty 6.0 of 10

    Using cached drafts to select the winner and regenerating only that winner captures 94.7% of best-of-8 search gain at 63% of the cost.

  6. Importance-Aware OBS Pruning for Diffusion Models

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Injecting spatial importance maps (e.g., CFG delta) into the OBS Hessian improves subject preservation in pruned diffusion models at high sparsity, but gains over the baseline are small and without error bars.

  7. DiTango: Cost-Effective Parallel Diffusion Generation with Selective Attention State Reuse

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Selective reuse of composed attention states across denoising steps lets DiTango skip both remote KV communication and attention compute for low-contribution sequence partitions, cutting multi-GPU diffusion latency by...

  8. Kaleido: Algorithm-Hardware Co-Design for Video Diffusion Transformers by Exploiting Latent Space Correlations

    cs.AR 2026-07 conditional novelty 6.0 of 10

    A channel-wise reuse algorithm plus a reconfigurable systolic accelerator skips redundant vDiT attention and MLP computation, achieving up to 5.9x speedup and 16x energy savings.

  9. Transition Matching Distillation for Fast Video Generation

    cs.CV 2026-01 conditional novelty 6.0 of 10

    Splitting a video diffusion model into a fixed feature extractor and a small recurrent flow head lets TMD generate videos in one to two effective steps with better VBench scores than prior distilled models.

  10. Timeripple: Accelerating vDiTs by Understanding the Spatio-Temporal Correlations in Latent Space

    cs.AR 2025-11 conditional novelty 6.0 of 10

    Timeripple cuts vDiT self-attention compute by up to 85% by reusing partial attention scores of spatially and temporally correlated tokens across channels, with VBench quality essentially unchanged.

  11. DiTReducio: A Training-Free Acceleration for DiT-Based TTS via Progressive Calibration

    cs.SD 2025-09 conditional novelty 6.0 of 10

    DiTReducio is a training-free, pattern-guided layer and branch skipping method that accelerates DiT-based TTS, reporting significant FLOP and RTF reductions with modest quality loss at tuned thresholds.

  12. OmniCache: A Trajectory-Oriented Global Perspective on Training-Free Cache Reuse for Diffusion Transformer Models

    cs.CV 2025-08 unverdicted novelty 6.0 of 10

    A training-free cache-reuse scheme that spreads computation across the full diffusion trajectory and subtracts estimated noise, accelerating DiT sampling with claimed competitive quality.

  13. Forecast then Calibrate: Feature Caching as ODE for Efficient Diffusion Transformers

    cs.CV 2025-08 conditional novelty 6.0 of 10

    A training-free predictor-corrector method that accelerates Diffusion Transformers by solving a feature-ODE, achieving large compute reductions with modest quality loss.

  14. Dual-Expert Consistency Model for Efficient and High-Quality Video Generation

    cs.CV 2025-06 conditional novelty 6.0 of 10

    By training a semantic expert and a LoRA-based detail expert, DCM reaches nearly teacher-level VBench scores with 4-step video sampling on HunyuanVideo and CogVideoX.

  15. SRDiffusion: Accelerate Video Diffusion Inference via Sketching-Rendering Cooperation

    cs.GR 2025-05 conditional novelty 6.0 of 10

    SRDiffusion accelerates video diffusion by switching from a large model to a smaller sibling model after early high-noise steps, using an adaptive threshold for the switch.

  16. dKV-Cache: The Cache for Diffusion Language Models

    cs.CL 2025-05 conditional novelty 6.0 of 10

    dKV-Cache reuses cached key and value states of decoded tokens during diffusion LM denoising, delivering 2-10x faster inference with near-lossless quality on several benchmarks.

  17. Efficient-vDiT: Efficient Video Diffusion Transformers With Attention Tile

    cs.CV 2025-02 conditional novelty 6.0 of 10

    A three-stage pipeline combining sparse 'tile' attention with multi-step consistency distillation makes Open-Sora-Plan video generation up to 7.8x faster while keeping the aggregate VBench final score within 1%.

  18. OmniCache: Multidimensional Hierarchical Feature Caching For Diffusion Models

    cs.CV 2026-07 conditional novelty 5.0 of 10

    Training-free hierarchical feature caching across token, frame, block, and layer axes cuts diffusion inference latency up to 35% while preserving quality better than averaging-based token merging.

  19. ACID: Adaptive Caching for vIDeo generation

    cs.CV 2026-07 conditional novelty 5.0 of 10

    Adaptive threshold switching on the drift signal's rate of change expands the quality-vs-speed Pareto frontier of TeaCache, EasyCache, and DiCache across HunyuanVideo, Wan 2.1, and CogVideoX.

  20. EC-Diff: Fast and High-Quality Edge-Cloud Collaborative Inference for Diffusion Models

    cs.CV 2025-07 reject novelty 5.0 of 10

    EC-Diff accelerates edge-cloud diffusion inference with a k-step noise approximation strategy and a two-stage greedy search for the cloud-edge handoff point, claiming about 2x speedup with preserved quality.

  21. PromptTea: Let Prompts Tell TeaCache the Optimal Threshold

    cs.CV 2025-07 conditional novelty 5.0 of 10

    PromptTea tunes diffusion-model cache reuse thresholds using prompt-derived complexity, achieving up to 2.79x speedup on Wan2.1 with PSNR 23.0 dB.

  22. Sparse-vDiT: Unleashing the Power of Sparse Attention to Accelerate Video Diffusion Transformers

    cs.CV 2025-06 conditional novelty 5.0 of 10

    Sparse-vDiT replaces dense attention with fixed per-head sparse patterns chosen offline, achieving 1.58-1.85x end-to-end speedups on CogVideoX1.5, HunyuanVideo, and Wan2.1 with minimal quality loss.

  23. Memory-Efficient Visual Autoregressive Modeling with Scale-Aware KV Cache Compression

    cs.LG 2025-05 conditional novelty 5.0 of 10

    ScaleKV cuts KV cache memory for Visual Autoregressive text-to-image generation to 10% by classifying layers as drafters or refiners per scale and pruning low-attention tokens while keeping benchmark scores nearly unchanged.

Pith tools