Pith. sign in

REVIEW 6 cited by

LiteVAR: Compressing Visual Autoregressive Modelling with Efficient Attention and Quantization

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2411.17178 v1 pith:UUWF3PEA submitted 2024-11-26 cs.CV

Rui Xie , Tianchen Zhao , Zhihang Yuan , Rui Wan , Wenxi Gao , Zhenhua Zhu , Xuefei Ning , Yu Wang This is my paper
classification cs.CV
keywords attentionefficientmodelsperformancereductionvisualautoregressivedeployment
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Visual Autoregressive (VAR) has emerged as a promising approach in image generation, offering competitive potential and performance comparable to diffusion-based models. However, current AR-based visual generation models require substantial computational resources, limiting their applicability on resource-constrained devices. To address this issue, we conducted analysis and identified significant redundancy in three dimensions of the VAR model: (1) the attention map, (2) the attention outputs when using classifier free guidance, and (3) the data precision. Correspondingly, we proposed efficient attention mechanism and low-bit quantization method to enhance the efficiency of VAR models while maintaining performance. With negligible performance lost (less than 0.056 FID increase), we could achieve 85.2% reduction in attention computation, 50% reduction in overall memory and 1.5x latency reduction. To ensure deployment feasibility, we developed efficient training-free compression techniques and analyze the deployment feasibility and efficiency gain of each technique.

Discussion (0). Sign in to comment.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Token Radius Attention for Efficient Video Generation

    cs.CV 2026-08 conditional novelty 6.0 of 10

    Video diffusion transformers can run ~1.5-2x faster with competitive quality by converting each query's attention entropy into a spatially decayed retention radius instead of dense attention.

  2. FPQVAR: Floating Point Quantization for Visual Autoregressive Model with FPGA Hardware Co-design

    cs.CV 2025-05 conditional novelty 6.0 of 10

    A post-training floating-point quantization framework with grouped Hadamard rotation and learned smoothing brings 4-bit visual autoregressive image generation to near-FP16 quality, plus a matching FPGA accelerator.

  3. Adaptive Visual Autoregressive Acceleration via Dual-Linkage Entropy Analysis

    cs.CV 2026-02 conditional novelty 5.0 of 10

    A training-free entropy-guided token-pruning framework accelerates VAR image generation up to 2.9× with negligible benchmark loss by activating pruning at an adaptive entropy-growth inflection point and adjusting rati...

  4. FasterVAR: Plug-and-Play Acceleration for Visual Autoregressive Models

    cs.CV 2025-12 conditional novelty 5.0 of 10

    Stage-aware pruning of late generation steps, using random projection and cached-feature restoration, speeds up VAR text-to-image models by up to 3.4x with minimal quality loss.

  5. Memory-Efficient Visual Autoregressive Modeling with Scale-Aware KV Cache Compression

    cs.LG 2025-05 conditional novelty 5.0 of 10

    ScaleKV cuts KV cache memory for Visual Autoregressive text-to-image generation to 10% by classifying layers as drafters or refiners per scale and pruning low-attention tokens while keeping benchmark scores nearly unchanged.

  6. SkipVAR: Accelerating Visual Autoregressive Modeling via Adaptive Frequency-Aware Skipping

    cs.CV 2025-06 conditional novelty 4.0 of 10

    SkipVAR selects, per sample, between step skipping and unconditional branch replacement using handcrafted frequency features and a trained logistic regression, to accelerate visual autoregressive generation.

Pith tools