Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

Memory-Efficient Visual Autoregressive Modeling with Scale-Aware KV Cache Compression

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read ScaleKV cuts the KV cache for visual autoregressive text-to-image to 10% without visible quality loss.

desk verdict ScaleKV shows a real improvement in VAR KV-cache compression, but the pixel-fidelity claim outruns the evidence until full-cache baselines and a token-selection ablation are added. read the letter →

arxiv 2505.19602 v1 pith:BF6UBQEZ submitted 2025-05-26 cs.LG

Kunjun Li , Zigeng Chen , Cheng-Yen Yang , Jenq-Neng Hwang This is my paper
classification cs.LG
keywords KVcachecompressionvisualautoregressivemodelingnext-scalepredictionattentionselectivityindextext-to-imagegenerationmemory-efficientinferenceInfinitymodeltokeneviction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the memory bottleneck in visual autoregressive image generation can be broken by treating the key-value cache differently at each transformer layer and each generation scale. It proposes ScaleKV, which classifies layers as drafters, needing large caches because they spread attention across many earlier scales, or refiners, needing little cache because they focus on the current token map. On the Infinity model family, the paper reports shrinking the KV cache from 85 GB to 8.5 GB at batch size 8 while keeping GenEval at 0.79 and DPG nearly unchanged. That matters because the exponentially growing token maps of VAR models have made high-resolution text-to-image generation impractical on available GPUs.

What carries the argument

The central mechanism is the Attention Selectivity Index (ASI), a per-layer, per-scale score defined as the product of the fraction of attention a query in the current token map pays to the current map and the sum of its top-$K'$ attention weights toward history tokens. Layers with low ASI are classified as drafters (globally dispersed attention, large cache budgets), while layers with high ASI are refiners (locally concentrated attention, small budgets). Z-scores normalize ASI across scales, and a fixed number of layer-scale pairs with the lowest scores become drafters. Cache budgets are then allocated so refiners' budgets decay linearly with scale while drafters receive the freed capacity. At inference, a 16-token observation window samples each token map spatially, and the remaining tokens are scored by the cumulative attention they receive from this window; only the top tokens per layer keep their KV states. The ASI carries the argument because it turns the qualitative drafters-versus-refiners observation into a concrete, calibration-based budget assignment.

What would settle it

Generate prompts from distributions far outside the calibration set, such as dense multi-object scenes versus single-object scenes, compute ScaleKV's per-layer token selections with and without recalibration, and measure FID on held-out images; if recalibration changes the drafter/refiner assignments or improves FID by a large margin, the fixed-calibration assumption is false.

Watch

Extended reading notes

Core claim

The central claim is that a scale-aware KV cache compression scheme can reduce the cache memory of visual autoregressive text-to-image models to one tenth of its original size with negligible degradation in generated image quality. The paper identifies two regularities in attention behavior: different layers have different cache demands, and the same layer shifts its attention pattern as generation moves to finer scales. ScaleKV measures these patterns with an Attention Selectivity Index, assigns each layer-scale pair a cache budget, and prunes KV states according to attention importance. On Infinity-8B, the paper reports a reduction from 85 GB to 8.5 GB of KV cache at batch size 8, with GenEval holding at 0.79 and DPG moving from 86.61 to 86.49; on Infinity-2B the compressed model matches the full model on DPG and slightly improves the overall GenEval score.

Load-bearing premise

The load-bearing premise is that a layer's drafter/refiner role, fixed from a few calibration prompts, and the importance scores read from a 16-token observation window, remain correct for unseen prompts and for all scales; if attention is prompt-dependent or later scales need pruned tokens, the fixed budgets will degrade generated images.

Editorial extensions

If this is right

  • At a 10% budget, Infinity-8B needs 8.5 GB instead of 85 GB for the KV cache, which is the difference between running out of memory and fitting on a single high-end GPU.
  • ScaleKV's savings grow with batch size: batch-16 inference fits under 80 GB total, while the uncompressed model cannot run with a batch of 8.
  • Reduced memory traffic yields up to 1.25x inference speedup, with larger speedups at higher resolutions.
  • The method holds up across model sizes and budgets, with quality preserved at 4%, 10%, and 20% cache budgets on both Infinity-2B and Infinity-8B.
  • Because ScaleKV is post-training and only prunes cache states, the paper positions it as compatible with other efficiency techniques such as quantization or decoding-speed methods.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The near-instant convergence of drafter/refiner classification with a single calibration prompt suggests the split may be an architectural property of VAR transformers, so a purely structural rule-of-thumb might eventually replace calibration.
  • The 16-token observation window is a stress point: prompts with many small objects may need tokens outside that window, which could be tested by comparing ScaleKV's selected tokens against full-attention importance on such prompts.
  • Scale-aware eviction of this kind could generalize to other hierarchical or multi-scale generative models, because the drafters/refiners pattern is about attention geometry rather than the next-scale training objective.
  • If combined with 4-bit or 2-bit KV quantization, the 10% cache could shrink further, though quantization error may change the attention scores that token selection relies on; the paper does not test this interaction.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes ScaleKV, a training-free KV cache compression method for Visual Autoregressive (VAR) text-to-image models. The method classifies transformer layers into drafters and refiners based on an Attention Selectivity Index computed from a small calibration set, then allocates scale-dependent cache budgets: drafters retain larger budgets while refiner budgets decay linearly across scales. Token-level pruning selects the top-k KV states using cumulative attention from a 16-token observation window. Experiments on Infinity-2B and Infinity-8B show that at 10% KV cache memory, GenEval and DPG scores remain nearly unchanged, and the method outperforms prior KV compression baselines by a large margin on MS-COCO FID, LPIPS, and PSNR. The paper also reports latency and memory usage improvements.

Significance. If the results hold, ScaleKV is a practical contribution: it addresses a real memory bottleneck in VAR inference, is post-hoc (no retraining), and is orthogonal to other efficiency techniques. The paper provides a useful empirical characterization of drafter/refiner attention patterns and shows that scale-aware, layer-adaptive budget allocation can substantially outperform fixed or uniform compression policies. The reported comparison against four baselines on two model sizes is a strength, as is the public code release. However, the central 'pixel-level fidelity' claim is not directly evidenced because the full-cache reference values for FID/LPIPS/PSNR are omitted, and the robustness of the 16-token observation window is not established. The method also involves several hyperparameters tuned on the evaluation metric, so the headline numbers should be interpreted with that in mind.

major comments (3)
  1. [Table 1] The Full Cache row leaves FID, LPIPS, and PSNR as '-' for both Infinity-2B and Infinity-8B. The paper's headline claim is that ScaleKV 'preserves pixel-level fidelity' at 10% cache, but without the full-cache reference values the reader cannot quantify fidelity loss. For example, ScaleKV's FID of 2.53 (2B) or 2.12 (8B) may be close to or far from the original model's FID, and the absence of this baseline also weakens the comparison to prior compression methods. Please report the full-cache FID, LPIPS, and PSNR for both models, and include them in Figure 7(a) as a reference point.
  2. [Section 3.3, Eq. (5)] Token importance is computed from cumulative attention received from a 16-token observation window, but the paper provides no evidence that this tiny window is a stable proxy for the attention behavior of future scales or unseen prompts. Figure 6(a) actually shows that attention distributions change substantially between small and large scales, which motivates a concern that a window sampled at scale k may not contain the queries that will later need the cached KV states. Please add an ablation that varies the observation window size, an analysis of the stability of selected token sets across scales, or an evaluation on held-out prompt distributions to directly test this load-bearing assumption.
  3. [Sections 3.3 and 4.3] The method introduces several hyperparameters—N_d (number of drafter layer-scale pairs), K' (top-K in ASI), B_r(0) and delta (refiner budget schedule), and the observation window size—and the reported FID is used to select some of them. Figure 6(c) reports an ablation for delta, but there is no ablation for N_d or K', and no evidence that their chosen values generalize across prompt distributions. Given that FID is the optimization target, the headline 10% result may partly reflect overfitting to the MS-COCO evaluation set. Please report sensitivity to N_d and K', and preferably select hyperparameters on a validation split disjoint from the test set.
minor comments (6)
  1. [Section 3.3, Eq. (2)] The notation 'TopKSum′' is defined only in prose as 'the sum of the top-K′ attention scores'; the prime is unexplained and could be confused with a derivative. Please use a standard notation such as TopKSum_{K'} and state the value of K' used in experiments.
  2. [Table 2] The table header contains two 'Overall' columns, one for GenEval and one for DPG, which is confusing. Please relabel the DPG columns (e.g., 'DPG Overall' and 'DPG Global Relation') and clarify in the caption that DPG-Bench reports both an overall score and a global-relation sub-score.
  3. [Appendix A] The reported FID of 2.53 with zero standard deviation across calibration set sizes from 1 to 128 is surprisingly constant. In addition to FID, please report the agreement of the resulting drafter/refiner assignments (e.g., Jaccard index or overlap percentage) across calibration set sizes, since that would more directly demonstrate the stability of the classification.
  4. [Section 4.1] The calibration prompts are said to be generated by GPT-4o, but the prompt template and the actual 10 prompts are not provided. Releasing this data would make the method reproducible, given that the drafter/refiner assignment depends on it.
  5. [Figure 5] The 'Original' panel shows full-cache outputs, but the corresponding quantitative FID/LPIPS/PSNR values are absent from Table 1. Adding the full-cache values to the figure or caption would help the reader judge the visual claim of 'negligible degradation'.
  6. [Related Work] Reference [44] (Head-aware KV cache compression for visual autoregressive modeling) is cited in the related-work discussion but never compared or mentioned again. A brief description of its method and how ScaleKV differs would be helpful.

Circularity Check

1 steps flagged · score 4.0 of 10

ScaleKV's headline 10%-budget FID is partly a fitted value: delta and the ASI metric are chosen by best FID on the same MS-COCO validation set, so that specific fidelity number is not an independent prediction, though GenEval/DPG and baseline comparisons provide independent support.

  1. fitted input called prediction [Section 4.3, Figure 6(c) and Table 1; also Figure 6(b)]
    "Figure 6(c) shows the effectiveness of refiner budget decay strategy (Equation 4) under a 10% budget constraint (650 tokens per head/layer). With an initial refiner budget of 600 tokens, we observe a consistent improvement in FID from 3.49 to 2.53 as decay rate increases from 0 to 70, confirming our observation that refiner attention becomes increasingly focused at higher scales, requiring fewer resources."

    The decay-rate parameter delta in Eq. (4) is tuned by minimizing FID on the MS-COCO 2017 validation set, and Table 1 reports ScaleKV's FID of 2.53 at the 10% budget on that same validation set. Thus the headline 'pixel-level fidelity' number is the selected minimum of the tuning curve, not an independent prediction; Figure 6(b) similarly selects the ASI metric over Top-K based on the same FID. The GenEval/DPG numbers and comparisons against external baselines are independent, so the circularity is partial rather than total.

full rationale

No load-bearing self-citation or uniqueness-imported-from-authors pattern appears: the only self-references ([7], [9]) are related-work mentions and are not used to justify ScaleKV's mechanism. The core derivation (ASI -> drafter/refiner classification -> scale-aware budget allocation -> attention-based token selection) is not definitionally equivalent to its output; the quality is measured against an external model (Infinity) and external benchmarks (GenEval, DPG) and against previously published compression baselines. The main circularity burden is evaluation-side: the refiner budget decay rate and the choice of ASI as the drafter-identification metric are selected by best FID on the same MS-COCO validation set used in Table 1, so the reported FID at 10% budget is partly a fitted value rather than a prediction. The observation-window size (16 tokens) and other hyperparameters are also not ablated, but that is a robustness gap, not a circular derivation. Overall score 4 reflects partial circularity in the headline fidelity metric while acknowledging independent external validation.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claim rests on a small number of postulates: the drafter/refiner dichotomy is stable, a 16-token observation window reveals which KV states matter, refiner budgets should decay linearly, and Z-scored ASI values are comparable across scales. The method also depends on several fitted or hand-chosen values: N_d, K', B_r(0), delta, and the observation window size. There are no invented physical entities. The 10% memory figure is partly by construction because the total budget is fixed; the quality preservation is an empirical claim.

free parameters (6)
  • N_d (number of drafter layer-scale pairs) = not specified
    Eq. (3) defines drafters as the N_d pairs with lowest ASI Z-scores, but N_d is never defined or justified; it determines how much cache is moved from refiners to drafters.
  • K' (history Top-K window in ASI) = not specified
    Eq. (2) depends on TopKSum' over top K' history scores; the paper never states K' or ablates it.
  • B_r(0) (initial refiner budget) = 600 tokens (per head/layer at 10% budget)
    Section 4.3 Figure 6(c) reports an initial refiner budget of 600 tokens; this value is chosen by the authors, not derived.
  • delta (refiner budget decay rate) = 70 (best value in Figure 6(c))
    Eq. (4) linearly decays refiner budgets as Br(k)=Br(0)-delta*k; Figure 6(c) sweeps delta and reports the best FID, indicating the value was selected on the evaluation dataset.
  • Observation window size = 16 tokens (example)
    Section 3.3 says the observation window is 'typically comprising only 16 tokens'; the number of patches and centroid selection rule are not specified and no ablation is given.
  • Calibration prompt set = 10 prompts from GPT-4o
    Section 4.1 uses 10 GPT-4o prompts to identify drafters/refiners; the appendix claims 1 prompt suffices, so the choice is not load-bearing, but it is part of the method recipe.
assumptions (4)
  • domain assumption VAR layer attention can be dichotomized into drafters and refiners, and this assignment is stable across prompts
    Section 3.2 and Eq. (3) build the entire budget allocation on this categorization; if the pattern shifts with prompt content or during generation, the fixed budgets are wrong.
  • domain assumption A small observation window's cumulative attention scores identify the KV tokens needed by future scales
    Section 3.3 KV Cache Selection; every pruning decision relies on this proxy, and the paper does not validate it against full attention or an oracle.
  • ad hoc to paper Refiner cache budgets decrease linearly with scale
    Eq. (4) imposes Br(k)=Br(0)-delta*k; neither the linear form nor the chosen delta (70) is derived from a model of attention.
  • ad hoc to paper Z-scoring ASI within each scale makes all layer-scale pairs comparable for ranking
    Eq. (3) normalizes per scale then ranks across scales; comparability across different token counts and attention distributions is assumed, not proved.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Memory-Efficient Visual Autoregressive Modeling with Scale-Aware KV Cache Compression." pith.science (2026). https://pith.science/paper/BF6UBQEZ

@misc{pith2026250519602,
  author       = {Pith},
  title        = {Pith review of: Memory-Efficient Visual Autoregressive Modeling with Scale-Aware KV Cache Compression},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BF6UBQEZ}},
  note         = {Machine review of arXiv:2505.19602}
}
read the original abstract

Visual Autoregressive (VAR) modeling has garnered significant attention for its innovative next-scale prediction approach, which yields substantial improvements in efficiency, scalability, and zero-shot generalization. Nevertheless, the coarse-to-fine methodology inherent in VAR results in exponential growth of the KV cache during inference, causing considerable memory consumption and computational redundancy. To address these bottlenecks, we introduce ScaleKV, a novel KV cache compression framework tailored for VAR architectures. ScaleKV leverages two critical observations: varying cache demands across transformer layers and distinct attention patterns at different scales. Based on these insights, ScaleKV categorizes transformer layers into two functional groups: drafters and refiners. Drafters exhibit dispersed attention across multiple scales, thereby requiring greater cache capacity. Conversely, refiners focus attention on the current token map to process local details, consequently necessitating substantially reduced cache capacity. ScaleKV optimizes the multi-scale inference pipeline by identifying scale-specific drafters and refiners, facilitating differentiated cache management tailored to each scale. Evaluation on the state-of-the-art text-to-image VAR model family, Infinity, demonstrates that our approach effectively reduces the required KV cache memory to 10% while preserving pixel-level fidelity.

Figures

Figures reproduced from arXiv: 2505.19602 by the authors.

Figure 1
Figure 1. We introduce a new KV cache compression framework for Visual Autoregressive modeling [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. By implementing scale-aware layer budget allocation, ScaleKV enables differentiated cache [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. (a) Exponential KV cache growth. (b) Visualization of two distinct attention patterns. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Overview of ScaleKV. Our method categorizes transformer layers into drafters (require [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Qualitative comparison between the original Infinity-8B model and our proposed ScaleKV. [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: (a) Kernel Density Estimation of normalized current attention scores at small scales [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: (a) FID under different KV cache budgets. (b) Inference latency for different resolutions. [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: FID score consistency across calibration set sizes. ScaleKV maintains stable performance [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: Visualization of Drafter Layer Attention Maps. [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: Visualization of Refiner Layer Attention Maps. [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]
Figure 11
Figure 11. Figure 11: Generated Images from ScaleKV-Compressed Infinity-8B. [PITH_FULL_IMAGE:figures/full_fig_p018_11.png]
Figure 12
Figure 12. Figure 12: Generated Images from ScaleKV-Compressed Infinity-2B. [PITH_FULL_IMAGE:figures/full_fig_p019_12.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Token Radius Attention for Efficient Video Generation

    cs.CV 2026-08 conditional novelty 6.0 of 10

    Video diffusion transformers can run ~1.5-2x faster with competitive quality by converting each query's attention entropy into a spatially decayed retention radius instead of dense attention.

Reference graph

Works this paper leans on

81 extracted references · 6 canonical work pages · cited by 1 Pith paper

  1. [44]

    Head-aware kv cache compression for efficient visual autoregressive modeling.arXiv preprint arXiv:2504.09261, 2025

    Ziran Qin, Youru Lv, Mingbao Lin, Zeren Zhang, Danping Zou, and Weiyao Lin. Head-aware kv cache compression for efficient visual autoregressive modeling.arXiv preprint arXiv:2504.09261, 2025

    arXiv 2025

  2. [1]

    Longformer: The long-document transformer.arXiv preprint arXiv:2004.05150, 2020

    Iz Beltagy, Matthew E Peters, and Arman Cohan. Longformer: The long-document transformer.arXiv preprint arXiv:2004.05150, 2020

    arXiv 2004

  3. [2]

    Improving image generation with better captions.Computer Science

    James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. Improving image generation with better captions.Computer Science. https://cdn. openai. com/papers/dall-e-3. pdf, 2(3):8, 2023

    2023

  4. [3]

    Pyramidkv: Dynamic kv cache compression based on pyramidal information funneling.arXiv preprint arXiv:2406.02069, 2024

    Zefan Cai, Yichi Zhang, Bofei Gao, Yuliang Liu, Tianyu Liu, Keming Lu, Wayne Xiong, Yue Dong, Baobao Chang, Junjie Hu, and Xiao Wen. Pyramidkv: Dynamic kv cache compression based on pyramidal information funneling.arXiv preprint arXiv:2406.02069, 2024

    arXiv 2024

  5. [4]

    Maskgit: Masked generative image transformer

    Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and William T Freeman. Maskgit: Masked generative image transformer. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11315–11325, 2022

    2022

  6. [5]

    PixArt-Sigma: Weak-to-strong training of diffusion transformer for 4k text-to-image generation

    Junsong Chen, Chongjian Ge, Enze Xie, Yue Wu, Lewei Yao, Xiaozhe Ren, Zhongdao Wang, Ping Luo, Huchuan Lu, and Zhenguo Li. PixArt-Sigma: Weak-to-strong training of diffusion transformer for 4k text-to-image generation. InECCV, pages 74–91. Springer, 2024

    work page 2024

  7. [6]

    Generative pretraining from pixels

    Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, and Ilya Sutskever. Generative pretraining from pixels. InInternational conference on machine learning, pages 1691–1703. PMLR, 2020

    2020

  8. [7]

    Collaborative decoding makes visual auto-regressive modeling efficient.arXiv preprint arXiv:2411.17787, 2024

    Zigeng Chen, Xinyin Ma, Gongfan Fang, and Xinchao Wang. Collaborative decoding makes visual auto-regressive modeling efficient.arXiv preprint arXiv:2411.17787, 2024

    arXiv 2024

Show all 81 references
  1. [8]

    Taming transformers for high-resolution image synthesis

    Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12873–12883, 2021

    2021

  2. [9]

    Tinyfusion: Diffusion transformers learned shallow.arXiv preprint arXiv:2412.01199, 2024

    Gongfan Fang, Kunjun Li, Xinyin Ma, and Xinchao Wang. Tinyfusion: Diffusion transformers learned shallow.arXiv preprint arXiv:2412.01199, 2024

    2024 arXiv

  3. [10]

    Ada-kv: Optimizing kv cache eviction by adaptive budget allocation for efficient llm inference.arXiv preprint arXiv:2407.11550, 2024

    Yuan Feng, Junlin Lv, Yukun Cao, Xike Xie, and S Kevin Zhou. Ada-kv: Optimizing kv cache eviction by adaptive budget allocation for efficient llm inference.arXiv preprint arXiv:2407.11550, 2024

    2024 arXiv

  4. [11]

    Not all heads matter: A head-level kv cache compression method with integrated retrieval and reasoning.arXiv preprint arXiv:2410.19258, 2024

    Yu Fu, Zefan Cai, Abedelkadir Asi, Wayne Xiong, Yue Dong, and Wen Xiao. Not all heads matter: A head-level kv cache compression method with integrated retrieval and reasoning.arXiv preprint arXiv:2410.19258, 2024

    2024

  5. [12]

    Model tells you what to discard: Adaptive kv cache compression for llms.arXiv preprint arXiv:2310.01801, 2023

    Suyu Ge, Yunan Zhang, Liyuan Liu, Minjia Zhang, Jiawei Han, and Jianfeng Gao. Model tells you what to discard: Adaptive kv cache compression for llms.arXiv preprint arXiv:2310.01801, 2023

    2023 arXiv

  6. [13]

    Geneval: An object-focused framework for evaluating text-to-image alignment

    Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. Geneval: An object-focused framework for evaluating text-to-image alignment. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors,Advances in Neural Information Processing Systems, volume 36, pag...

    2023

  7. [14]

    Dart: Denoising autoregressive transformer for scalable text-to-image generation

    Jiatao Gu, Yuyang Wang, Yizhe Zhang, Qihang Zhang, Dinghuai Zhang, Navdeep Jaitly, Josh Susskind, and Shuangfei Zhai. Dart: Denoising autoregressive transformer for scalable text-to-image generation. arXiv preprint arXiv:2410.08159, 2024

    arXiv

  8. [15]

    Fastvar: Linear visual autoregressive modeling via cached token pruning.arXiv preprint arXiv:2503.23367, 2025

    Hang Guo, Yawei Li, Taolin Zhang, Jiangshan Wang, Tao Dai, Shu-Tao Xia, and Luca Benini. Fastvar: Linear visual autoregressive modeling via cached token pruning.arXiv preprint arXiv:2503.23367, 2025

    2025 arXiv

  9. [16]

    Infinity: Scaling bitwise autoregressive modeling for high-resolution image synthesis, 2024

    Jian Han, Jinlai Liu, Yi Jiang, Bin Yan, Yuqi Zhang, Zehuan Yuan, Bingyue Peng, and Xiaobing Liu. Infinity: Scaling bitwise autoregressive modeling for high-resolution image synthesis, 2024

    2024

  10. [17]

    Zipcache: Accurate and efficient kv cache quantization with salient token identification.arXiv preprint arXiv:2405.14256, 2024

    Yefei He, Luoming Zhang, Weijia Wu, Jing Liu, Hong Zhou, and Bohan Zhuang. Zipcache: Accurate and efficient kv cache quantization with salient token identification.arXiv preprint arXiv:2405.14256, 2024

    2024 arXiv

  11. [18]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium.Advances in neural information processing systems, 30, 2017

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium.Advances in neural information processing systems, 30, 2017

    2017

  12. [19]

    Distilling the knowledge in a neural network.arXiv preprint arXiv:1503.02531, 2(7), 2015

    Geoffrey Hinton, Oriol Vinyals, Jeff Dean, et al. Distilling the knowledge in a neural network.arXiv preprint arXiv:1503.02531, 2(7), 2015

    2015 arXiv

  13. [20]

    Ella: Equip diffusion models with llm for enhanced semantic alignment.arXiv preprint arXiv:2403.05135, 2024

    Xiwei Hu, Rui Wang, Yixiao Fang, Bin Fu, Pei Cheng, and Gang Yu. Ella: Equip diffusion models with llm for enhanced semantic alignment.arXiv preprint arXiv:2403.05135, 2024

    2024 arXiv

  14. [21]

    Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024. 10

    2024 arXiv

  15. [22]

    Gear: An efficient kv cache compression recipefor near-lossless generative inference of llm.arXiv preprint arXiv:2403.05527, 2024

    Hao Kang, Qingru Zhang, Souvik Kundu, Geonhwa Jeong, Zaoxing Liu, Tushar Krishna, and Tuo Zhao. Gear: An efficient kv cache compression recipefor near-lossless generative inference of llm.arXiv preprint arXiv:2403.05527, 2024

    2024 arXiv

  16. [23]

    Scalable autoregressive image generation with mamba.arXiv preprint arXiv:2408.12245, 2024

    Haopeng Li, Jinyue Yang, Kexin Wang, Xuerui Qiu, Yuhong Chou, Xin Li, and Guoqi Li. Scalable autoregressive image generation with mamba.arXiv preprint arXiv:2408.12245, 2024

    2024

  17. [24]

    Svdquant: Absorbing outliers by low-rank components for 4-bit diffusion models.arXiv preprint arXiv:2411.05007, 2024

    Muyang Li*, Yujun Lin*, Zhekai Zhang*, Tianle Cai, Xiuyu Li, Junxian Guo, Enze Xie, Chenlin Meng, Jun-Yan Zhu, and Song Han. Svdquant: Absorbing outliers by low-rank components for 4-bit diffusion models.arXiv preprint arXiv:2411.05007, 2024

    2024

  18. [25]

    Faster diffusion: Rethinking the role of unet encoder in diffusion models.arXiv preprint arXiv:2312.09608, 2023

    Senmao Li, Taihang Hu, Fahad Shahbaz Khan, Linxuan Li, Shiqi Yang, Yaxing Wang, Ming-Ming Cheng, and Jian Yang. Faster diffusion: Rethinking the role of unet encoder in diffusion models.arXiv preprint arXiv:2312.09608, 2023

    2023 arXiv

  19. [26]

    Autoregressive image generation without vector quantization.arXiv preprint arXiv:2406.11838, 2024

    Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He. Autoregressive image generation without vector quantization.arXiv preprint arXiv:2406.11838, 2024

    2024 arXiv

  20. [27]

    Controlvar: Exploring controllable visual autoregressive modeling.arXiv preprint arXiv:2406.09750, 2024

    Xiang Li, Kai Qiu, Hao Chen, Jason Kuen, Zhe Lin, Rita Singh, and Bhiksha Raj. Controlvar: Exploring controllable visual autoregressive modeling.arXiv preprint arXiv:2406.09750, 2024

    2024 arXiv

  21. [28]

    Q-diffusion: Quantizing diffusion models

    Xiuyu Li, Yijiang Liu, Long Lian, Huanrui Yang, Zhen Dong, Daniel Kang, Shanghang Zhang, and Kurt Keutzer. Q-diffusion: Quantizing diffusion models. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 17535–17545, 2023

    2023

  22. [29]

    Snapfusion: Text-to-image diffusion model on mobile devices within two seconds.Advances in Neural Information Processing Systems, 36, 2024

    Yanyu Li, Huan Wang, Qing Jin, Ju Hu, Pavlo Chemerys, Yun Fu, Yanzhi Wang, Sergey Tulyakov, and Jian Ren. Snapfusion: Text-to-image diffusion model on mobile devices within two seconds.Advances in Neural Information Processing Systems, 36, 2024

    2024

  23. [30]

    Snapkv: Llm knows what you are looking for before generation.arXiv preprint arXiv:2404.14469, 2024

    Yuhong Li, Yingbing Huang, Bowen Yang, Bharat Venkitesh, Acyr Locatelli, Hanchen Ye, Tianle Cai, Patrick Lewis, and Deming Chen. Snapkv: Llm knows what you are looking for before generation.arXiv preprint arXiv:2404.14469, 2024

    2024 arXiv

  24. [31]

    Controlar: Controllable image generation with autoregressive models

    Zongming Li, Tianheng Cheng, Shoufa Chen, Peize Sun, Haocheng Shen, Longjin Ran, Xiaoxin Chen, Wenyu Liu, and Xinggang Wang. Controlar: Controllable image generation with autoregressive models. arXiv preprint arXiv:2410.02705, 2024

    arXiv

  25. [32]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. InComputer vision–ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings,...

    2014

  26. [33]

    Minicache: Kv cache compression in depth dimension for large language models.arXiv preprint arXiv:2405.14366, 2024

    Akide Liu, Jing Liu, Zizheng Pan, Yefei He, Gholamreza Haffari, and Bohan Zhuang. Minicache: Kv cache compression in depth dimension for large language models.arXiv preprint arXiv:2405.14366, 2024

    2024 arXiv

  27. [34]

    Lumina-mgpt: Illuminate flexible photorealistic text-to-image generation with multimodal generative pretraining.arXiv preprint arXiv:2408.02657, 2024

    Dongyang Liu, Shitian Zhao, Le Zhuo, Weifeng Lin, Yu Qiao, Hongsheng Li, and Peng Gao. Lumina-mgpt: Illuminate flexible photorealistic text-to-image generation with multimodal generative pretraining.arXiv preprint arXiv:2408.02657, 2024

    2024 arXiv

  28. [35]

    Scissorhands: Exploiting the persistence of importance hypothesis for llm kv cache compression at test time.Advances in Neural Information Processing Systems, 36, 2024

    Zichang Liu, Aditya Desai, Fangshuo Liao, Weitao Wang, Victor Xie, Zhaozhuo Xu, Anastasios Kyrillidis, and Anshumali Shrivastava. Scissorhands: Exploiting the persistence of importance hypothesis for llm kv cache compression at test time.Advances in Neural Information Processi...

    2024

  29. [36]

    Kivi: A tuning-free asymmetric 2bit quantization for kv cache.arXiv preprint arXiv:2402.02750, 2024

    Zirui Liu, Jiayi Yuan, Hongye Jin, Shaochen Zhong, Zhaozhuo Xu, Vladimir Braverman, Beidi Chen, and Xia Hu. Kivi: A tuning-free asymmetric 2bit quantization for kv cache.arXiv preprint arXiv:2402.02750, 2024

    2024 arXiv

  30. [37]

    Latent consistency models: Synthesizing high-resolution images with few-step inference.arXiv preprint arXiv:2310.04378, 2023

    Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hang Zhao. Latent consistency models: Synthesizing high-resolution images with few-step inference.arXiv preprint arXiv:2310.04378, 2023

    2023 arXiv

  31. [38]

    Open-magvit2: An open- source project toward democratizing auto-regressive visual generation.arXiv preprint arXiv:2409.04410, 2024

    Zhuoyan Luo, Fengyuan Shi, Yixiao Ge, Yujiu Yang, Limin Wang, and Ying Shan. Open-magvit2: An open- source project toward democratizing auto-regressive visual generation.arXiv preprint arXiv:2409.04410, 2024

    2024 arXiv

  32. [39]

    Accelerating diffusion models via early stop of the diffusion process.arXiv preprint arXiv:2205.12524, 2022

    Zhaoyang Lyu, Xudong Xu, Ceyuan Yang, Dahua Lin, and Bo Dai. Accelerating diffusion models via early stop of the diffusion process.arXiv preprint arXiv:2205.12524, 2022

    2022 arXiv

  33. [40]

    Star: Scale-wise text-to-image generation via auto-regressive representations.arXiv preprint arXiv:2406.10797, 2024

    Xiaoxiao Ma, Mohan Zhou, Tao Liang, Yalong Bai, Tiejun Zhao, Huaian Chen, and Yi Jin. Star: Scale-wise text-to-image generation via auto-regressive representations.arXiv preprint arXiv:2406.10797, 2024

    2024 arXiv

  34. [41]

    Deepcache: Accelerating diffusion models for free.arXiv preprint arXiv:2312.00858, 2023

    Xinyin Ma, Gongfan Fang, and Xinchao Wang. Deepcache: Accelerating diffusion models for free.arXiv preprint arXiv:2312.00858, 2023

    2023 arXiv

  35. [42]

    Transformers are multi-state rnns.arXiv preprint arXiv:2401.06104, 2024

    Matanel Oren, Michael Hassid, Yossi Adi, and Roy Schwartz. Transformers are multi-state rnns.arXiv preprint arXiv:2401.06104, 2024. 11

    2024 arXiv

  36. [43]

    SDXL: Improving latent diffusion models for high-resolution image synthesis.arXiv preprint arXiv:2307.01952, 2023

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. SDXL: Improving latent diffusion models for high-resolution image synthesis.arXiv preprint arXiv:2307.01952, 2023

    2023 arXiv

  37. [45]

    Efficient autoregressive audio modeling via next-scale prediction.arXiv preprint arXiv:2408.09027, 2024

    Kai Qiu, Xiang Li, Hao Chen, Jie Sun, Jinglu Wang, Zhe Lin, Marios Savvides, and Bhiksha Raj. Efficient autoregressive audio modeling via next-scale prediction.arXiv preprint arXiv:2408.09027, 2024

    2024 arXiv

  38. [46]

    On the efficacy of eviction policy for key-value constrained generative language model inference.arXiv preprint arXiv:2402.06262, 2024

    Siyu Ren and Kenny Q Zhu. On the efficacy of eviction policy for key-value constrained generative language model inference.arXiv preprint arXiv:2402.06262, 2024

    2024 arXiv

  39. [47]

    Progressive distillation for fast sampling of diffusion models.arXiv preprint arXiv:2202.00512, 2022

    Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models.arXiv preprint arXiv:2202.00512, 2022

    2022 arXiv

  40. [48]

    Adversarial diffusion distillation

    Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial diffusion distillation. arXiv preprint arXiv:2311.17042, 2023

    2023 arXiv

  41. [49]

    Post-training quantization on diffusion models

    Yuzhang Shang, Zhihang Yuan, Bin Xie, Bingzhe Wu, and Yan Yan. Post-training quantization on diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1972–1981, 2023

    1972

  42. [50]

    Frdiff: Feature reuse for exquisite zero-shot acceleration of diffusion models.arXiv preprint arXiv:2312.03517, 2023

    Junhyuk So, Jungwon Lee, and Eunhyeok Park. Frdiff: Feature reuse for exquisite zero-shot acceleration of diffusion models.arXiv preprint arXiv:2312.03517, 2023

    2023 arXiv

  43. [52]

    Autore- gressive model beats diffusion: Llama for scalable image generation.arXiv preprint arXiv:2406.06525, 2024

    Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autore- gressive model beats diffusion: Llama for scalable image generation.arXiv preprint arXiv:2406.06525, 2024

    2024 arXiv

  44. [53]

    HART: Efficient visual generation with hybrid autoregressive transformer

    Haotian Tang, Yecheng Wu, Shang Yang, Enze Xie, Junsong Chen, Junyu Chen, Zhuoyang Zhang, Han Cai, Yao Lu, and Song Han. HART: Efficient visual generation with hybrid autoregressive transformer. arXiv preprint arXiv:2410.10812, 2024

    arXiv

  45. [54]

    Visual autoregressive modeling: Scalable image generation via next-scale prediction.Advances in neural information processing systems, 37:84839–84865, 2024

    Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction.Advances in neural information processing systems, 37:84839–84865, 2024

    2024

  46. [55]

    Conditional image generation with pixelcnn decoders.Advances in neural information processing systems, 29, 2016

    Aaron Van den Oord, Nal Kalchbrenner, Lasse Espeholt, Oriol Vinyals, Alex Graves, et al. Conditional image generation with pixelcnn decoders.Advances in neural information processing systems, 29, 2016

    2016

  47. [56]

    Neural discrete representation learning.Advances in neural information processing systems, 30, 2017

    Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning.Advances in neural information processing systems, 30, 2017

    2017

  48. [57]

    D2o: Dynamic discriminative operations for efficient generative inference of large language models.arXiv preprint arXiv:2406.13035, 2024

    Zhongwei Wan, Xinjian Wu, Yu Zhang, Yi Xin, Chaofan Tao, Zhihong Zhu, Xin Wang, Siqi Luo, Jing Xiong, and Mi Zhang. D2o: Dynamic discriminative operations for efficient generative inference of large language models.arXiv preprint arXiv:2406.13035, 2024

    2024 arXiv

  49. [58]

    Look-m: Look-once optimization in kv cache for efficient multimodal long-context inference.arXiv preprint arXiv:2406.18139, 2024

    Zhongwei Wan, Ziang Wu, Che Liu, Jinfa Huang, Zhihong Zhu, Peng Jin, Longyue Wang, and Li Yuan. Look-m: Look-once optimization in kv cache for efficient multimodal long-context inference.arXiv preprint arXiv:2406.18139, 2024

    2024 arXiv

  50. [59]

    Emu3: Next-token prediction is all you need.arXiv preprint arXiv:2409.18869, 2024

    Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, et al. Emu3: Next-token prediction is all you need.arXiv preprint arXiv:2409.18869, 2024

    2024 arXiv

  51. [60]

    Cache me if you can: Accelerating diffusion models through block caching.arXiv preprint arXiv:2312.03209, 2023

    Felix Wimbauer, Bichen Wu, Edgar Schoenfeld, Xiaoliang Dai, Ji Hou, Zijian He, Artsiom Sanakoyeu, Peizhao Zhang, Sam Tsai, Jonas Kohler, et al. Cache me if you can: Accelerating diffusion models through block caching.arXiv preprint arXiv:2312.03209, 2023

    2023 arXiv

  52. [61]

    Janus: Decoupling visual encoding for unified multimodal understanding and generation.arXiv preprint arXiv:2410.13848, 2024

    Chengyue Wu, Xiaokang Chen, Zhiyu Wu, Yiyang Ma, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, Chong Ruan, et al. Janus: Decoupling visual encoding for unified multimodal understanding and generation.arXiv preprint arXiv:2410.13848, 2024

    arXiv

  53. [62]

    VILA-U: A unified foundation model integrating visual understanding and generation.arXiv preprint arXiv:2409.04429, 2024

    Yecheng Wu, Zhuoyang Zhang, Junyu Chen, Haotian Tang, Dacheng Li, Yunhao Fang, Ligeng Zhu, Enze Xie, Hongxu Yin, Li Yi, et al. VILA-U: A unified foundation model integrating visual understanding and generation.arXiv preprint arXiv:2409.04429, 2024

    2024 arXiv

  54. [63]

    Efficient streaming language models with attention sinks.arXiv, 2023

    Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. Efficient streaming language models with attention sinks.arXiv, 2023

    2023

  55. [64]

    Show-o: One single transformer to unify multimodal understanding and generation.arXiv preprint arXiv:2408.12528, 2024

    Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single transformer to unify multimodal understanding and generation.arXiv preprint arXiv:2408.12528, 2024. 12

    2024 arXiv

  56. [65]

    Litevar: Compressing visual autoregressive modelling with efficient attention and quantization.arXiv preprint arXiv:2411.17178, 2024

    Rui Xie, Tianchen Zhao, Zhihang Yuan, Rui Wan, Wenxi Gao, Zhenhua Zhu, Xuefei Ning, and Yu Wang. Litevar: Compressing visual autoregressive modelling with efficient attention and quantization.arXiv preprint arXiv:2411.17178, 2024

    2024 arXiv

  57. [66]

    Pyramidinfer: Pyramid kv cache compression for high-throughput llm inference.arXiv preprint arXiv:2405.12532, 2024

    Dongjie Yang, XiaoDong Han, Yan Gao, Yao Hu, Shilin Zhang, and Hai Zhao. Pyramidinfer: Pyramid kv cache compression for high-throughput llm inference.arXiv preprint arXiv:2405.12532, 2024

    2024 arXiv

  58. [67]

    Hash3d: Training-free acceleration for 3d generation.arXiv preprint arXiv:2404.06091, 2024

    Xingyi Yang and Xinchao Wang. Hash3d: Training-free acceleration for 3d generation.arXiv preprint arXiv:2404.06091, 2024

    2024 arXiv

  59. [68]

    Diffusion probabilistic model made slim

    Xingyi Yang, Daquan Zhou, Jiashi Feng, and Xinchao Wang. Diffusion probabilistic model made slim. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22552– 22562, 2023

    2023

  60. [69]

    Car: Controllable autoregressive modeling for visual generation.arXiv preprint arXiv:2410.04671, 2024

    Ziyu Yao, Jialin Li, Yifeng Zhou, Yong Liu, Xi Jiang, Chengjie Wang, Feng Zheng, Yuexian Zou, and Lei Li. Car: Controllable autoregressive modeling for visual generation.arXiv preprint arXiv:2410.04671, 2024

    2024 arXiv

  61. [70]

    One-step diffusion with distribution matching distillation.arXiv preprint arXiv:2311.18828, 2023

    Tianwei Yin, Michaël Gharbi, Richard Zhang, Eli Shechtman, Fredo Durand, William T Freeman, and Taesung Park. One-step diffusion with distribution matching distillation.arXiv preprint arXiv:2311.18828, 2023

    2023 arXiv

  62. [71]

    Wkvquant: Quan- tizing weight and key/value cache for large language models gains more.arXiv preprint arXiv:2402.12065, 2024

    Yuxuan Yue, Zhihang Yuan, Haojie Duanmu, Sifan Zhou, Jianlong Wu, and Liqiang Nie. Wkvquant: Quan- tizing weight and key/value cache for large language models gains more.arXiv preprint arXiv:2402.12065, 2024

    2024 arXiv

  63. [72]

    Resshift: Efficient diffusion model for image super-resolution by residual shifting.Advances in Neural Information Processing Systems, 36, 2024

    Zongsheng Yue, Jianyi Wang, and Chen Change Loy. Resshift: Efficient diffusion model for image super-resolution by residual shifting.Advances in Neural Information Processing Systems, 36, 2024

    2024

  64. [73]

    Laptop-diff: Layer pruning and normalized distillation for compressing diffusion models.arXiv preprint arXiv:2404.11098, 2024

    Dingkun Zhang, Sijia Li, Chen Chen, Qingsong Xie, and Haonan Lu. Laptop-diff: Layer pruning and normalized distillation for compressing diffusion models.arXiv preprint arXiv:2404.11098, 2024

    2024 arXiv

  65. [74]

    G3pt: Unleash the power of autoregressive modeling in 3d generation via cross-scale querying transformer.arXiv preprint arXiv:2409.06322, 2024

    Jinzhi Zhang, Feng Xiong, and Mu Xu. G3pt: Unleash the power of autoregressive modeling in 3d generation via cross-scale querying transformer.arXiv preprint arXiv:2409.06322, 2024

    2024 arXiv

  66. [75]

    Var-clip: Text-to- image generator with visual auto-regressive modeling.arXiv preprint arXiv:2408.01181, 2024

    Qian Zhang, Xiangzi Dai, Ninghua Yang, Xiang An, Ziyong Feng, and Xingyu Ren. Var-clip: Text-to- image generator with visual auto-regressive modeling.arXiv preprint arXiv:2408.01181, 2024

    2024 arXiv

  67. [76]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 586–595, 2018

    2018

  68. [77]

    Cross-attention makes inference cumbersome in text-to-image diffusion models.arXiv preprint arXiv:2404.02747, 2024

    Wentian Zhang, Haozhe Liu, Jinheng Xie, Francesco Faccio, Mike Zheng Shou, and Jürgen Schmid- huber. Cross-attention makes inference cumbersome in text-to-image diffusion models.arXiv preprint arXiv:2404.02747, 2024

    2024 arXiv

  69. [78]

    Cam: Cache merging for memory-efficient llms inference

    Yuxin Zhang, Yuxuan Du, Gen Luo, Yunshan Zhong, Zhenyu Zhang, Shiwei Liu, and Rongrong Ji. Cam: Cache merging for memory-efficient llms inference. InForty-first International Conference on Machine Learning, 2024

    2024

  70. [79]

    H2o: Heavy-hitter oracle for efficient generative inference of large language models.Advances in Neural Information Processing Systems, 36:34661–34710, 2023

    Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Ré, Clark Barrett, et al. H2o: Heavy-hitter oracle for efficient generative inference of large language models.Advances in Neural Information Processing System...

    2023

  71. [80]

    Real-time video generation with pyramid attention broadcast.arXiv preprint arXiv:2408.12588, 2024

    Xuanlei Zhao, Xiaolong Jin, Kai Wang, and Yang You. Real-time video generation with pyramid attention broadcast.arXiv preprint arXiv:2408.12588, 2024

    2024 arXiv

  72. [81]

    Mobilediffusion: Subsecond text-to-image generation on mobile devices.arXiv preprint arXiv:2311.16567, 2023

    Yang Zhao, Yanwu Xu, Zhisheng Xiao, and Tingbo Hou. Mobilediffusion: Subsecond text-to-image generation on mobile devices.arXiv preprint arXiv:2311.16567, 2023

    2023 arXiv

  73. [82]

    Transfusion: Predict the next token and diffuse images with one multi-modal model.arXiv preprint arXiv:2408.11039, 2024

    Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke Zettlemoyer, and Omer Levy. Transfusion: Predict the next token and diffuse images with one multi-modal model.arXiv preprint arXiv:2408.11039, 2024. 13 Technical ...

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.