Pith. sign in

REVIEW 3 major objections 6 minor 2 cited by

SeerAttention-R: Sparse Attention Adaptation for Long Reasoning

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A learned gate predicts which key-value blocks a reasoning model needs during decoding, keeping reasoning accuracy near-lossless at a 4K token budget and enabling up to 9x kernel speedups.

desk verdict A sensible SeerAttention extension with a real kernel contribution, but the headline near-lossless accuracy claim is not statistically supported by the reported numbers. read the letter →

arxiv 2506.08889 v1 pith:QN5FTDJA submitted 2025-06-10 cs.LG cs.AI

Yizhao Gao , Shuming Guo , Shijie Cao , Yuqing Xia , Yu Cheng , Lei Wang , Lingxiao Ma , Yutao Sun
show 7 more authors
Tianzhu Ye Li Dong Hayden Kwok-Hay So Yu Hua Ting Cao Fan Yang Mao Yang
This is my paper
classification cs.LGcs.AI
keywords sparseattentionreasoningmodelsKVcacheblocksparsityself-distillationgatelong-contextinferencedecoding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SeerAttention-R asks whether a reasoning model can generate long chains of thought while reading only a small slice of its key-value memory on each decoding step. The paper's answer is yes: a lightweight plug-in gate, distilled on just 0.4B tokens of math reasoning data, learns to predict which blocks of past keys and values matter, and a block-sparse decoding kernel skips the rest. Across four reasoning models and four benchmarks, the method stays near-lossless at a 4K token budget with coarse block sizes of 64 or 128, where a training-free sparse baseline loses accuracy. Because decoding is memory-bandwidth bound, skipping key-value blocks translates directly into speed: the kernel approaches the theoretical limit and reaches up to 9x over FlashAttention-3 on an H100 at 90% sparsity.

What carries the argument

The load-bearing object is the attention gate (AttnGate), a small set of linear layers with position embeddings that takes the current query and pooled key blocks and outputs a score per key-value block. The gate is trained to imitate a max-pooled version of the original model's own attention map via KL divergence, so the ground truth comes from the model itself and no base weights are changed. During decoding, the gate selects the top-scoring blocks, shares one selection across the query heads of each grouped-query attention group, and a cached compressed-key representation updates only once per block, keeping gate overhead below 1% of the KV cache size.

What would settle it

Run the distilled gate on a held-out reasoning domain such as code generation or multi-step tool use at a 4K token budget; if accuracy drops well below the dense baseline while oracle block selection under the same budget stays lossless, the gate has not learned a transferable sparsity predictor. A cheaper probe is to re-train the gate with average-pooled attention scores instead of max-pooled scores as ground truth; if the accuracy gap at 4K collapses, the method depends specifically on the max-pooled oracle signal.

Watch

Extended reading notes

Core claim

The central claim is that attention in reasoning models is sparse at the block level, and that the sparse set can be predicted rather than discovered at decode time. An oracle experiment shows that selecting key-value blocks by ground-truth attention is lossless with a 2K token budget on Qwen3-14B, and the learned gate nearly reproduces that at 4K, with the gap shrinking as model size grows. The paper also claims the gate stays accurate at block sizes 64 and 128, which keeps the sparse scheme hardware-efficient, and that its custom sparse decoding kernel achieves near-theoretical speedups of up to 9x over FlashAttention-3 at 90% sparsity. The result is a post-training adapter: only the gate's parameters are trained, while the original model weights stay frozen.

Load-bearing premise

The method assumes that the highest attention score inside each key-value block is enough to tell whether that block matters, and that a gate trained on 0.4B math tokens will keep making that call correctly on new reasoning problems.

Editorial extensions

If this is right

  • Reasoning models can decode long traces at a 4K token budget with near-lossless accuracy, so the practical cost of reading the KV cache becomes roughly independent of sequence length.
  • Because larger models tolerate sparsity better, further scaling of reasoning models should widen the accuracy gap between learned sparse attention and training-free heuristics.
  • The kernel speedup grows with batch size and sequence length and approaches the memory-bandwidth bound, making the method most attractive for serving many long-context requests.
  • Since the compressed key cache is tiny, the main KV cache can be offloaded to CPU memory or storage and only selected blocks fetched on demand, easing GPU memory limits for long reasoning.
  • Inaccurate sparse selection inflates generation length, so the accuracy of the gate itself is part of the efficiency story; near-lossless selection avoids the token-length blowup seen with weaker baselines.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves open that the gate's max-pooled ground truth may be a conservative oracle; a gate trained on a stricter or richer signal, such as per-head block scores before grouped-query pooling, might push the lossless budget from 4K toward the oracle's 2K.
  • The threshold sparsifier's smooth activation curve hints that an adaptive per-head budget could be learned, removing the fixed token-budget hyperparameter and automatically trading accuracy against speed as task difficulty varies.
  • Pairing the gate with speculative decoding or multi-token prediction would restore query-level parallelism that decoding lacks, which the paper names as future work and which could also let one gate serve both prefill and decoding.
  • If the distillation data were expanded beyond math, the same 0.4B-token recipe might transfer to code and agentic reasoning traces, whose block-level attention structure may differ; that is a direct test of the method's generality.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. SeerAttention-R proposes a post-training sparse attention method for the decoding phase of reasoning LLMs. A lightweight gate is trained by self-distillation to predict important key-value blocks at coarse granularity (block sizes 64/128), using max/min/average pooling of K and a GQA-aware query reduction. The paper reports near-lossless reasoning accuracy with a 4K token budget on AIME benchmarks across Qwen3-4B/8B/14B and DeepSeek-R1-Distill-Qwen-14B, claims consistent improvement over Quest, and presents a TileLang sparse flash-decoding kernel that achieves up to 9x speedup over FlashAttention-3 on H100 at 90% sparsity. Code is released publicly.

Significance. If the accuracy and efficiency claims hold, this is a practically valuable contribution: it shows that a small, model-agnostic gate can identify the sparse attention structure of already-trained reasoning models, enabling large-block sparse decoding without modifying base weights. The oracle-sparsity experiments (Figure 4) provide credible evidence that attention in reasoning models is indeed sparse enough to be exploited. The kernel engineering is a genuine strength: the TileLang implementation reaches high fractions of the theoretical I/O-bound speedup, and the comparison with Triton and FA3 is informative. However, the headline accuracy claim rests on point estimates with very small sample sizes, and the Quest baseline is evaluated in a configuration the paper itself shows is disadvantageous to Quest. These issues must be addressed before the central claims can be considered robust.

major comments (3)
  1. [§4.1, Figure 5, Table 1] The central near-lossless accuracy claim at 4K token budget is not statistically supported by the reported numbers. For AIME24/25 the evaluation uses only 64 samples per condition; the binomial standard error for the Table 1 Qwen3-8B dense accuracy of 74.5% is about 5.4 points, so the 2.2-point gap to the 4K-budget SeerAttention-R result (72.3%) is far within sampling noise. For MATH-500 (n=8) and GPQA-Diamond (n=16), the standard errors are even larger, making double-digit differences statistically indistinguishable. The paper should provide confidence intervals, repeated-seed variation, or a pre-specified equivalence margin for "near-lossless", and should state explicitly whether the claim is intended as a statistical equivalence or merely a mean-difference statement.
  2. [§4.3, §5.2, Figure 7] The Quest baseline in the main comparison is configured in a way that is known to disadvantage it: block size 64 and all layers sparse, whereas Quest's default is block size 16 with the first two layers dense. The paper's own ablation in §5.2 shows that using hybrid dense first-two-layers substantially improves Quest, and §5.1 shows Quest's accuracy degrades as block size increases. Because the claim "SeerAttention-R consistently outperforms Quest" is a headline result, the paper should also report Quest under its default configuration (or at least the hybrid-dense variant), or provide a clear justification for why the chosen configuration is the appropriate basis for comparison.
  3. [§4.2 and §5 (absent analysis)] The paper never directly measures how well the learned gate approximates the oracle block selection that the method is designed to imitate. The oracle experiment (Figure 4) shows that ideal selection preserves accuracy at low budgets, but the learned gate may be far from ideal, and the accuracy results alone do not distinguish gate-quality effects from the base model's robustness to missing blocks. The authors should report gate-vs-oracle agreement metrics (e.g., recall/precision of selected blocks or correlation with ground-truth block scores) across layers and heads, especially because the 4K gate budget is needed where the oracle is already lossless at 2K. Such an analysis would also inform why the gate's selection degrades and whether the gap could be closed.
minor comments (6)
  1. [§1 and §4.4] The reported maximum speedup appears inconsistently: §1 states "up to 8.6x" while the abstract and §4.4 state "up to 9x". Please unify the numbers and specify the exact configuration that achieves the maximum.
  2. [§5.1 and Figure 7] The text says block size 16 was excluded from experiments due to out-of-memory errors, but Figure 7 shows measurements at block size 16 for both Quest and SeerAttention-R. Please reconcile this contradiction.
  3. [§4.1 and §4.3] The paper notes that all generations reached the 32K token cap, meaning many outputs are truncated. This truncation could affect the effective token budgets and should be discussed as a potential confound for the accuracy-budget relationship.
  4. [§4.4, Figure 6] The claim of "near-theoretical" speedup would be easier to verify if Figure 6 included a theoretical speedup curve or if the formula for the theoretical bound were given explicitly.
  5. [§3.2 and §4.4] The overhead of the K Compression Cache and the AttnGate is described qualitatively but not quantified in the kernel benchmark. Please state explicitly whether the reported speedups exclude gate computation and cache updates, and if so, give a rough breakdown of their end-to-end cost.
  6. [§2.2, Eq. (1a)] The reshape operation in Eq. (1a) is not fully specified; clarifying the intended dimension ordering (e.g., [batch, heads, seq, group_size*d] to [batch, heads_kv, seq, dgate]) would improve reproducibility.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: the gate is a supervised fit to the target model's attention maps and accuracy is measured on external benchmarks; self-citations are descriptive, not load-bearing.

full rationale

SeerAttention-R's claimed derivation chain is self-contained. The AttnGate is trained with a KL divergence loss to imitate block-level max-pooled attention scores generated by the frozen original model (Eq. 1, Section 2.3), and the resulting sparse masks are evaluated by downstream pass@1 accuracy on AIME24/AIME25, MATH-500, and GPQA-Diamond, which are external benchmarks not used to fit the gate. The oracle-sparsity experiment (Section 4.2) independently establishes that selecting KV blocks by true attention scores preserves accuracy, providing an upper bound rather than a circular target. The self-citations to SeerAttention [19] and Rectified Sparse Attention [56] describe inherited design choices and future directions, but the paper reproduces the gating equations and the training kernel, and the central near-lossless accuracy claim rests on new experiments in Figures 4-5 and Table 1, not on the cited prior work. No fitted parameter is renamed as a prediction: the gate outputs block scores, but the headline result is end-task accuracy on held-out benchmarks, not agreement with the training target. No uniqueness theorem is imported, no ansatz is smuggled in via citation, and no known result is repackaged under new coordinates. The small-sample point-estimate issue (n=64 for AIME, n=8 for MATH-500, n=16 for GPQA) is a statistical-reporting concern, not a circularity concern. Overall circularity: none beyond descriptive self-citation.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method introduces no new physical or conceptual entities; the K Compression Cache and AttnGate are engineered components. The central commitments are the gate's compression architecture and the distillation target. The listed free parameters are operational choices that the reported accuracy claims depend on, and the axioms are unproved assumptions about the teacher signal, the compression, GQA sharing, and domain transfer.

free parameters (4)
  • block size = 64 (default; 32 and 128 tested)
    Block granularity sets the sparsity unit; 64 was chosen after the oracle sweep in Section 4.2. Larger blocks reduce gate overhead but increase the risk of dropping important tokens, and block size 16 was excluded due to OOM during training.
  • token budget for near-lossless claim = 4K tokens for AIME24/25; 2K for MATH-500/GPQA
    The near-lossless claim is tied to this budget. At 2K budgets accuracy visibly degrades, and at 8K gaps shrink, so the headline result depends on the chosen operating point.
  • training tokens = 0.4B tokens, 800 steps, batch size 16
    The method is reported to need only this much distillation data, but generalization to other domains is not demonstrated and the training distribution may overlap with evaluation benchmarks.
  • threshold values in ablation = 2e-3, 3e-3, 4e-3, 5e-3, 6e-3
    Hand-picked thresholds used in Section 5.3 to trace the sparsity-accuracy tradeoff; they are evaluation knobs, not learned parameters.
assumptions (4)
  • domain assumption The original model's block-level max-pooled attention scores are a sufficient teacher for task-preserving sparsity.
    Used to generate ground truth in Section 2.3 and Figure 2a. If maxpooling misses blocks whose joint removal is harmful, the gate learns the wrong target even when oracle sparsity exists.
  • ad hoc to paper A compressed K representation (max/min/avg pooling plus a linear layer) retains enough information for block selection.
    This is the design in Eq. 1b and Section 2.2. No proof of sufficiency is given; only empirical support on math benchmarks is provided.
  • domain assumption All query heads in a GQA group can share the same sparsity mask without meaningful accuracy loss.
    Adopted from prior work (SAAP, NSA) and relied on for hardware efficiency. Section 5.1's robustness claim is conditional on this sharing.
  • domain assumption The gate trained on OpenR1-MATH-220K generalizes to the evaluation traces.
    Training uses only 0.4B math tokens, but evaluation includes AIME25 and GPQA. Cross-domain generalization is assumed, and no decontamination or domain-shift study is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SeerAttention-R: Sparse Attention Adaptation for Long Reasoning." pith.science (2026). https://pith.science/paper/QN5FTDJA

@misc{pith2026250608889,
  author       = {Pith},
  title        = {Pith review of: SeerAttention-R: Sparse Attention Adaptation for Long Reasoning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QN5FTDJA}},
  note         = {Machine review of arXiv:2506.08889}
}
read the original abstract

We introduce SeerAttention-R, a sparse attention framework specifically tailored for the long decoding of reasoning models. Extended from SeerAttention, SeerAttention-R retains the design of learning attention sparsity through a self-distilled gating mechanism, while removing query pooling to accommodate auto-regressive decoding. With a lightweight plug-in gating, SeerAttention-R is flexible and can be easily integrated into existing pretrained model without modifying the original parameters. We demonstrate that SeerAttention-R, trained on just 0.4B tokens, maintains near-lossless reasoning accuracy with 4K token budget in AIME benchmark under large sparse attention block sizes (64/128). Using TileLang, we develop a highly optimized sparse decoding kernel that achieves near-theoretical speedups of up to 9x over FlashAttention-3 on H100 GPU at 90% sparsity. Code is available at: https://github.com/microsoft/SeerAttention.

Figures

Figures reproduced from arXiv: 2506.08889 by the authors.

Figure 1
Figure 1. SeerAttention (Sparse Prefill) and SeerAttention-R (Sparse Decode). In SeerAttention-R, [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Training Diagram and Training Kernel of SeerAttention-R. (a) Self-distillation training of [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Inference Diagram of SeerAttention-R. During inference, a K Compression Cache is used [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Oracle Sparse Results of Qwen3-14B with block size 32, 64, 128. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Accuracy Results of Full Attention, SeerAttention-R, and Quest. The Quest sparse [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Kernel Speedup of our Block Sparse Flash-Decoding Kernel on H100 GPU. Our TileLang [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: AIME24 results using different block sizes with 4k token budget. SeerAttention-R achieves [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: AIME24 results of whether using dense attention in first two layers (Qwen3-4B). [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: Threshold vs. Token Budget. Results are obtained using Qwen3-4B models on AIME24 [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. PhyCheck: Fine-Grained Evidence-Grounded Dataset for Physical Law Understanding in Video-LLMs

    cs.CV 2026-08 conditional novelty 7.0 of 10

    PhyCheck is a 69,825-pair video QA benchmark that tests and improves Video-LLMs' ability to judge whether events obey physical laws, with fine-grained evidence questions and a context-sensitivity pilot.

  2. PIVOT: Efficient Query-Group Indexing for Token-Level Sparse Attention

    cs.CL 2026-07 conditional novelty 6.0 of 10

    Sharing one proxy full-prefix scan across nearby query groups matches dense DSA indexer accuracy while accelerating indexing up to 4× and end-to-end latency up to 1.6×.

Reference graph

Works this paper leans on

78 extracted references · 14 canonical work pages · cited by 2 Pith papers

  1. [1]

    URLhttps://github.com/tile-ai/tilelang

    TileLang. URLhttps://github.com/tile-ai/tilelang

    work page

  2. [2]

    Keyformer: Kv cache reduction through key tokens selection for efficient generative inference.Proceedings of Machine Learning and Systems, 6:114–127, 2024

    Muhammad Adnan, Akhil Arunkumar, Gaurav Jain, Prashant J Nair, Ilya Soloveychik, and Purushotham Kamath. Keyformer: Kv cache reduction through key tokens selection for efficient generative inference.Proceedings of Machine Learning and Systems, 6:114–127, 2024

    work page 2024

  3. [3]

    Gqa: Training generalized multi-query transformer models from multi-head checkpoints.arXiv preprint arXiv:2305.13245, 2023

    Joshua Ainslie, James Lee-Thorp, Michiel De Jong, Yury Zemlyanskiy, Federico Lebrón, and Sumit Sanghai. Gqa: Training generalized multi-query transformer models from multi-head checkpoints.arXiv preprint arXiv:2305.13245, 2023

    arXiv 2023

  4. [4]

    xlstm: Extended long short-term memory.arXiv preprint arXiv:2405.04517, 2024

    Maximilian Beck, Korbinian Pöppel, Markus Spanring, Andreas Auer, Oleksandra Prudnikova, Michael Kopp, Günter Klambauer, Johannes Brandstetter, and Sepp Hochreiter. xlstm: Extended long short-term memory.arXiv preprint arXiv:2405.04517, 2024

    arXiv 2024

  5. [5]

    Rocketkv: Accelerating long-context llm inference via two-stage kv cache compression.arXiv preprint arXiv:2502.14051, 2025

    Payman Behnam, Yaosheng Fu, Ritchie Zhao, Po-An Tsai, Zhiding Yu, and Alexey Tumanov. Rocketkv: Accelerating long-context llm inference via two-stage kv cache compression.arXiv preprint arXiv:2502.14051, 2025

    arXiv 2025

  6. [6]

    Longformer: The long-document transformer

    Iz Beltagy, Matthew E Peters, and Arman Cohan. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150, 2020

    arXiv 2004

  7. [7]

    Reducing transformer key-value cache size with cross-layer attention

    William Brandon, Mayank Mishra, Aniruddha Nrusimha, Rameswar Panda, and Jonathan Ragan-Kelley. Reducing transformer key-value cache size with cross-layer attention. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    work page 2024

  8. [8]

    R-kv: Redundancy-aware kv cache compression for training-free reasoning models acceleration.arXiv preprint arXiv:2505.24133, 2025

    Zefan Cai, Wen Xiao, Hanshi Sun, Cheng Luo, Yikai Zhang, Ke Wan, Yucheng Li, Yeyang Zhou, Li-Wen Chang, Jiuxiang Gu, Zhen Dong, Anima Anandkumar, Abedelkadir Asi, and Junjie Hu. R-kv: Redundancy-aware kv cache compression for training-free reasoning models acceleration.arXiv preprint arXiv:2505.24133, 2025

    arXiv 2025

Show all 78 references
  1. [9]

    Sepllm: Accelerate large language models by compressing one segment into one separator.arXiv preprint arXiv:2412.12094, 2024

    Guoxuan Chen, Han Shi, Jiawei Li, Yihang Gao, Xiaozhe Ren, Yimeng Chen, Xin Jiang, Zhenguo Li, Weiyang Liu, and Chao Huang. Sepllm: Accelerate large language models by compressing one segment into one separator.arXiv preprint arXiv:2412.12094, 2024

    2024 arXiv

  2. [10]

    Retroinfer: A vector-storage approach for scalable long-context llm inference, 2025

    Yaoqi Chen, Jinkai Zhang, Baotong Lu, Qianxi Zhang, Chengruidong Zhang, Jingjia Luo, Di Liu, Huiqiang Jiang, Qi Chen, Jing Liu, Bailu Ding, Xiao Yan, Jiawei Jiang, Chen Chen, Mingxing Zhang, Yuqing Yang, Fan Yang, and Mao Yang. Retroinfer: A vector-storage approach for scalabl...

    2025 arXiv

  3. [11]

    Magicpig: Lsh sampling for efficient llm generation.arXiv preprint arXiv:2410.16179, 2024

    Zhuoming Chen, Ranajoy Sadhukhan, Zihao Ye, Yang Zhou, Jianyu Zhang, Niklas Nolte, Yuandong Tian, Matthijs Douze, Leon Bottou, Zhihao Jia, et al. Magicpig: Lsh sampling for efficient llm generation.arXiv preprint arXiv:2410.16179, 2024

    arXiv

  4. [12]

    PipeThreader: Software-defined pipelining for efficient dnn execution

    Yu Cheng, Lei Wang, Yining Shi, Yuqing Xia, Lingxiao Ma, Jilong Xue, Yang Wang, Zhiwen Mo, Feiyang Chen, Fan Yang, Mao Yang, and Zhi Yang. PipeThreader: Software-defined pipelining for efficient dnn execution. In19th USENIX Symposium on Operating Systems De- sign and Implement...

    2025

  5. [13]

    Generating long sequences with sparse transformers.arXiv preprint arXiv:1904.10509, 2019

    Rewon Child, Scott Gray, Alec Radford, and Ilya Sutskever. Generating long sequences with sparse transformers.arXiv preprint arXiv:1904.10509, 2019

    1904 arXiv

  6. [14]

    Flashattention-2: Faster attention with better parallelism and work partitioning

    Tri Dao. Flashattention-2: Faster attention with better parallelism and work partitioning. 2023. URLhttps://arxiv.org/abs/2307.08691

    2023 arXiv

  7. [15]

    Transformers are ssms: Generalized models and efficient algorithms through structured state space duality.arXiv preprint arXiv:2405.21060, 2024

    Tri Dao and Albert Gu. Transformers are ssms: Generalized models and efficient algorithms through structured state space duality.arXiv preprint arXiv:2405.21060, 2024

    2024 arXiv

  8. [16]

    Hymba: A hybrid-head architecture for small language models.arXiv preprint arXiv:2411.13676, 2024

    Xin Dong, Yonggan Fu, Shizhe Diao, Wonmin Byeon, Zijia Chen, Ameya Sunil Mahabalesh- warkar, Shih-Yang Liu, Matthijs Van Keirsbilck, Min-Hung Chen, Yoshi Suhara, et al. Hymba: A hybrid-head architecture for small language models.arXiv preprint arXiv:2411.13676, 2024

    2024 arXiv

  9. [17]

    Open r1: A fully open reproduction of deepseek-r1, January 2025

    Hugging Face. Open r1: A fully open reproduction of deepseek-r1, January 2025. URL https://github.com/huggingface/open-r1

    2025

  10. [18]

    Moa: Mixture of sparse attention for automatic large language model compression.arXiv preprint arXiv:2406.14909, 2024

    Tianyu Fu, Haofeng Huang, Xuefei Ning, Genghan Zhang, Boju Chen, Tianqi Wu, Hongyi Wang, Zixiao Huang, Shiyao Li, Shengen Yan, et al. Moa: Mixture of sparse attention for automatic large language model compression.arXiv preprint arXiv:2406.14909, 2024

    2024

  11. [19]

    Seerattention: Learning intrinsic sparse attention in your llms.arXiv preprint arXiv:2410.13276, 2024

    Yizhao Gao, Zhichen Zeng, Dayou Du, Shijie Cao, Peiyuan Zhou, Jiaxing Qi, Junjie Lai, Hayden Kwok-Hay So, Ting Cao, Fan Yang, et al. Seerattention: Learning intrinsic sparse attention in your llms.arXiv preprint arXiv:2410.13276, 2024

    arXiv

  12. [20]

    Model tells you what to discard: Adaptive kv cache compression for llms.arXiv preprint arXiv:2310.01801, 2023

    Suyu Ge, Yunan Zhang, Liyuan Liu, Minjia Zhang, Jiawei Han, and Jianfeng Gao. Model tells you what to discard: Adaptive kv cache compression for llms.arXiv preprint arXiv:2310.01801, 2023

    2023 arXiv

  13. [21]

    Better & faster large language models via multi-token prediction.arXiv preprint arXiv:2404.19737, 2024

    Fabian Gloeckle, Badr Youbi Idrissi, Baptiste Rozière, David Lopez-Paz, and Gabriel Syn- naeve. Better & faster large language models via multi-token prediction.arXiv preprint arXiv:2404.19737, 2024

    2024 arXiv

  14. [22]

    Mamba: Linear-time sequence modeling with selective state spaces

    Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752, 2023

    2023 arXiv

  15. [23]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  16. [24]

    Omnikv: Dynamic context selection for efficient long-context llms

    Jitai Hao, Yuke Zhu, Tian Wang, Jun Yu, Xin Xin, Bo Zheng, Zhaochun Ren, and Sheng Guo. Omnikv: Dynamic context selection for efficient long-context llms. InThe Thirteenth International Conference on Learning Representations, 2025

    2025

  17. [25]

    Measuring massive multitask language understanding.arXiv preprint arXiv:2009.03300, 2020

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding.arXiv preprint arXiv:2009.03300, 2020

    2009 arXiv

  18. [26]

    Squeezed attention: Accelerating long context length llm inference.arXiv preprint arXiv:2411.09688, 2024

    Coleman Hooper, Sehoon Kim, Hiva Mohammadzadeh, Monishwaran Maheswaran, June Paik, Michael W Mahoney, Kurt Keutzer, and Amir Gholami. Squeezed attention: Accelerating long context length llm inference.arXiv preprint arXiv:2411.09688, 2024

    2024

  19. [27]

    Efficient long-decoding inference with reasoning-aware attention sparsity.arXiv preprint arXiv:2502.11147, 2025

    Junhao Hu, Wenrui Huang, Weidong Wang, Zhenwen Li, Tiancheng Hu, Zhixia Liu, Xusheng Chen, Tao Xie, and Yizhou Shan. Efficient long-decoding inference with reasoning-aware attention sparsity.arXiv preprint arXiv:2502.11147, 2025. 14

    2025 arXiv

  20. [28]

    Openai o1 system card.arXiv preprint arXiv:2412.16720, 2024

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai o1 system card.arXiv preprint arXiv:2412.16720, 2024

    2024 arXiv

  21. [29]

    Minference 1.0: Accelerating pre-filling for long-context llms via dynamic sparse attention.arXiv preprint arXiv:2407.02490, 2024

    Huiqiang Jiang, Yucheng Li, Chengruidong Zhang, Qianhui Wu, Xufang Luo, Surin Ahn, Zhenhua Han, Amir H Abdi, Dongsheng Li, Chin-Yew Lin, et al. Minference 1.0: Accelerating pre-filling for long-context llms via dynamic sparse attention.arXiv preprint arXiv:2407.02490, 2024

    2024 arXiv

  22. [30]

    Kullback-leibler divergence

    James M Joyce. Kullback-leibler divergence. InInternational encyclopedia of statistical science, pages 720–722. Springer, 2011

    2011

  23. [31]

    Transformers are rnns: Fast autoregressive transformers with linear attention

    Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, and François Fleuret. Transformers are rnns: Fast autoregressive transformers with linear attention. InInternational conference on machine learning, pages 5156–5165. PMLR, 2020

    2020

  24. [32]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large lan- guage model serving with pagedattention. InProceedings of the ACM SIGOPS 29th Symposium on Operating System...

    2023

  25. [33]

    Flexprefill: A context- aware sparse attention mechanism for efficient long-sequence inference.arXiv preprint arXiv:2502.20766, 2025

    Xunhao Lai, Jianqiao Lu, Yao Luo, Yiyuan Ma, and Xun Zhou. Flexprefill: A context- aware sparse attention mechanism for efficient long-sequence inference.arXiv preprint arXiv:2502.20766, 2025

    2025 arXiv

  26. [34]

    Fast inference from transformers via speculative decoding

    Yaniv Leviathan, Matan Kalman, and Yossi Matias. Fast inference from transformers via speculative decoding. InInternational Conference on Machine Learning, pages 19274–19286. PMLR, 2023

    2023

  27. [35]

    Minimax-01: Scaling foundation models with lightning attention.arXiv preprint arXiv:2501.08313, 2025

    Aonian Li, Bangwei Gong, Bo Yang, Boji Shan, Chang Liu, Cheng Zhu, Chunhao Zhang, Congchao Guo, Da Chen, Dong Li, et al. Minimax-01: Scaling foundation models with lightning attention.arXiv preprint arXiv:2501.08313, 2025

    2025 arXiv

  28. [36]

    Scbench: A kv cache-centric analysis of long-context methods.arXiv preprint arXiv:2412.10319, 2024

    Yucheng Li, Huiqiang Jiang, Qianhui Wu, Xufang Luo, Surin Ahn, Chengruidong Zhang, Amir H Abdi, Dongsheng Li, Jianfeng Gao, Yuqing Yang, et al. Scbench: A kv cache-centric analysis of long-context methods.arXiv preprint arXiv:2412.10319, 2024

    2024 arXiv

  29. [37]

    Snapkv: Llm knows what you are looking for before generation.Advances in Neural Information Processing Systems, 37:22947–22970, 2024

    Yuhong Li, Yingbing Huang, Bowen Yang, Bharat Venkitesh, Acyr Locatelli, Hanchen Ye, Tianle Cai, Patrick Lewis, and Deming Chen. Snapkv: Llm knows what you are looking for before generation.Advances in Neural Information Processing Systems, 37:22947–22970, 2024

    2024

  30. [38]

    Twilight: Adaptive attention sparsity with hierarchical top- p pruning.arXiv preprint arXiv:2502.02770, 2025

    Chaofan Lin, Jiaming Tang, Shuo Yang, Hanshuo Wang, Tian Tang, Boyu Tian, Ion Stoica, Song Han, and Mingyu Gao. Twilight: Adaptive attention sparsity with hierarchical top- p pruning.arXiv preprint arXiv:2502.02770, 2025

    2025

  31. [39]

    Adaptive computation pruning for the forgetting transformer.arXiv preprint arXiv:2504.06949, 2025

    Zhixuan Lin, Johan Obando-Ceron, Xu Owen He, and Aaron Courville. Adaptive computation pruning for the forgetting transformer.arXiv preprint arXiv:2504.06949, 2025

    2025 arXiv

  32. [40]

    Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model.arXiv preprint arXiv:2405.04434, 2024

    Aixin Liu, Bei Feng, Bin Wang, Bingxuan Wang, Bo Liu, Chenggang Zhao, Chengqi Dengr, Chong Ruan, Damai Dai, Daya Guo, et al. Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model.arXiv preprint arXiv:2405.04434, 2024

    2024 arXiv

  33. [41]

    Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024

    2024 arXiv

  34. [42]

    Retrievalattention: Accelerating long- context llm inference via vector retrieval.arXiv preprint arXiv:2409.10516, 2024

    Di Liu, Meng Chen, Baotong Lu, Huiqiang Jiang, Zhenhua Han, Qianxi Zhang, Qi Chen, Chengruidong Zhang, Bailu Ding, Kai Zhang, et al. Retrievalattention: Accelerating long- context llm inference via vector retrieval.arXiv preprint arXiv:2409.10516, 2024

    2024 arXiv

  35. [43]

    Quantization hurts reasoning? an empirical study on quantized reasoning models

    Ruikang Liu, Yuxuan Sun, Manyi Zhang, Haoli Bai, Xianzhi Yu, Tiezheng Yu, Chun Yuan, and Lu Hou. Quantization hurts reasoning? an empirical study on quantized reasoning models. arXiv preprint arXiv:2504.04823, 2025. 15

    2025 arXiv

  36. [44]

    Zichang Liu, Aditya Desai, Fangshuo Liao, Weitao Wang, Victor Xie, Zhaozhuo Xu, Anastasios Kyrillidis, and Anshumali Shrivastava. Scissorhands: Exploiting the persistence of impor- tance hypothesis for llm kv cache compression at test time.Advances in Neural Information Proces...

    2023

  37. [45]

    Moba: Mixture of block attention for long-context llms.arXiv preprint arXiv:2502.13189, 2025

    Enzhe Lu, Zhejun Jiang, Jingyuan Liu, Yulun Du, Tao Jiang, Chao Hong, Shaowei Liu, Weiran He, Enming Yuan, Yuzhi Wang, Zhiqi Huang, Huan Yuan, Suting Xu, Xinran Xu, Guokun Lai, Yanru Chen, Huabin Zheng, Junjie Yan, Jianlin Su, Yuxin Wu, Yutao Zhang, Zhilin Yang, Xinyu Zhou, Mi...

    2025 arXiv

  38. [47]

    Inference-time sparse attention with asymmetric indexing

    Pierre-Emmanuel Mazaré, Gergely Szilvasy, Maria Lomeli, Francisco Massa, Naila Murray, Hervé Jégou, and Matthijs Douze. Inference-time sparse attention with asymmetric indexing. arXiv preprint arXiv:2502.08246, 2025

    2025 arXiv

  39. [48]

    Aime problems and solutions

    Art of Problem Solving. Aime problems and solutions. https://artofproblemsolving. com/wiki/index.php/AIME_Problems_and_Solutions

  40. [49]

    Rwkv: Reinventing rnns for the transformer era.arXiv preprint arXiv:2305.13048, 2023

    Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Stella Biderman, Huanqi Cao, Xin Cheng, Michael Chung, Matteo Grella, et al. Rwkv: Reinventing rnns for the transformer era.arXiv preprint arXiv:2305.13048, 2023

    2023 arXiv

  41. [50]

    Gpqa: A graduate-level google-proof q&a benchmark

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. Gpqa: A graduate-level google-proof q&a benchmark. InFirst Conference on Language Modeling, 2024

    2024

  42. [51]

    Flashattention-3: Fast and accurate attention with asynchrony and low-precision.Advances in Neural Information Processing Systems, 37:68658–68685, 2024

    Jay Shah, Ganesh Bikshandi, Ying Zhang, Vijay Thakkar, Pradeep Ramani, and Tri Dao. Flashattention-3: Fast and accurate attention with asynchrony and low-precision.Advances in Neural Information Processing Systems, 37:68658–68685, 2024

    2024

  43. [52]

    Fast transformer decoding: One write-head is all you need.arXiv preprint arXiv:1911.02150, 2019

    Noam Shazeer. Fast transformer decoding: One write-head is all you need.arXiv preprint arXiv:1911.02150, 2019

    1911 arXiv

  44. [53]

    Roformer: Enhanced transformer with rotary position embedding.Neurocomputing, 568:127063, 2024

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding.Neurocomputing, 568:127063, 2024

    2024

  45. [54]

    Retentive network: A successor to transformer for large language models.arXiv preprint arXiv:2307.08621, 2023

    Yutao Sun, Li Dong, Shaohan Huang, Shuming Ma, Yuqing Xia, Jilong Xue, Jianyong Wang, and Furu Wei. Retentive network: A successor to transformer for large language models.arXiv preprint arXiv:2307.08621, 2023

    2023 arXiv

  46. [55]

    You only cache once: Decoder-decoder architectures for language models.Advances in Neural Information Processing Systems, 37:7339–7361, 2024

    Yutao Sun, Li Dong, Yi Zhu, Shaohan Huang, Wenhui Wang, Shuming Ma, Quanlu Zhang, Jianyong Wang, and Furu Wei. You only cache once: Decoder-decoder architectures for language models.Advances in Neural Information Processing Systems, 37:7339–7361, 2024

    2024

  47. [56]

    Rectified sparse attention.arXiv preprint arXiv:2506.04108, 2025

    Yutao Sun, Tianzhu Ye, Dong Li, Yuqing Xia, Jian Chen, Yizhao Gao, Shijie Cao, Jianyong Wang, and Furu Wei. Rectified sparse attention.arXiv preprint arXiv:2506.04108, 2025

    2025 arXiv

  48. [57]

    Quest: Query-aware sparsity for efficient long-context llm inference.arXiv preprint arXiv:2406.10774, 2024

    Jiaming Tang, Yilong Zhao, Kan Zhu, Guangxuan Xiao, Baris Kasikci, and Song Han. Quest: Query-aware sparsity for efficient long-context llm inference.arXiv preprint arXiv:2406.10774, 2024

    2024 arXiv

  49. [58]

    Minicpm4: Ultra-efficient llms on end devices

    MiniCPM Team. Minicpm4: Ultra-efficient llms on end devices. 2025

    2025

  50. [59]

    Triton: an intermediate language and compiler for tiled neural network computations

    Philippe Tillet, Hsiang-Tsung Kung, and David Cox. Triton: an intermediate language and compiler for tiled neural network computations. InProceedings of the 3rd ACM SIGPLAN International Workshop on Machine Learning and Programming Languages, pages 10–19, 2019

    2019

  51. [60]

    Attention is all you need.Advances in neural information processing systems, 30, 2017

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.Advances in neural information processing systems, 30, 2017. 16

    2017

  52. [61]

    Ladder: Enabling efficient low-precision deep learning computing through hardware-aware tensor transformation

    Lei Wang, Lingxiao Ma, Shijie Cao, Quanlu Zhang, Jilong Xue, Yining Shi, Ningxin Zheng, Ziming Miao, Fan Yang, Ting Cao, Yuqing Yang, and Mao Yang. Ladder: Enabling efficient low-precision deep learning computing through hardware-aware tensor transformation. In 18th USENIX Sym...

    2024

  53. [62]

    Infllm: Training-free long-context extrapolation for llms with an efficient context memory.arXiv preprint arXiv:2402.04617, 2024

    Chaojun Xiao, Pengle Zhang, Xu Han, Guangxuan Xiao, Yankai Lin, Zhengyan Zhang, Zhiyuan Liu, and Maosong Sun. Infllm: Training-free long-context extrapolation for llms with an efficient context memory.arXiv preprint arXiv:2402.04617, 2024

    2024 arXiv

  54. [63]

    Efficient streaming language models with attention sinks.arXiv preprint arXiv:2309.17453, 2023

    Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. Efficient streaming language models with attention sinks.arXiv preprint arXiv:2309.17453, 2023

    2023 arXiv

  55. [64]

    Duoattention: Efficient long-context llm inference with retrieval and streaming heads.arXiv preprint arXiv:2410.10819, 2024

    Guangxuan Xiao, Jiaming Tang, Jingwei Zuo, Junxian Guo, Shang Yang, Haotian Tang, Yao Fu, and Song Han. Duoattention: Efficient long-context llm inference with retrieval and streaming heads.arXiv preprint arXiv:2410.10819, 2024

    arXiv

  56. [65]

    Xattention: Block sparse attention with antidiagonal scoring.arXiv preprint arXiv:2503.16428, 2025

    Ruyi Xu, Guangxuan Xiao, Haofeng Huang, Junxian Guo, and Song Han. Xattention: Block sparse attention with antidiagonal scoring.arXiv preprint arXiv:2503.16428, 2025

    2025 arXiv

  57. [66]

    Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025

    2025 arXiv

  58. [67]

    Lserve: Efficient long-sequence llm serving with unified sparse attention.arXiv preprint arXiv:2502.14866, 2025

    Shang Yang, Junxian Guo, Haotian Tang, Qinghao Hu, Guangxuan Xiao, Jiaming Tang, Yujun Lin, Zhijian Liu, Yao Lu, and Song Han. Lserve: Efficient long-sequence llm serving with unified sparse attention.arXiv preprint arXiv:2502.14866, 2025

    2025 arXiv

  59. [68]

    Post-training sparse attention with double sparsity.arXiv preprint arXiv:2408.07092, 2024

    Shuo Yang, Ying Sheng, Joseph E Gonzalez, Ion Stoica, and Lianmin Zheng. Post-training sparse attention with double sparsity.arXiv preprint arXiv:2408.07092, 2024

    2024 arXiv

  60. [69]

    Gated linear attention transformers with hardware-efficient training.arXiv preprint arXiv:2312.06635, 2023

    Songlin Yang, Bailin Wang, Yikang Shen, Rameswar Panda, and Yoon Kim. Gated linear attention transformers with hardware-efficient training.arXiv preprint arXiv:2312.06635, 2023

    2023 arXiv

  61. [70]

    Parallelizing linear transformers with the delta rule over sequence length.arXiv preprint arXiv:2406.06484, 2024

    Songlin Yang, Bailin Wang, Yu Zhang, Yikang Shen, and Yoon Kim. Parallelizing linear transformers with the delta rule over sequence length.arXiv preprint arXiv:2406.06484, 2024

    2024 arXiv

  62. [71]

    Native sparse attention: Hardware-aligned and natively trainable sparse attention.arXiv preprint arXiv:2502.11089, 2025

    Jingyang Yuan, Huazuo Gao, Damai Dai, Junyu Luo, Liang Zhao, Zhengyan Zhang, Zhenda Xie, YX Wei, Lean Wang, Zhiping Xiao, et al. Native sparse attention: Hardware-aligned and natively trainable sparse attention.arXiv preprint arXiv:2502.11089, 2025

    2025 arXiv

  63. [72]

    Hardware-efficient attention for fast decoding.arXiv preprint arXiv:2505.21487, 2025

    Ted Zadouri, Hubert Strauss, and Tri Dao. Hardware-efficient attention for fast decoding.arXiv preprint arXiv:2505.21487, 2025

    2025 arXiv

  64. [73]

    Big bird: Transformers for longer sequences.Advances in neural information processing systems, 33: 17283–17297, 2020

    Manzil Zaheer, Guru Guruganesh, Kumar Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontanon, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, et al. Big bird: Transformers for longer sequences.Advances in neural information processing systems, 33: 17283–17297, 2020

    2020

  65. [74]

    In-context kv-cache eviction for llms via attention-gate.arXiv preprint arXiv:2410.12876, 2024

    Zihao Zeng, Bokai Lin, Tianqi Hou, Hao Zhang, and Zhijie Deng. In-context kv-cache eviction for llms via attention-gate.arXiv preprint arXiv:2410.12876, 2024

    2024 arXiv

  66. [75]

    Pqcache: Product quantization-based kvcache for long context llm inference

    Hailin Zhang, Xiaodong Ji, Yilin Chen, Fangcheng Fu, Xupeng Miao, Xiaonan Nie, Weipeng Chen, and Bin Cui. Pqcache: Product quantization-based kvcache for long context llm inference. arXiv preprint arXiv:2407.12820, 2024

    2024 arXiv

  67. [76]

    Spargeattn: Accurate sparse attention accelerating any model inference

    Jintao Zhang, Chendong Xiang, Haofeng Huang, Jia Wei, Haocheng Xi, Jun Zhu, and Jianfei Chen. Spargeattn: Accurate sparse attention accelerating any model inference. InInternational Conference on Machine Learning (ICML), 2025. 17

    2025

  68. [77]

    H2o: Heavy-hitter oracle for efficient generative inference of large language models

    Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Re, Clark Barrett, Zhangyang Wang, and Beidi Chen. H2o: Heavy-hitter oracle for efficient generative inference of large language models. In Thirty-seventh Conf...

    2023

  69. [78]

    Sglang: Efficient execution of structured language model programs.Advances in Neural Information Processing Systems, 37:62557–62583, 2024

    Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Livia Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E Gonzalez, et al. Sglang: Efficient execution of structured language model programs.Advances in Neural Information Processing Systems, 37:...

    2024

  70. [79]

    ROLLER: Fast and efficient tensor compilation for deep learning

    Hongyu Zhu, Ruofan Wu, Yijia Diao, Shanbin Ke, Haoyu Li, Chen Zhang, Jilong Xue, Lingxiao Ma, Yuqing Xia, Wei Cui, Fan Yang, Mao Yang, Lidong Zhou, Asaf Cidon, and Gennady Pekhimenko. ROLLER: Fast and efficient tensor compilation for deep learning. In16th USENIX Symposium on O...

    2022

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.