Pith. sign in

REVIEW 18 cited by

FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2502.20766 v1 pith:F7LOUTKT submitted 2025-02-28 cs.LG cs.CL

Xunhao Lai , Jianqiao Lu , Yao Luo , Yiyuan Ma , Xun Zhou This is my paper · ORCID
classification cs.LGcs.CL
keywords attentionsparsepatternsinferenceflexprefilllong-sequenceadaptivelychallenges
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Large language models (LLMs) encounter computational challenges during long-sequence inference, especially in the attention pre-filling phase, where the complexity grows quadratically with the prompt length. Previous efforts to mitigate these challenges have relied on fixed sparse attention patterns or identifying sparse attention patterns based on limited cases. However, these methods lacked the flexibility to efficiently adapt to varying input demands. In this paper, we introduce FlexPrefill, a Flexible sparse Pre-filling mechanism that dynamically adjusts sparse attention patterns and computational budget in real-time to meet the specific requirements of each input and attention head. The flexibility of our method is demonstrated through two key innovations: 1) Query-Aware Sparse Pattern Determination: By measuring Jensen-Shannon divergence, this component adaptively switches between query-specific diverse attention patterns and predefined attention patterns. 2) Cumulative-Attention Based Index Selection: This component dynamically selects query-key indexes to be computed based on different attention patterns, ensuring the sum of attention scores meets a predefined threshold. FlexPrefill adaptively optimizes the sparse pattern and sparse ratio of each attention head based on the prompt, enhancing efficiency in long-sequence inference tasks. Experimental results show significant improvements in both speed and accuracy over prior methods, providing a more flexible and efficient solution for LLM inference.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 18 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Text Template Tokens Are Implicit Semantic Registers in Diffusion Transformers

    cs.CV 2026-07 conditional novelty 7.0 of 10

    Chat-template tokens in LLM-conditioned DiTs act as implicit semantic registers: they absorb object identity from image latents and maintain it, while direct prompt-reading heads are causally inert.

  2. Evidence-Driven Dynamic Visual Selector for Efficient Long Video Understanding

    cs.CV 2026-08 conditional novelty 6.0 of 10

    EviSelect uses the target multimodal model's internal attention as a prior to dynamically select frames, sampling rates, and resolutions, achieving about 50% token reduction and a 3.9x speedup with better benchmark accuracy.

  3. CoSA: Accelerating Long-Context Inference via Proxy-Kernel Co-Designed Sparse Attention

    cs.CL 2026-07 conditional novelty 6.0 of 10

    CoSA couples an order-aware sparse-mask proxy with a skip-capable attention kernel, achieving higher long-context accuracy at lower compute budgets than existing sparse-attention baselines.

  4. PIVOT: Efficient Query-Group Indexing for Token-Level Sparse Attention

    cs.CL 2026-07 conditional novelty 6.0 of 10

    Sharing one proxy full-prefix scan across nearby query groups matches dense DSA indexer accuracy while accelerating indexing up to 4× and end-to-end latency up to 1.6×.

  5. SAF3R: Dynamic Sparse Attention for Feed-Forward 3D Reconstruction Transformers

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Training-free head-wise dynamic sparse attention preserves F3R pose and reconstruction quality while delivering multi-fold end-to-end speedups on long image sequences.

  6. Dropping the Anchor: Statistical Context Summarization for Distributed Systems via Pulsar Attention

    cs.CL 2026-05 conditional novelty 6.0 of 10

    IDF-selected chunk summaries replace Star Attention's static anchor, cutting Phase 1 FLOPs up to 3.3× and reportedly beating dense attention on RULER at 32K–128K.

  7. RAT+: Train Dense, Infer Sparse -- Recurrence Augmented Attention for Dilated Inference

    cs.LG 2026-02 unverdicted novelty 6.0 of 10

    RAT+ pretrains a single dense recurrent-augmented attention model that supports flexible dilated sparse inference after short adaptation, matching dense accuracy at moderate dilation and losing only 1-3 points at high...

  8. Prism: Spectral-Aware Block-Sparse Attention

    cs.CL 2026-02 conditional novelty 6.0 of 10

    Mean-pooling under RoPE attenuates high-frequency position signals (a spectral blind spot), and Prism's dual-band RMS-calibrated block selection recovers enough signal to match full attention at up to 5.1× prefill speedup.

  9. ParisKV: Fast and Drift-Robust KV-Cache Retrieval for Long-Context LLMs

    cs.LG 2026-02 conditional novelty 6.0 of 10

    ParisKV retrieves top-K KV pairs on GPU using analytic sign-pattern centroids and calibrated 4-bit reranking, matching full attention accuracy in most settings while cutting million-token decode latency up to 44x.

  10. SPADE: An Input-Adaptive Sparse Attention Engine for Fast Video Diffusion Models Inference

    cs.CV 2026-08 conditional novelty 5.0 of 10

    SPADE combines static, semi-static, and dynamic block-sparse attention with a cheap SICS-based blocking heuristic to speed up video diffusion inference by up to 1.80x end-to-end.

  11. Stem: Rethinking Causal Information Flow in Sparse Attention

    cs.LG 2026-03 conditional novelty 5.0 of 10

    Stem prunes causal attention with a position-decaying budget and a value-magnitude-aware score, matching dense accuracy on LongBench/RULER at a 25-31% compute budget.

  12. UltraImageGen: Efficient Ultra-High-Resolution Image Generation with Hierarchical Local Attention

    cs.CV 2025-10 conditional novelty 5.0 of 10

    A pretrained FLUX diffusion model is adapted with local-window attention plus low-resolution global guidance, allowing 4K text-to-image generation from 1K-only training data at about 2x lower cost.

  13. DeltaLLM: A Training-Free Framework Exploiting Temporal Sparsity for Efficient Edge LLM Inference

    cs.AI 2025-07 conditional novelty 5.0 of 10

    DeltaLLM exploits temporal sparsity in key vectors (delta encoding) with a hybrid local-window attention to cut attention computation by up to 60% on 1-2B parameter LLMs with minimal accuracy loss.

  14. Sparse Fine-Tuning of Transformers for Generative Tasks

    cs.CV 2025-07 conditional novelty 5.0 of 10

    A frozen transformer is fine-tuned by adding a sparse dictionary of feature atoms to each layer's output, enabling atom-level control that improves image editing and concept customization.

  15. Lag-Relative Sparse Attention In Long Context Training

    cs.CL 2025-06 conditional novelty 5.0 of 10

    Training a language model with LagKV's static sparse attention mask improves its robustness to KV-cache compression at 2x and 4x rates, while claims of overall average gains are not consistently supported.

  16. Sparse-vDiT: Unleashing the Power of Sparse Attention to Accelerate Video Diffusion Transformers

    cs.CV 2025-06 conditional novelty 5.0 of 10

    Sparse-vDiT replaces dense attention with fixed per-head sparse patterns chosen offline, achieving 1.58-1.85x end-to-end speedups on CogVideoX1.5, HunyuanVideo, and Wan2.1 with minimal quality loss.

  17. AnchorAttention: Difference-Aware Sparse Attention with Stripe Granularity

    cs.LG 2025-05 conditional novelty 5.0 of 10

    AnchorAttention uses the maximum attention score from initial and local tokens as an anchor to threshold-select important key-value positions at stripe granularity, achieving faster prefill with comparable accuracy.

  18. SeerAttention-R: Sparse Attention Adaptation for Long Reasoning

    cs.LG 2025-06 conditional novelty 4.0 of 10

    A learned gate selects the important KV blocks during long decoding, preserving math reasoning accuracy while skipping up to 90% of attention work.

Pith tools