Pith. sign in

REVIEW 3 major objections 5 minor 31 references

A new speculative decoding runtime verifies multiple draft tokens in one pass for linear-attention models and achieves up to 1.70x faster generation on a 1.3B Gated DeltaNet target.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 20:15 UTC pith:7ERME3HB

load-bearing objection The core result is plausible and the engineering is real, but the paper's target-model description is internally contradictory—'single GDN layer' vs. the full GDN-1.3B checkpoint—and that has to be fixed before the headline speedup can be trusted. the 3 major comments →

arxiv 2607.16673 v1 pith:7ERME3HB submitted 2026-07-18 cs.CL

SpecLA: Efficient Speculative Decoding for Linear-Attention Models

Zhibin Wang , Xuying Han , Zhaohua Yang , Fuliang Liu , Xue Li , Rong Gu , Sheng Zhong , Chen Tian This is my paper
classification cs.CL
keywords speculative decodinglinear attentionrecurrent stateGated DeltaNettree verificationstate commitmentdraft pruningdecoding acceleration
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Linear-attention models avoid a growing KV cache by folding the prefix into a dense recurrent state, but decoding still reads, updates, and writes that state one token at a time. The paper argues that speculative decoding can be made to work for these stateful targets, despite the failure of naive adaptations: verification kernels can keep recurrent state tiles on-chip for chains, use a tree-masked factorized form for branches, and commit accepted states from compact factors rather than snapshots or replays. Supported by a runtime implementation on an H100 with a 1.3B Gated DeltaNet target, the paper reports up to 1.70x end-to-end speedup over autoregressive decoding. The broader point is that the abstraction separating draft, verify, and accept should treat the recurrent state as a first-class scheduling object, not as a token-indexed cache.

Core claim

At the center of the paper is a claim about verification. For a Gated DeltaNet target, the draft-tree topology can be inserted directly into the Delta-rule factorization: build Atree = Mtree ⊙ (Kb Kaᵀ), solve Ttree = solve_tril(I − Atree), and compute OGDN = bQ S0ᵀ − Attn(W S0ᵀ) + Attn U. This produces logits for every candidate node from the same committed state S0 while letting updates propagate only along ancestor paths, so tree-shaped drafts are verified without root-to-leaf replay or full-state snapshots. Acceptance is then handled by buffering the small per-token factors produced during verification and applying only the accepted path inside the next verification kernel. The paper vali

What carries the argument

The load-bearing object is the tree-masked factorized verifier, a factorized form of the GDN Delta-rule recurrence with the draft topology as a mask: Atree = Mtree ⊙ (Kb Kaᵀ), Ttree = solve_tril(I − Atree), OGDN = bQ S0ᵀ − Attn(W S0ᵀ) + Attn U. It carries the argument by turning tree verification into a parallel, mask-controlled computation over candidate nodes, avoiding both decode-kernel replay and prefill-kernel reuse. Its supporting mechanisms are V-dimension tiling for the serial chain path and a chain-decomposed hybrid schedule that runs serial kernels inside dependency-respecting chains and parallelizes across ready chains.

Load-bearing premise

The tree-masked factorized GDN verifier must exactly reproduce the GDN Delta-rule recurrence along ancestor-only paths; the paper checks this empirically against a sequential reference but supplies no formal proof or error guarantee.

What would settle it

Construct a draft tree in which a node has two ancestors whose Delta-rule updates interact, run the tree-masked factorized verifier and a sequential token-by-token GDN recurrence in high precision, and compare the candidate logits and committed state; any divergence beyond floating-point rounding at the first branching node would show the factorization is not target-distribution preserving.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Verified multi-token batches amortize the dense recurrent-state read/write cost, so memory-bound linear-attention decode no longer pays the state round trip per generated token.
  • Tree-shaped drafts can be accepted for stateful targets without snapshotting states or replaying shared prefixes; branch parallelism is exposed while sibling leakage is masked out.
  • Accepted recurrent states can be reconstructed from compact verification factors rather than token replay, cutting accepted-state recovery latency by 2.74–4.28x in the reported microbenchmarks.
  • The paper's controlled acceptance study implies that draft length should be tuned to the drafter's accuracy: on the measured 1.3B pair, acceptance probability below about 0.7 makes speculation unprofitable at all tested lengths.
  • On H100 with a public GDN-1.3B target, end-to-end speedup over autoregressive decoding reaches 1.70x on GSM8K, 1.42x on the mixed suite, and 1.06x on HumanEval.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the factor-buffering design is layer-agnostic in the sense the paper itself notes (GLA, DeltaNet, and Mamba variants expose different factor records), so the same commit-and-verify runtime could likely be ported to other stateful architectures with per-layer factor definitions.
  • Beyond the paper, the tree-masked factorization suggests a testable extension: if the identity is given a formal error bound, the same kernel pattern could serve as a general verification primitive for any delta-rule linear attention, not only GDN.
  • Beyond the paper, the sharp drop on HumanEval suggests the bottleneck there is draft quality rather than kernel speed; an adaptive drafter that switches strategy for code-like prompts is a concrete next experiment implied by the paper's data.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. SpecLA is a speculative decoding runtime for stateful linear-attention (Gated DeltaNet, GDN) targets. It replaces KV-cache-oriented speculation with three mechanisms: (i) topology-aware verification kernels (state-resident serial verification, tree-masked factorized parallel verification using a masked Delta-rule UT factorization, and a chain-decomposed hybrid kernel); (ii) accepted-state recovery via buffered per-token factors with a delayed state update fused into the next verification pass; and (iii) confidence-guided tree pruning plus a target-aligned EAGLE-style drafter. On an NVIDIA H100 the paper reports end-to-end speedups of 1.42x, 1.70x, and 1.06x over autoregressive decoding on a mixed suite, GSM8K, and HumanEval with a public GDN-1.3B target, together with kernel microbenchmarks and a controlled acceptance study.

Significance. If the headline result holds, this is a useful contribution to inference serving for linear-attention models. The paper addresses a real gap: existing speculative decoding targets KV-cache Transformers, while dense recurrent state changes both verification and rollback. The factor-buffer commit and delayed update are elegant and avoid full-state snapshots and token replay. The tree-masked factorization is an interesting adaptation of the DeltaNet UT transform. The paper should be credited for checking exact output hashes against autoregressive decoding on all reported routes, for validating custom kernels against sequential references, and for explicitly labeling the 9B-scale projection as not measured. The main deficiencies are an internal ambiguity about what was actually timed in the end-to-end experiments and the absence of released code, which makes this ambiguity difficult to resolve from the text.

major comments (3)
  1. [Section 7 / Section 8.1] The implementation and evaluation sections contain incompatible statements about the target. Section 7 (and the repeated paragraph opening Section 8) states: "The target is a single GDN layer configured to match GDN-1.3B: h_v=8, d_k=d_v=256..." Section 8.1 then states: "Our end-to-end experiments use a public GDN-1.3B checkpoint [13] as the target model." A 1.3B checkpoint is a full multi-layer network, not a single GDN layer. If the timed route is the single layer, the abstract's "up to 1.70x end-to-end speedup" is unsupported, because the speedup would exclude embedding, MLP, norm, LM-head, and other layers' state traffic; per-layer state movement is only a fraction of autoregressive decode cost. If the end-to-end experiments did use the full checkpoint, Section 7's wording is misleading and the microbenchmarks must be clearly separated from the full-model setup. Please state exactly w
  2. [Section 4.2] The tree-masked factorized GDN verifier is stated as Atree = Mtree ⊙ (K_b K_a^T), Ttree = solve_tril(I − Atree), and OGDN = bQ S0^T − Attn(W S0^T) + AttnU, with no proof that this exactly reproduces the recurrent GDN update along ancestor-only paths. The empirical validation against a sequential reference and the output-hash checks in Section 8.1 are reassuring, but the central claim of exact target-side verification rests on this factorization. Please provide a derivation (or a precise reference to one) and state the floating-point/error behavior. If the factorization is only approximate, the correctness-preserving claim should be weakened accordingly.
  3. [Section 8.2 / Section 8.3] The end-to-end section reports speedup ratios only; there are no absolute latencies or a breakdown of where wall-clock time goes. Combined with the target ambiguity above, the reader cannot determine whether the reported 1.70x comes from the full GDN-1.3B checkpoint or from a single-layer proxy. The tree-verification microbenchmarks in Figure 10 and Table 1 use the single-layer "target shape" and are valuable as kernel studies, but they should not be conflated with the end-to-end result. Please report full-model timings for the Autoregressive, FLA-SD, Chain, and SpecLA routes, including drafter and commit overheads.
minor comments (5)
  1. [Section 7 / Section 8] The implementation paragraph is duplicated verbatim at the start of Section 8; remove the duplicate.
  2. [Section 6.2] The target-aligned drafter is described as trained on recurrent target features, but no training set, size, hyperparameters, or measured training cost is given. A short appendix or paragraph would make the end-to-end timing reproducible.
  3. [Section 8.1] No code or artifact link is provided. For a systems paper with custom Triton kernels, code release would substantially help verify the kernel-level claims.
  4. [Section 6.1] The tree margin τ_tree, top-k, and draft budget are introduced without sensitivity analysis. Since pruning changes the candidate set, report how end-to-end speedup varies with τ_tree.
  5. [Section 8.5] The 9B-scale projection is correctly labeled as not measured; please retain this caveat if the result is referenced in future versions or in a conclusion summary.

Circularity Check

0 steps flagged

No significant circularity; the claimed speedups are measured end-to-end and the verifier factorization follows published GDN math, not fitted inputs or self-citations.

full rationale

The paper's central results are empirical: end-to-end speedups are measured latencies on an H100, kernel overheads are benchmarked, and output hashes are checked against autoregressive decoding. The tree-masked GDN verifier (Section 4.2) is implemented by inserting the tree mask into the known GDN UT transform from [24,26]; it is validated against a sequential GDN reference, so it is not an input fitted to the speedup. State recovery via factor buffering follows from the algebraic form of the linear-attention update (Equations 1–3) and is again benchmarked, not asserted as a prediction. Draft pruning changes only the candidate set while acceptance remains target-side, so it cannot force the acceptance statistics. The 9B-scale projection in Section 8.5 is explicitly labeled as a projection, not a measured result, and therefore is not a fitted parameter passed off as a prediction. There are no load-bearing self-citations: prior work is cited for well-known speculative decoding, GDN kernels, and model cards, and none of the cited results are used to justify the paper's measured outcomes. The apparent inconsistency between Section 7's 'single GDN layer configured to match GDN-1.3B' and Section 8.1's 'public GDN-1.3B checkpoint' is a real measurement-support concern, but it is a correctness/experimental-design issue, not a circular derivation: the speedup numbers are reported measurements rather than consequences of the paper's own definitions or fitted inputs. No step in the derivation chain reduces by construction to its own inputs.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

The central result is an empirical systems claim. The runtime introduces no new physical or mathematical entities; it depends on the GDN recurrence, the exactness of the factorized verification kernels, and standard speculative-sampling assumptions. The listed configuration choices are hand-selected hyperparameters rather than quantities fitted to the reported speedups.

free parameters (3)
  • Tree margin τ_tree
    Pruning threshold in Section 6.1; its value is not reported, but the tree route's submitted topology and hence end-to-end speedup depend on it.
  • Draft budget (max nodes / top-k) = 16 nodes, k=4
    Selected for the SpecLA end-to-end route in Section 8.1; no sensitivity analysis is provided.
  • 9B-scale projection calibration
    Section 8.5 replaces autoregressive target latency with a Qwen3.5-9B proxy while retaining GDN-1.3B speculative-side overhead; explicitly not a measured 9B result.
axioms (4)
  • domain assumption GDN state updates follow the Delta-rule recurrence and can be reconstructed from compact factors (k, v, alpha, beta).
    Invoked in Section 4.2 (tree-masked UT factorization) and Section 5 (factor buffering, Eq. 3), relying on prior GDN/DeltaNet formulations [24,26].
  • domain assumption Layer-major verification (all draft tokens through layer ℓ before layer ℓ+1) is equivalent to token-major autoregressive decode.
    Core of Section 4.1 and Figure 3; no formal proof is given, only empirical kernel validation against a sequential GDN reference.
  • standard math Speculative sampling with posterior acceptance preserves the target distribution when verification is exact.
    Assumed from [4,9] and used throughout; not re-derived in this paper.
  • domain assumption The tree mask Mtree correctly encodes ancestor-only state flow for GDN candidates.
    Used in Section 4.2 to construct Atree = Mtree ⊙ (Kb Kaᵀ); no proof that the Delta-rule correction is exactly captured by this mask.

pith-pipeline@v1.3.0-alltime-deepseek · 17847 in / 12283 out tokens · 120217 ms · 2026-08-01T20:15:29.931426+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of SpecLA: Efficient Speculative Decoding for Linear-Attention Models." pith.science (2026). https://pith.science/paper/7ERME3HB

@misc{pith2026260716673,
  author       = {Pith},
  title        = {Pith review of: SpecLA: Efficient Speculative Decoding for Linear-Attention Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7ERME3HB}},
  note         = {Machine review of arXiv:2607.16673}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Linear-attention models replace the growing KV cache with recurrent states, but autoregressive decoding still reads, updates, and writes these states one token at a time. Speculative decoding can reduce this cost by verifying several draft tokens in one target pass, yet existing speculative systems are designed for Transformer KV caches. For stateful linear-attention targets, verification must follow recurrent dependencies across chains and branches, acceptance must update only the accepted state trajectory, and the drafter must avoid submitting candidates that waste stateful verification work. This paper presents SpecLA, a speculative decoding runtime for stateful linear-attention models. SpecLA verifies chains and trees with topology-aware kernels, stores compact factors produced during verification to recover accepted states, and uses confidence pruning plus a target-aligned EAGLE-style drafter to feed useful candidates to the verifier. On an NVIDIA H100 with a public GDN-1.3B target, SpecLA achieves up to 1.70x end-to-end speedup over autoregressive decoding.

Figures

Figures reproduced from arXiv: 2607.16673 by Chen Tian, Fuliang Liu, Rong Gu, Sheng Zhong, Xue Li, Xuying Han, Zhaohua Yang, Zhibin Wang.

Figure 1
Figure 1. Figure 1: Suffix truncation in Transformer-oriented spec￾ulative decoding. Accepted draft tokens remain in the KV cache, while rejected tokens are discarded as a cache suffix. used by the acceptance rule. This amortizes one expensive target invocation over multiple possible output tokens: par￾allel scoring of a short continuation can be comparable to a single target decode call, while potentially confirming mul￾tipl… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of SpecLA. submitted candidates are likely to be useful. Existing draft heads and tree-construction policies are largely designed around Transformer hidden states and KV-cache verifica￾tion [3, 10, 11]; their scores and features may not match a linear-attention target’s recurrent execution path. Draft construction also has a different cost profile: low-confidence long paths or broad branches waste… view at source ↗
Figure 3
Figure 3. Figure 3: Execution order in autoregressive decoding and speculative verification. the same layer execute consecutively. This schedule removes the layer gap between neighboring token updates, provides an opportunity to keep the recurrent state resident on SRAM across candidate tokens, and therefore avoids the round-trip cost of writing the state back to HBM after each token. However, the state size is still too larg… view at source ↗
Figure 6
Figure 6. Figure 6: shows the scheduling unit used by this design. The submitted topology in [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: State management in SpecLA. The left side compares full-state buffering, factor buffering, and token replay at the draft-verification boundary. The right side shows how delayed state update applies the accepted-factor buffer inside the next verification kernel, reusing the same state tiles loaded from HBM and avoiding a standalone recurrent-state round trip. snapshotting 𝐿 candidates requires 𝐿 times the s… view at source ↗
Figure 8
Figure 8. Figure 8: Confidence-guided tree pruning. Nodes within 𝜏tree of 𝑞 ★ are kept; monotonicity of 𝑞 makes the window ancestor-closed, and budget 𝑇 then truncates the selection (e.g., cutting 𝐵1). submitted tree size therefore measures the verification work that survives pruning, while accepted length measures what the target actually commits. 6.2 Target-Aligned Drafting Draft pruning reduces wasted verification only if … view at source ↗
Figure 9
Figure 9. Figure 9: End-to-end performance [PITH_FULL_IMAGE:figures/full_fig_p011_9.png] view at source ↗
Figure 12
Figure 12. Figure 12: Controlled acceptance regimes [PITH_FULL_IMAGE:figures/full_fig_p012_12.png] view at source ↗
Figure 11
Figure 11. Figure 11: State-management latency [PITH_FULL_IMAGE:figures/full_fig_p012_11.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

31 extracted references · 7 linked inside Pith

  1. [1]

    Zachary Ankner, Rishab Parthasarathy, Aniruddha Nrusimha, Christo- pher Rinard, Jonathan Ragan-Kelley, and William Brandon. 2024. Hy- dra: Sequentially-Dependent Draft Heads for Medusa Decoding. arXiv preprint arXiv:2402.05109. doi:10.48550/arXiv.2402.05109

    Pith/arXiv arXiv doi:10.48550/arxiv.2402.05109 2024

  2. [2]

    Maximilian Beck, Korbinian Pöppel, Phillip Lippe, and Sepp Hochreiter

  3. [3]

    Lee, Deming Chen, and Tri Dao

    Tianle Cai, Yuhong Li, Zhengyang Geng, Hongwu Peng, Jason D. Lee, Deming Chen, and Tri Dao. 2024. Medusa: Simple LLM Inference Ac- celeration Framework with Multiple Decoding Heads. InProceedings of the 41st International Conference on Machine Learning (Proceed- ings of Machine Learning Research, Vol. 235). PMLR, Vienna, Austria, 5209–5235

    2024

  4. [4]

    Charlie Chen, Sebastian Borgeaud, Geoffrey Irving, Jean-Baptiste Lespiau, Laurent Sifre, and John Jumper. 2023. Accelerating Large Language Model Decoding with Speculative Sampling. arXiv preprint arXiv:2302.01318. doi:10.48550/arXiv.2302.01318

    Pith/arXiv arXiv doi:10.48550/arxiv.2302.01318 2023

  5. [5]

    Tri Dao and Albert Gu. 2024. Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Du- ality. InProceedings of the 41st International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 235). PMLR, Vienna, Austria, 7805–7845

    2024

  6. [6]

    Albert Gu and Tri Dao. 2024. Mamba: Linear-Time Sequence Modeling with Selective State Spaces. arXiv preprint arXiv:2312.00752. doi:10. 48550/arXiv.2312.00752

    Pith/arXiv arXiv doi:10.48550/arxiv.2312.00752 2024

  7. [7]

    Prasanna

    Neelesh Gupta, Peter Wang, Rajgopal Kannan, and Viktor K. Prasanna

  8. [8]

    Gonzalez, Hao Zhang, and Ion Sto- ica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Sto- ica. 2023. Efficient Memory Management for Large Language Model Serving with PagedAttention. InProceedings of the 29th ACM Sym- posium on Operating Systems Principles. Association for Computing Machinery, Koblenz, Germany, 611–626. d...

    arXiv 2023

  9. [9]

    Yaniv Leviathan, Matan Kalman, and Yossi Matias. 2023. Fast Inference from Transformers via Speculative Decoding. InProceedings of the 40th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 202). PMLR, Honolulu, HI, USA, 19274–19286

    2023

  10. [10]

    Yuhui Li, Fangyun Wei, Chao Zhang, and Hongyang Zhang. 2024. EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees. arXiv preprint arXiv:2406.16858. doi:10.48550/arXiv.2406.16858

    Pith/arXiv arXiv doi:10.48550/arxiv.2406.16858 2024

  11. [11]

    Yuhui Li, Fangyun Wei, Chao Zhang, and Hongyang Zhang. 2024. EA- GLE: Speculative Sampling Requires Rethinking Feature Uncertainty. InProceedings of the 41st International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 235). PMLR, Vienna, Austria, 28935–28948

    2024

  12. [12]

    Yuhui Li, Fangyun Wei, Chao Zhang, and Hongyang Zhang. 2025. EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test. arXiv preprint arXiv:2503.01840. doi:10.48550/ arXiv.2503.01840

    Pith/arXiv arXiv doi:10.48550/arxiv.2503.01840 2025

  13. [13]

    m-a-p. 2026. 1.3B-100B-GatedDeltaNet-pure Model Card. Hug- ging Face model card.https://huggingface.co/m-a-p/1.3B-100B- GatedDeltaNet-pure

    2026

  14. [14]

    Xupeng Miao, Gabriele Oliaro, Zhihao Zhang, Xinhao Cheng, Zeyu Wang, Zhengxin Zhang, Rae Ying Yee Wong, Alan Zhu, Lijie Yang, Xiaoxiang Shi, Chunan Shi, Zhuoming Chen, Daiyaan Arfeen, Reyna Abhyankar, and Zhihao Jia. 2024. SpecInfer: Accelerating Large Lan- guage Model Serving with Tree-based Speculative Inference and Veri- fication. InProceedings of the ...

    arXiv 2024

  15. [15]

    NVIDIA Research. 2026. GatedDeltaNet: Official PyTorch Imple- mentation of Gated Delta Networks. GitHub repository.https: //github.com/NVlabs/GatedDeltaNet

    2026

  16. [16]

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chil- amkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala

  17. [17]

    Zhen Qin, Weigao Sun, Dong Li, Xuyang Shen, Weixuan Sun, and Yiran Zhong. 2024. Various Lengths, Constant Speed: Efficient Lan- guage Modeling with Lightning Attention. InProceedings of the 41st International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 235). PMLR, Vienna, Austria, 41517–41535

    2024

  18. [18]

    Qwen Team. 2025. Qwen3-Next-80B-A3B-Instruct Model Card. Hug- ging Face model card.https://huggingface.co/Qwen/Qwen3-Next- 80B-A3B-Instruct

    2025

  19. [19]

    Qwen Team. 2025. Qwen3 Technical Report. arXiv preprint arXiv:2505.09388. doi:10.48550/arXiv.2505.09388

    Pith/arXiv arXiv doi:10.48550/arxiv.2505.09388 2025

  20. [20]

    Philippe Tillet, H. T. Kung, and David Cox. 2019. Triton: An Intermedi- ate Language and Compiler for Tiled Neural Network Computations. InProceedings of the 3rd ACM SIGPLAN International Workshop on Ma- chine Learning and Programming Languages. Association for Comput- ing Machinery, Phoenix, AZ, USA, 10–19. doi:10.1145/3315508.3329973

    arXiv 2019

  21. [21]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention Is All You Need. InAdvances in Neural Information Processing Systems, Vol. 30. Curran Associates, Inc., Long Beach, CA, USA, 5998– 6008

    2017

  22. [22]

    Yangchao Wu, Yonatan Dukler, Matthew Trager, Alessandro Achille, Wei Xia, and Stefano Soatto. 2024. Snakes and Ladders: Accelerating SSM Inference with Speculative Decoding. InProceedings of the 4th NeurIPS Efficient Natural Language and Speech Processing Workshop, Vol. 262. PMLR, Vancouver, BC, Canada, 292–304

    2024

  23. [23]

    Yangchao Wu, Zongyue Qin, Alex Wong, and Stefano Soatto. 2025. STree: Speculative Tree Decoding for Hybrid State-Space Models. arXiv preprint arXiv:2505.14969. doi:10.48550/arXiv.2505.14969 13 Wang et al

    doi:10.48550/arxiv.2505.14969 2025

  24. [24]

    Songlin Yang, Jan Kautz, and Ali Hatamizadeh. 2025. Gated Delta Networks: Improving Mamba2 with Delta Rule. arXiv preprint arXiv:2412.06464. doi:10.48550/arXiv.2412.06464ICLR 2025 camera- ready version

    Pith/arXiv arXiv doi:10.48550/arxiv.2412.06464 2025

  25. [25]

    Songlin Yang, Bailin Wang, Yikang Shen, Rameswar Panda, and Yoon Kim. 2024. Gated Linear Attention Transformers with Hardware- Efficient Training. InProceedings of the 41st International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 235). PMLR, Vienna, Austria, 56501–56523

    2024

  26. [26]

    Songlin Yang, Bailin Wang, Yu Zhang, Yikang Shen, and Yoon Kim

  27. [27]

    Songlin Yang and Yu Zhang. 2024. FLA: A Triton-Based Library for Hardware-Efficient Implementations of Linear Attention Mech- anism. GitHub repository.https://github.com/sustcsonglin/flash- linear-attention 14

    2024

  28. [2019]

    InAdvances in Neural Information Processing Systems, Vol

    PyTorch: An Imperative Style, High-Performance Deep Learning Library. InAdvances in Neural Information Processing Systems, Vol. 32. Curran Associates, Inc., Vancouver, BC, Canada, 8024–8035

  29. [2024]

    InAdvances in Neural Information Processing Sys- tems, Vol

    Parallelizing Linear Transformers with the Delta Rule over Sequence Length. InAdvances in Neural Information Processing Sys- tems, Vol. 37. Curran Associates, Inc., Vancouver, BC, Canada, 137230– 137264

  30. [2025]

    arXiv preprint arXiv:2503.14376

    Tiled Flash Linear Attention: More Efficient Linear RNN and xLSTM Kernels. arXiv preprint arXiv:2503.14376. doi:10.48550/arXiv. 2503.14376Accepted at NeurIPS 2025

    doi:10.48550/arxiv.2503.14376 2025

  31. [2026]

    arXiv preprint arXiv:2603.05931

    A Persistent-State Dataflow Accelerator for Memory-Bound Linear Attention Decode on FPGA. arXiv preprint arXiv:2603.05931. doi:10.48550/arXiv.2603.05931

    doi:10.48550/arxiv.2603.05931