Pith. sign in

REVIEW 2 major objections 6 minor 4 cited by

A semi-autoregressive draft model plus load-aware verification length scheduling cuts LLM serving latency without quality loss and shifts the live throughput-interactivity frontier.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 08:04 UTC pith:5YIZGR46

load-bearing objection Solid systems paper: semi-AR drafting plus load-aware verification that actually moves the production Pareto on DeepSeek-V4, with clean offline ablations and open artifacts. the 2 major comments →

arxiv 2607.05147 v1 pith:5YIZGR46 submitted 2026-07-06 cs.AI cs.CL

DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation

Xin Cheng , Xingkai Yu , Chenze Shao , Jiashi Li , Yunfan Xiong , Yi Qian , Jiaqi Zhu , Shirong Ma
show 25 more authors
Xiaokang Zhang Jiasheng Ye Qinyu Chen Chengqi Deng Jiping Yu Damai Dai Zhengyan Zhang Yixuan Wei Yixuan Tan Wenkai Yang Runxin Xu Yu Wu Zhean Xu Xuanyu Wang Muyang Chen Rui Tian Xiao Bi Zhewen Hao Shaoyuan Chen Huanqi Cao Wentao Zhang Anyi Xu Huishuai Zhang Dongyan Zhao Wenfeng Liang
This is my paper
classification cs.AI cs.CL
keywords speculative decodingsemi-autoregressive generationparallel drafterconfidence schedulingLLM servingacceptance decayhardware-aware verification
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Speculative decoding speeds up large language models by drafting candidate tokens cheaply and verifying them in one target-model pass, but pure parallel drafters lose accuracy deeper in a block and fixed-length verification wastes capacity under load. DSpark keeps a heavy parallel backbone for speed, adds only a light sequential head so later draft tokens depend on earlier ones, and uses a calibrated confidence head to estimate how far each draft prefix will survive. A hardware-aware scheduler then chooses, for every request, how many of those tokens to verify so that expected accepted tokens times measured engine steps-per-second is maximized. Offline, the design raises accepted length over both autoregressive and parallel baselines across math, code, and chat. In live production traffic it raises per-user generation speed by tens of percent at the same system throughput and keeps useful capacity under strict speed SLAs that collapse the prior baseline.

Core claim

Coupling a parallel draft backbone with a lightweight sequential head restores intra-block token dependence enough to curb suffix decay, while confidence-scheduled verification that maximizes expected throughput given a profiled engine capacity curve converts those longer drafts into higher end-to-end serving speed without changing the target distribution.

What carries the argument

Confidence-scheduled verification: a calibrated confidence head supplies per-position prefix survival probabilities; a hardware-aware prefix scheduler greedily admits draft tokens by those probabilities until expected accepted tokens times the profiled steps-per-second curve is maximized, with an asynchronous top-K adaptation that preserves the lossless non-anticipating property.

Load-bearing premise

The engine’s steps-per-second depends mostly on the verification batch size, and an asynchronous schedule that uses confidence scores from two steps earlier still finds near-optimal, lossless lengths on real non-smooth hardware curves.

What would settle it

Under the same live traffic and engine configuration, disable the load-aware scheduler (force fixed full-block verification) and measure whether the claimed 60–85 percent per-user speedups at matched throughput and the non-collapse under the strict 120 / 50 tok/s/user SLAs disappear.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. DSpark unifies a semi-autoregressive drafter (parallel backbone plus a lightweight Markov or RNN sequential head) with confidence-scheduled, hardware-aware verification. The sequential head injects intra-block dependence to reduce the suffix decay of pure parallel drafting, while a calibrated confidence head and a greedy prefix scheduler (Algorithm 1, with asynchronous ZOS adaptations in §5.2) select per-request verification lengths to maximize expected throughput Θ = τ · SPS(B). Offline, with the scheduler disabled, DSpark raises accepted length over re-trained Eagle3 and DFlash across Qwen3-{4,8,14}B and Gemma4-12B on math, code, and chat. Deployed on DeepSeek-V4 under live traffic, it improves per-user generation speed by 60–85% (Flash) and 57–78% (Pro) at matched aggregate throughput versus the MTP-1 production baseline and extends the feasible interactivity frontier under strict SLAs.

Significance. If the reported gains hold, the paper is of clear practical importance for high-concurrency LLM serving: it shows how to keep long parallel drafts without collapsing batch capacity, and the live Pareto shift (Figures 7–8) is more informative than offline τ alone. Explicit strengths include matched re-training of Eagle3/DFlash on the same data and block size, position-wise conditional acceptance curves that separate capacity from dependence (Figure 2), post-hoc STS calibration with reliability diagrams, a concrete losslessness counterexample (Appendix A), and the release of DSpark checkpoints plus the DeepSpec training repository. These artifacts make the algorithmic claims checkable beyond the authors’ stack.

major comments (2)
  1. [§3.2.2, §5.2, Algorithm 1] §3.2.2 (incl. footnote 2) and §5.2: The central system claim rests on treating SPS(B) as predominantly a function of verification batch size and on the two-step-lagged asynchronous top-K remaining near-optimal and lossless on real jagged SPS curves. The manuscript scopes the first assumption to DeepSeek-V4’s disaggregated, load-balanced regime and restores causality via historical K plus rank-preserving live scores, but it does not quantify the throughput gap between the lagged policy and an oracle synchronous scheduler (or report sensitivity of Θ to lag). A short bound, offline replay on the profiled cost table, or ablation would make the 60–85% / 57–78% matched-throughput claims more transferable.
  2. [§5.4, Figure 7] §5.4 / Figure 7: Production comparisons are only against MTP-1. The text argues that static multi-token MTP was not production-viable under high concurrency, which is plausible, but without a controlled static multi-token (or fixed-threshold DSpark) arm under the same live traffic, the contribution of the adaptive scheduler versus the semi-AR draft quality alone cannot be isolated at the system level. Offline Table 1 already isolates draft quality; a production or replay ablation that freezes verification length would complete the causal picture for the Pareto shift.
minor comments (6)
  1. [Figure 2, §4.3.1] Figure 2 caption and §4.3.1: “conditional acceptance” is well defined in the text; restating the exact denominator (only prefixes accepted through k−1) in the figure legend would prevent misreading as ordinary prefix survival.
  2. [§3.2.1] Eq. (7)–(8) and §3.2.1: STS is described as left-to-right 1D grid search minimizing ECE of cumulative products; stating the validation-set size and whether temperatures are frozen after calibration would aid reproduction.
  3. [Table 1] Table 1: Macro-average relative gains are given in the text but not in the table; adding a bottom row (or supplementary macro column) would make the 16–31% claims easier to verify at a glance.
  4. [§3.1] §3.1: The minor DFlash input change (anchor as first prediction position) is stated to preserve quality; a one-line ablation of that change would remove residual ambiguity about backbone parity.
  5. [Figure 1, §3] Notation: “anchor token” and “bonus token” are declared interchangeable in a footnote; using one term consistently in Figure 1 and the main text would reduce cognitive load.
  6. [§6] Related work §6 correctly situates Domino/DFlare as concurrent; a single sentence on how the Markov head differs from Domino’s CausalEncoder (local low-rank bias vs. full causal encoder) would sharpen the novelty claim.

Circularity Check

0 steps flagged

No significant circularity; empirical speedups and accepted-length gains are measured against external/retrained baselines and live traffic, not forced by definition or self-citation chains.

full rationale

DSpark is an engineering systems paper whose central claims (higher accepted length offline; 60–85 % / 57–78 % per-user speedups and Pareto-frontier shift online) rest on direct measurements: Table 1 and Figures 2–4 report accepted lengths of retrained Eagle3/DFlash/DSpark on public math/code/chat suites; Figures 7–8 report live-traffic throughput–TPS curves versus the production MTP-1 baseline. The confidence head is supervised by the analytic total-variation acceptance rate (Eq. 8) of a frozen target and post-hoc calibrated by STS on held-out data; the hardware-aware scheduler (Alg. 1) maximizes a profiled SPS(B)·τ objective under an explicit non-anticipating early-stop (or its asynchronous causal approximation in §5.2). None of these steps define the reported metrics into existence, fit a parameter and then “predict” a near-identical quantity, or rest on an unverified uniqueness theorem imported from the same authors. Self-citations to DeepSeek-V3/V4 and internal engine details merely identify the production baseline and deployment substrate; they do not force the measured gains. The derivation chain is therefore self-contained against external benchmarks and open-sourced artifacts.

Axiom & Free-Parameter Ledger

5 free parameters · 3 axioms · 2 invented entities

The work rests on the standard speculative-decoding acceptance rule, a profiled engine capacity curve, and a handful of architectural hyper-parameters chosen for latency/quality trade-offs. No new physical entities are postulated; free parameters are ordinary ML knobs.

free parameters (5)
  • Markov rank r = 256
    Low-rank factorization dimension of the transition bias (default 256); chosen for storage/compute, not derived.
  • loss weights α_ce, α_tv, α_conf = 0.1 / 0.9 / 1.0
    Default 0.1 / 0.9 / 1.0; control the training objective balance.
  • position weight decay 1/γ
    Exponential position weighting w_k = exp(-(k-1)/γ) emphasizing early tokens.
  • STS temperature scalars = grid-searched
    Per-position temperatures found by 1-D grid search on held-out ECE of cumulative products.
  • max draft length γ = 5–7
    Block size (7 offline, 5 production); chosen under latency budget.
axioms (3)
  • domain assumption Standard speculative decoding with rejection sampling exactly preserves the target distribution when the non-anticipating property holds.
    Invoked throughout Sections 2–3 and Appendix A; foundational to the lossless claim.
  • domain assumption Engine step throughput SPS(B) is predominantly a function of verification batch size B under the paper's disaggregated, load-balanced regime.
    Stated as simplifying assumption 2 in Section 3.2.2; underpins the global throughput objective Θ = τ · SPS(B).
  • standard math Per-step acceptance probability equals 1 − ½‖p_d − p_t‖₁.
    Used to define soft labels c*_k and the TV loss (Leviathan et al. 2023).
invented entities (2)
  • Hardware-aware prefix scheduler (Algorithm 1 + async ZOS adaptation) independent evidence
    purpose: Dynamically select per-request verification lengths to maximize expected system throughput while preserving causality.
    Core system contribution; independent evidence is the live-traffic Pareto shift, but the entity itself is an algorithmic construct.
  • Semi-autoregressive draft head (Markov / RNN) independent evidence
    purpose: Inject intra-block token dependence on top of a parallel backbone without destroying O(1) draft latency.
    Architectural module; ablations show it reduces suffix decay relative to pure parallel DFlash.

pith-pipeline@v1.1.0-grok45 · 34539 in / 2845 out tokens · 26084 ms · 2026-07-11T08:04:25.461700+00:00 · methodology

0 comments
read the original abstract

Speculative decoding accelerates Large Language Model (LLM) inference by decoupling draft generation from target verification. While recent parallel drafters efficiently propose long token sequences in a single forward pass, they suffer from rapid acceptance decay due to a lack of inter-token dependencies. Furthermore, indiscriminately verifying these extended blocks wastes critical batch capacity on tokens with high rejection risks, severely degrading throughput in high-concurrency serving systems. We introduce DSpark, a speculative decoding framework that unifies high-throughput parallel generation with adaptive, load-aware verification. To maintain draft quality, DSpark utilizes a semi-autoregressive architecture, coupling a parallel backbone with a lightweight sequential module, to introduce intra-block dependency modeling and mitigate suffix decay. To optimize system efficiency, DSpark employs confidence-scheduled verification, dynamically tailoring the verification length for each request based on estimated prefix survival probabilities and engine-specific throughput profiles. On offline benchmarks across diverse domains, DSpark substantially improves the accepted length over state-of-the-art autoregressive and parallel drafters. When deployed within the DeepSeek-V4 serving system under live user traffic, DSpark successfully mitigates verification waste. Compared to the established production baseline (MTP-1), DSpark accelerates per-user generation speeds by 60 to 85 percent at matched throughput levels. More importantly, by preventing severe throughput degradation under strict interactivity constraints, it enables performance tiers that were previously unattainable, shifting the Pareto frontier of our serving system.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. From Expert Reduction to Behavioral Divergence: Tracing Numerical State through Sparse MoE Inference

    cs.LG 2026-07 conditional novelty 7.0

    Mathematically equivalent expert-reduction orders in native DeepSeek-V4-Flash produce distinct routes, persistent states, and continuation basins; BF16 operands with FP32 accumulation match the native path.

  2. SEED: Self-Evolving On-Policy Distillation for Agentic Reinforcement Learning

    cs.CL 2026-07 conditional novelty 6.0

    SEED improves agentic RL by converting a policy's own completed trajectories into natural-language skills and distilling the skill-induced probability shift back into the policy as dense token-level supervision.

  3. AngelSpec: Towards Real-World High Performance Inference with Speculative Decoding

    cs.CL 2026-07 conditional novelty 5.0

    AngelSpec + DFly pair a chat MTP drafter with a code/math block-diffusion drafter and load-aware verification pruning, reaching up to 2.4x AR throughput on Hy3-A21B.

  4. DeLS-Spec: Decoupled Long-Short Contexts for Parallel Speculative Drafting

    cs.CL 2026-07 conditional novelty 5.0

    DeLS-Spec improves block-parallel speculative decoding by fusing DFlash logits with an independently trained lightweight local head and a unigram prior correction.

Reference graph

Works this paper leans on

181 extracted references · 10 canonical work pages · cited by 4 Pith papers

  1. [1]

    2026 , eprint=

    TAPS: Target-Aware Prefix Tree Selection for Diffusion-Drafted Speculative Decoding , author=. 2026 , eprint=

    2026

  2. [3]

    2026 , eprint=

    Attention Drift: What Autoregressive Speculative Decoding Models Learn , author=. 2026 , eprint=

    2026

  3. [4]

    2026 , eprint=

    JetSpec: Breaking the Scaling Ceiling of Speculative Decoding with Parallel Tree Drafting , author=. 2026 , eprint=

    2026

  4. [6]

    2020 IEEE International Parallel and Distributed Processing Symposium (IPDPS) , year=

    Demystifying Tensor Cores to Optimize Half-Precision Matrix Multiply , author=. 2020 IEEE International Parallel and Distributed Processing Symposium (IPDPS) , year=

    2020

  5. [9]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Fr-spec: Accelerating large-vocabulary language models via frequency-ranked speculative sampling , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  6. [10]

    2024 , editor =

    Du, Cunxiao and Jiang, Jing and Yuanchen, Xu and Wu, Jiawei and Yu, Sicheng and Li, Yongqi and Li, Shenggui and Xu, Kai and Nie, Liqiang and Tu, Zhaopeng and You, Yang , booktitle =. 2024 , editor =

    2024

  7. [11]

    2024 , eprint=

    Recurrent Drafter for Fast Speculative Decoding in Large Language Models , author=. 2024 , eprint=

    2024

  8. [12]

    The Thirteenth International Conference on Learning Representations , year=

    MagicDec: Breaking the Latency-Throughput Tradeoff for Long Context Generation with Speculative Decoding , author=. The Thirteenth International Conference on Learning Representations , year=

  9. [13]

    2026 , howpublished =

    2026

  10. [15]

    2026 , eprint=

    Nightjar: Dynamic Adaptive Speculative Decoding for Large Language Models Serving , author=. 2026 , eprint=

    2026

  11. [17]

    2026 , eprint=

    Not-a-Bandit: Provably No-Regret Drafter Selection in Speculative Decoding for LLMs , author=. 2026 , eprint=

    2026

  12. [18]

    Proceedings of The 4th NeurIPS Efficient Natural Language and Speech Processing Workshop , pages =

    Dynamic Speculation Lookahead Accelerates Speculative Decoding of Large Language Models , author =. Proceedings of The 4th NeurIPS Efficient Natural Language and Speech Processing Workshop , pages =. 2024 , editor =

    2024

  13. [19]

    2026 , eprint=

    DFlare: Scaling Up Draft Capacity for Block Diffusion Speculative Decoding , author=. 2026 , eprint=

    2026

  14. [20]

    2026 , eprint=

    Domino: Decoupling Causal Modeling from Autoregressive Drafting in Speculative Decoding , author=. 2026 , eprint=

    2026

  15. [22]

    2026 , eprint=

    P-EAGLE: Parallel-Drafting EAGLE with Scalable Training , author=. 2026 , eprint=

    2026

  16. [23]

    2023 , eprint=

    REST: Retrieval-Based Speculative Decoding , author=. 2023 , eprint=

    2023

  17. [24]

    2026 , eprint=

    Draft Less, Retrieve More: Hybrid Tree Construction for Speculative Decoding , author=. 2026 , eprint=

    2026

  18. [25]

    2025 , url=

    Heming Xia and Yongqi Li and Jun Zhang and Cunxiao Du and Wenjie Li , booktitle=. 2025 , url=

    2025

  19. [29]

    2025 , eprint=

    FastMTP: Accelerating LLM Inference with Enhanced Multi-Token Prediction , author=. 2025 , eprint=

    2025

  20. [31]

    2023 , month =

    Prompt Lookup Decoding , author =. 2023 , month =

    2023

  21. [33]

    First Conference on Language Modeling , year=

    Hydra: Sequentially-Dependent Draft Heads for Medusa Decoding , author=. First Conference on Language Modeling , year=

  22. [36]

    Blockwise Parallel Decoding for Deep Autoregressive Models , url =

    Stern, Mitchell and Shazeer, Noam and Uszkoreit, Jakob , booktitle =. Blockwise Parallel Decoding for Deep Autoregressive Models , url =

  23. [37]

    2025 , eprint=

    Learning Harmonized Representations for Speculative Sampling , author=. 2025 , eprint=

    2025

  24. [38]

    Ninth Conference on Machine Learning and Systems , year=

    SpecDiff-2: Scaling Diffusion Drafter Alignment For Faster Speculative Decoding , author=. Ninth Conference on Machine Learning and Systems , year=

  25. [39]

    2025 , eprint=

    DiffuSpec: Unlocking Diffusion Language Models for Speculative Decoding , author=. 2025 , eprint=

    2025

  26. [40]

    Radiology , volume=

    The meaning and use of the area under a receiver operating characteristic (ROC) curve , author=. Radiology , volume=. 1982 , doi=

    1982

  27. [41]

    The Thirteenth International Conference on Learning Representations , year=

    Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling , author=. The Thirteenth International Conference on Learning Representations , year=

  28. [42]

    Advances in neural information processing systems , volume=

    Accelerating diffusion llms via adaptive parallel decoding , author=. Advances in neural information processing systems , volume=

  29. [43]

    Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages=

    A study of non-autoregressive model for sequence generation , author=. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages=

  30. [46]

    2026 , eprint=

    DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence , author=. 2026 , eprint=

    2026

  31. [47]

    arXiv e-prints , pages=

    Optimizing speculative decoding for serving large language models using goodput , author=. arXiv e-prints , pages=

  32. [51]

    2026 , eprint=

    SpecBound: Adaptive Bounded Self-Speculation with Layer-wise Confidence Calibration , author=. 2026 , eprint=

    2026

  33. [52]

    2026 , url=

    Zihao An and Huajun Bai and Ziqiong Liu and Dong Li and Emad Barsoum , booktitle=. 2026 , url=

    2026

  34. [53]

    Chenze Shao and Zhengrui Ma and Min Zhang and Yang Feng , booktitle=. Beyond. 2023 , url=

    2023

  35. [57]

    Proceedings of the AAAI conference on artificial intelligence , volume=

    Obtaining well calibrated probabilities using bayesian binning , author=. Proceedings of the AAAI conference on artificial intelligence , volume=

  36. [58]

    Advances in neural information processing systems , volume=

    Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift , author=. Advances in neural information processing systems , volume=

  37. [59]

    International conference on machine learning , pages=

    On calibration of modern neural networks , author=. International conference on machine learning , pages=. 2017 , organization=

    2017

  38. [60]

    2026 , url=

    Zhichao Wang and Dongyang Ma and Xinting Huang and Deng Cai and Tian Lan and Jiahao Xu and Haitao Mi and Xiaoying Tang and Yan Wang , booktitle=. 2026 , url=

    2026

  39. [63]

    Proceedings of the 52nd Annual International Symposium on Computer Architecture , pages=

    Insights into deepseek-v3: Scaling challenges and reflections on hardware for ai architectures , author=. Proceedings of the 52nd Annual International Symposium on Computer Architecture , pages=

  40. [64]

    18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24) , pages=

    \ DistServe \ : Disaggregating prefill and decoding for goodput-optimized large language model serving , author=. 18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24) , pages=

  41. [65]

    Proceedings of the 29th symposium on operating systems principles , pages=

    Efficient memory management for large language model serving with pagedattention , author=. Proceedings of the 29th symposium on operating systems principles , pages=

  42. [66]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Deepseekmoe: Towards ultimate expert specialization in mixture-of-experts language models , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  43. [67]

    Zhenda Xie and Yixuan Wei and Huanqi Cao and Chenggang Zhao and Chengqi Deng and Jiashi Li and Damai Dai and Huazuo Gao and Mingyu Xu and Kuai Yu and Liang Zhao and Shangyan Zhou and Zhean Xu and Zhengyan Zhang and Wangding Zeng and Shengding Hu and Yuqing Wang and Jingyang Yuan and Lean Wang and Wenfeng Liang , booktitle=. m. 2026 , url=

    2026

  44. [69]

    Proceedings of the 42nd International Conference on Machine Learning , pages =

    From Crowdsourced Data to High-quality Benchmarks: Arena-Hard and Benchbuilder Pipeline , author =. Proceedings of the 42nd International Conference on Machine Learning , pages =. 2025 , editor =

    2025

  45. [70]

    Gonzalez and Ion Stoica , month =

    Tianle Li and Wei-Lin Chiang and Evan Frick and Lisa Dunlap and Banghua Zhu and Joseph E. Gonzalez and Ion Stoica , month =. From Live Data to High-Quality Benchmarks: The Arena-Hard Pipeline , url =

  46. [71]

    Hashimoto , title =

    Rohan Taori and Ishaan Gulrajani and Tianyi Zhang and Yann Dubois and Xuechen Li and Carlos Guestrin and Percy Liang and Tatsunori B. Hashimoto , title =. GitHub repository , howpublished =. 2023 , publisher =

    2023

  47. [72]

    Advances in neural information processing systems , volume=

    Judging llm-as-a-judge with mt-bench and chatbot arena , author=. Advances in neural information processing systems , volume=

  48. [73]

    International Conference on Learning Representations , volume=

    Livecodebench: Holistic and contamination free evaluation of large language models for code , author=. International Conference on Learning Representations , volume=

  49. [74]

    2021 , eprint=

    Evaluating Large Language Models Trained on Code , author=. 2021 , eprint=

    2021

  50. [75]

    American Invitational Mathematics Examination (AIME) 2025 , author=

    2025

  51. [76]

    The Twelfth International Conference on Learning Representations , year=

    Let's Verify Step by Step , author=. The Twelfth International Conference on Learning Representations , year=

  52. [79]

    2026 , howpublished =

    Gemma 4 Model Card , author =. 2026 , howpublished =

    2026

  53. [83]

    International Conference on Learning Representations , year=

    Non-Autoregressive Neural Machine Translation , author=. International Conference on Learning Representations , year=

  54. [84]

    Proceedings of the 39th International Conference on Machine Learning , pages =

    On the Learning of Non-Autoregressive Transformers , author =. Proceedings of the 39th International Conference on Machine Learning , pages =. 2022 , editor =

    2022

  55. [85]

    Proceedings of the 40th International Conference on Machine Learning , pages =

    Fast Inference from Transformers via Speculative Decoding , author =. Proceedings of the 40th International Conference on Machine Learning , pages =. 2023 , editor =

    2023

  56. [86]

    The Twelfth International Conference on Learning Representations , year=

    DistillSpec: Improving Speculative Decoding via Knowledge Distillation , author=. The Twelfth International Conference on Learning Representations , year=

  57. [87]

    Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3 , pages=

    Specinfer: Accelerating large language model serving with tree-based speculative inference and verification , author=. Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3 , pages=

  58. [88]

    2024 , editor =

    Li, Yuhui and Wei, Fangyun and Zhang, Chao and Zhang, Hongyang , booktitle =. 2024 , editor =

    2024

  59. [90]

    2026 , url=

    Yuhui Li and Fangyun Wei and Chao Zhang and Hongyang Zhang , booktitle=. 2026 , url=

    2026

  60. [91]

    and Chen, Deming and Dao, Tri , booktitle =

    Cai, Tianle and Li, Yuhong and Geng, Zhengyang and Peng, Hongwu and Lee, Jason D. and Chen, Deming and Dao, Tri , booktitle =. Medusa: Simple. 2024 , editor =

    2024

  61. [92]

    Proceedings of the 41st International Conference on Machine Learning , pages =

    Better & Faster Large Language Models via Multi-token Prediction , author =. Proceedings of the 41st International Conference on Machine Learning , pages =. 2024 , editor =

    2024

  62. [95]

    Proceedings of the 38th International Conference on Machine Learning , pages =

    Order-Agnostic Cross Entropy for Non-Autoregressive Machine Translation , author =. Proceedings of the 38th International Conference on Machine Learning , pages =. 2021 , editor =

    2021

  63. [98]

    Advances in Neural Information Processing Systems , editor=

    Structured Denoising Diffusion Models in Discrete State-Spaces , author=. Advances in Neural Information Processing Systems , editor=. 2021 , url=

    2021

  64. [99]

    Diffusion-

    Xiang Lisa Li and John Thickstun and Ishaan Gulrajani and Percy Liang and Tatsunori Hashimoto , booktitle=. Diffusion-. 2022 , url=

    2022

  65. [100]

    The Thirteenth International Conference on Learning Representations , year=

    Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models , author=. The Thirteenth International Conference on Learning Representations , year=

  66. [102]

    Fast Structured Decoding for Sequence Models , url =

    Sun, Zhiqing and Li, Zhuohan and Wang, Haoqing and He, Di and Lin, Zi and Deng, Zhihong , booktitle =. Fast Structured Decoding for Sequence Models , url =

  67. [105]

    Proceedings of the 39th International Conference on Machine Learning , pages =

    Directed Acyclic Transformer for Non-Autoregressive Machine Translation , author =. Proceedings of the 39th International Conference on Machine Learning , pages =. 2022 , editor =

    2022

  68. [106]

    Thirty-seventh Conference on Neural Information Processing Systems , year=

    Non-autoregressive Machine Translation with Probabilistic Context-free Grammar , author=. Thirty-seventh Conference on Neural Information Processing Systems , year=

  69. [107]

    Proceedings of the 35th International Conference on Machine Learning , pages =

    Fast Decoding in Sequence Models Using Discrete Latent Variables , author =. Proceedings of the 35th International Conference on Machine Learning , pages =. 2018 , editor =

    2018

  70. [108]

    Proceedings of the 19th USENIX Conference on Operating Systems Design and Implementation , articleno =

    Zhu, Kan and Gao, Yufei and Zhao, Yilong and Zhao, Liangyu and Zuo, Gefei and Gu, Yile and Xie, Dedong and Tang, Tian and Xu, Qinyu and Ye, Zihao and Kamahori, Keisuke and Lin, Chien-Yu and Wang, Ziren and Wang, Stephanie and Krishnamurthy, Arvind and Kasikci, Baris , title =. Proceedings of the 19th USENIX Conference on Operating Systems Design and Imple...

    2025

  71. [109]

    2026 , eprint=

    DART: Diffusion-Inspired Speculative Decoding for Fast LLM Inference , author=. 2026 , eprint=

    2026

  72. [110]

    Abramovich, M

    T. Abramovich, M. Ashkenazi, I. Putterman, B. Chislett, T. Mitra, B. D. Rouhani, R. Zilberstein, and Y. Geifman. Speed-bench: A unified and diverse benchmark for speculative decoding. arXiv preprint arXiv:2604.09557, 2026

    Pith/arXiv arXiv 2026

  73. [111]

    Z. An, H. Bai, Z. Liu, D. Li, and E. Barsoum. PARD : Accelerating LLM inference with low -cost PAR allel draft model adaptation. In The Fourteenth International Conference on Learning Representations, 2026 a . URL https://openreview.net/forum?id=XbOyv7iVGL

    2026

  74. [112]

    Z. An, T. Liu, Z. Liu, D. Li, R. Liu, and E. Barsoum. Pard-2: Target-aligned parallel draft model for dual-mode speculative decoding. arXiv preprint arXiv:2605.08632, 2026 b

    Pith/arXiv arXiv 2026

  75. [113]

    D-Cut : Adaptive verification depth pruning for speculative decoding, 2026

    AngelSlim Team . D-Cut : Adaptive verification depth pruning for speculative decoding, 2026. URL https://angelslim.readthedocs.io/zh-cn/latest/dcut.html

    2026

  76. [114]

    Ankner, R

    Z. Ankner, R. Parthasarathy, A. Nrusimha, C. Rinard, J. Ragan-Kelley, and W. Brandon. Hydra: Sequentially-dependent draft heads for medusa decoding. In First Conference on Language Modeling, 2024. URL https://openreview.net/forum?id=FbhjirzvJG

    2024

  77. [115]

    Arriola, S

    M. Arriola, S. S. Sahoo, A. Gokaslan, Z. Yang, Z. Qi, J. Han, J. T. Chiu, and V. Kuleshov. Block diffusion: Interpolating between autoregressive and diffusion language models. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=tyEyYT267x

    2025

  78. [116]

    Austin, D

    J. Austin, D. D. Johnson, J. Ho, D. Tarlow, and R. van den Berg. Structured denoising diffusion models in discrete state-spaces. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. W. Vaughan, editors, Advances in Neural Information Processing Systems, 2021 a . URL https://openreview.net/forum?id=h7-XixPCAL

    2021

  79. [117]

    Austin, A

    J. Austin, A. Odena, M. Nye, M. Bosma, H. Michalewski, D. Dohan, E. Jiang, C. Cai, M. Terry, Q. Le, et al. Program synthesis with large language models. arXiv preprint arXiv:2108.07732, 2021 b

    Pith/arXiv arXiv 2021

  80. [118]

    T. Cai, Y. Li, Z. Geng, H. Peng, J. D. Lee, D. Chen, and T. Dao. Medusa: Simple LLM inference acceleration framework with multiple decoding heads. In R. Salakhutdinov, Z. Kolter, K. Heller, A. Weller, N. Oliver, J. Scarlett, and F. Berkenkamp, editors, Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machi...

    2024

Showing first 80 references.