Pith. sign in

REVIEW 10 cited by

Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2401.07851 v3 pith:RQB5FVJO submitted 2024-01-15 cs.CL

Heming Xia , Zhe Yang , Qingxiu Dong , Peiyi Wang , Yongqi Li , Tao Ge , Tianyu Liu , Wenjie Li
show 1 more author
Zhifang Sui
This is my paper · ORCID
classification cs.CL
keywords decodinginferencespeculativeanalysisautoregressivecomprehensivelanguagelarge
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

To mitigate the high inference latency stemming from autoregressive decoding in Large Language Models (LLMs), Speculative Decoding has emerged as a novel decoding paradigm for LLM inference. In each decoding step, this method first drafts several future tokens efficiently and then verifies them in parallel. Unlike autoregressive decoding, Speculative Decoding facilitates the simultaneous decoding of multiple tokens per step, thereby accelerating inference. This paper presents a comprehensive overview and analysis of this promising decoding paradigm. We begin by providing a formal definition and formulation of Speculative Decoding. Then, we organize in-depth discussions on its key facets, such as drafter selection and verification strategies. Furthermore, we present a comparative analysis of leading methods under third-party testing environments. We aim for this work to serve as a catalyst for further research on Speculative Decoding, ultimately contributing to more efficient LLM inference.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Revisiting Lossy Verification in Speculative Decoding: Mechanisms, Trade-offs, and Failure Modes

    cs.CL 2026-07 conditional novelty 6.0 of 10

    Lossy speculative-decoding verification splits into truncation-based and collaborative methods; truncation-based methods underperform their matched baselines, and capping draft overshoot preserves quality.

  2. PyroDash: Cost-Efficient Token-Level Small-Large Language Model Collaborative Inference

    cs.CL 2026-07 conditional novelty 6.0 of 10

    PyroDash lets a small model decide mid-generation to hand off once to a frozen LLM, achieving 64.04% accuracy (+6.36 points vs LLM) at lambda=0.05 and 96.4% lower estimated cost at lambda=0.6.

  3. AdaDecode: Accelerating LLM Decoding with Adaptive Layer Parallelism

    cs.CL 2025-06 conditional novelty 6.0 of 10

    AdaDecode speeds up LLM generation by predicting tokens at early layers when confidence is high, running the skipped layers in parallel, and verifying the output exactly matches standard decoding.

  4. Think Before You Accept: Semantic Reflective Verification for Faster Speculative Decoding

    cs.LG 2025-05 conditional novelty 6.0 of 10

    Reflective Verification fuses a target LLM's normal and reflection-prompted logits to accept semantically correct draft tokens, increasing accepted draft length and decoding speed by 5-15%.

  5. VeriThinker: Learning to Verify Makes Reasoning Model Efficient

    cs.LG 2025-05 conditional novelty 6.0 of 10

    VeriThinker shows that fine-tuning a reasoning model only on a solution-verification task reduces chain-of-thought length on MATH500 and AIME by 20-45% while preserving or slightly improving accuracy.

  6. AngelSpec: Towards Real-World High Performance Inference with Speculative Decoding

    cs.CL 2026-07 conditional novelty 5.0 of 10

    AngelSpec + DFly pair a chat MTP drafter with a code/math block-diffusion drafter and load-aware verification pruning, reaching up to 2.4x AR throughput on Hy3-A21B.

  7. D-cut: Adaptive Verification Depth Pruning for Batched Speculative Decoding

    cs.CL 2026-07 conditional novelty 5.0 of 10

    D-Cut prunes low-confidence draft suffixes across a batch of requests, using a confidence-based ranking and a profiled hardware cost model, to raise batched speculative decoding speedup from 1.26x to 1.65x.

  8. DeltaLLM: A Training-Free Framework Exploiting Temporal Sparsity for Efficient Edge LLM Inference

    cs.AI 2025-07 conditional novelty 5.0 of 10

    DeltaLLM exploits temporal sparsity in key vectors (delta encoding) with a hybrid local-window attention to cut attention computation by up to 60% on 1-2B parameter LLMs with minimal accuracy loss.

  9. S$^4$C: Speculative Sampling with Syntactic and Semantic Coherence for Efficient Inference of Large Language Models

    cs.CL 2025-06 conditional novelty 4.0 of 10

    S4C accelerates LLM generation by combining multi-head autoregressive draft heads with a continuous verification tree, measuring 2.26x to 2.60x speedups on Spec-bench.

  10. Consultant Decoding: Yet Another Synergistic Mechanism

    cs.CL 2025-06 conditional novelty 4.0 of 10

    Consultant Decoding speeds up LLM generation by accepting draft tokens whose negative log-likelihood under the target model falls below a fixed threshold, reaching 2-3x speedups with comparable quality.

Pith tools