Pith. sign in

REVIEW 7 cited by

Dynamic Speculation Lookahead Accelerates Speculative Decoding of Large Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2405.04304 v5 pith:GBPHB5LS submitted 2024-05-07 cs.CL

Jonathan Mamou , Oren Pereg , Daniel Korat , Moshe Berchansky , Nadav Timor , Moshe Wasserblat , Roy Schwartz This is my paper
classification cs.CL
keywords lookaheadspeculationdecodingdiscodynamiclanguagelargemodels
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Speculative decoding is commonly used for reducing the inference latency of large language models. Its effectiveness depends highly on the speculation lookahead (SL)-the number of tokens generated by the draft model at each iteration. In this work we show that the common practice of using the same SL for all iterations (static SL) is suboptimal. We introduce DISCO (DynamIc SpeCulation lookahead Optimization), a novel method for dynamically selecting the SL. Our experiments with four datasets show that DISCO reaches an average speedup of 10% compared to the best static SL baseline, while generating the exact same text.

Discussion (0). Sign in to comment.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. A Sparse Glimpse of the Whole: Train-Free Self-Speculative Decoding

    cs.CL 2026-07 conditional novelty 6.0 of 10

    A training-free self-speculative decoding system with a recallable sparse KV cache and entropy-guided adaptive speculation achieves up to 2.79× speedup while preserving the target distribution.

  2. Utility-Driven Speculative Decoding for Mixture-of-Experts

    cs.DC 2025-06 conditional novelty 6.0 of 10

    Cascade is a utility-driven speculation manager that makes speculative decoding practical for MoE LLMs by disabling it when expert-activation cost exceeds token gain and hill-climbing to the best speculation length.

  3. AdaptiveSD A Stability-Aware, Runtime-Adaptive Speculative Decoding Framework with Multi-Policy Orchestration for CPU-Constrained LLM Inference

    cs.LG 2026-07 conditional novelty 5.5 of 10

    A runtime-adaptive speculative decoder with an 11-rule hierarchy and multi-policy engine keeps wasted draft compute under ~32% and bounds latency variance on CPU-constrained GGUF inference.

  4. AngelSpec: Towards Real-World High Performance Inference with Speculative Decoding

    cs.CL 2026-07 conditional novelty 5.0 of 10

    AngelSpec + DFly pair a chat MTP drafter with a code/math block-diffusion drafter and load-aware verification pruning, reaching up to 2.4x AR throughput on Hy3-A21B.

  5. Advancing Decoding Strategies: Enhancements in Locally Typical Sampling for LLMs

    cs.CL 2025-06 reject novelty 5.0 of 10

    ASTS extends locally typical sampling with semantic scoring and dynamic thresholds, reporting improved perplexity, MAUVE, and diversity on story and summarization tasks.

  6. Consultant Decoding: Yet Another Synergistic Mechanism

    cs.CL 2025-06 conditional novelty 4.0 of 10

    Consultant Decoding speeds up LLM generation by accepting draft tokens whose negative log-likelihood under the target model falls below a fixed threshold, reaching 2-3x speedups with comparable quality.

  7. AutoChemSchematic AI: Agentic Physics-Aware Automation for Chemical Manufacturing Scale-Up

    cs.LG 2025-05 reject novelty 4.0 of 10

    The framework trains small models on synthetic AI-generated data to produce PFD/PID text, then validates two examples by manual DWSIM setup, leaving the industrial-viability claim unproven.

Pith tools