Pith. sign in

REVIEW 4 cited by

ParallelSpec: Parallel Drafter for Efficient Speculative Decoding

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2410.05589 v1 pith:3C3RFU6H submitted 2024-10-08 cs.CL cs.LG

Zilin Xiao , Hongming Zhang , Tao Ge , Siru Ouyang , Vicente Ordonez , Dong Yu This is my paper · ORCID
classification cs.CLcs.LG
keywords speculativemodeldecodingdrafterparallelparallelspecefficienttokens
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Speculative decoding has proven to be an efficient solution to large language model (LLM) inference, where the small drafter predicts future tokens at a low cost, and the target model is leveraged to verify them in parallel. However, most existing works still draft tokens auto-regressively to maintain sequential dependency in language modeling, which we consider a huge computational burden in speculative decoding. We present ParallelSpec, an alternative to auto-regressive drafting strategies in state-of-the-art speculative decoding approaches. In contrast to auto-regressive drafting in the speculative stage, we train a parallel drafter to serve as an efficient speculative model. ParallelSpec learns to efficiently predict multiple future tokens in parallel using a single model, and it can be integrated into any speculative decoding framework that requires aligning the output distributions of the drafter and the target model with minimal training cost. Experimental results show that ParallelSpec accelerates baseline methods in latency up to 62% on text generation benchmarks from different domains, and it achieves 2.84X overall speedup on the Llama-2-13B model using third-party evaluation criteria.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MineDraft: A Framework for Batch Parallel Speculative Decoding

    cs.CL 2026-02 conditional novelty 6.0 of 10

    MineDraft speeds up speculative decoding by keeping two request batches — drafting one batch while verifying the other — gaining up to ~40% per-GPU throughput at moderate draft lengths.

  2. SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding

    cs.DC 2026-02 unverdicted novelty 6.0 of 10

    SPEED-Bench is a new standardized benchmark for speculative decoding that supplies semantically diverse qualitative data and throughput-oriented splits across concurrency levels, integrated with vLLM and TensorRT-LLM.

  3. Your LLM Knows the Future: Uncovering Its Multi-Token Prediction Potential

    cs.CL 2025-07 conditional novelty 5.0 of 10

    A gated LoRA and a small sampler let an autoregressive LLM draft multiple future tokens per step, and self-speculative verification converts those drafts into up to roughly 5x fewer generation steps.

  4. SpecASR: Accelerating LLM-based Automatic Speech Recognition via Speculative Decoding

    eess.AS 2025-07 reject novelty 4.0 of 10

    SpecASR accelerates LLM-based ASR by 3.04x-3.79x over autoregressive decoding using adaptive draft lengths, draft token recycling, and sparse token trees, but the speedups are simulated from Whisper proxy models rathe...

Pith tools