REVIEW 3 major objections 4 minor 37 references
AsymSpec: Efficient Cloud-Edge Speculative Decoding over Asymmetric Networks
T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read AsymSpec claims that cloud–edge speculative decoding can keep a stable 438–446 tokens/s across strong, medium, and weak asymmetric networks, and beat the strongest baseline by 2.82–28.03×, by shipping acceptance data only and certifying…
desk verdict A genuinely new and mathematically sound protocol for cloud-edge speculative decoding, but the headline speedup rests on a high-concurrency workload assumption that the paper itself makes explicit. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
Two mechanisms carry the argument. The first is the residual-distribution TV certificate. At a rejected position, with target distribution $p$, draft distribution $q$, residual $r(v)=[p(v)-q(v)]_+/Z$, and returned support $S=\mathrm{TopK}(p)$, the edge computes $\delta_p=1-\sum_{v\in S}p(v)$ and $Z_S=\sum_{v\in S}[p(v)-q(v)]_+$, and Proposition 1 bounds the error of the truncated residual $\hat r$ by $D_{\mathrm{TV}}(r,\hat r)=Z_{\mathrm{tail}}/(Z_S+Z_{\mathrm{tail}}) \le \delta_p/(Z_S+\delta_p)$. This certificate decides, per rejection, whether a small support suffices within the request's accumulated TV budget, or whether the protocol must escalate through proposal-based exact recovery to full fallback. The second is the confirmed-prefix invariant $N_{\mathrm{open}}(\rho)\le 1$: a block is open from the start of drafting until it commits or aborts, and no request may have more than one open block, so the edge only generates independent, already-valid work while a verification is pending, and the cloud can re-batch arrived blocks across requests.
What would settle it
Run the end-to-end system with exactly one active request at a time, or with arrival intervals longer than the verification round-trip, and measure output-token throughput across the three network profiles; if AsymSpec then converges to or falls below the stop-and-wait baseline rather than preserving the cross-request gain, the confirmed-prefix pipeline's reliance on a ready pool of independent requests is the load-bearing condition.
Extended reading notes
Core claim
The central claim is that the verification exchange and the scheduling policy of cloud–edge speculative decoding can be reshaped to fit asymmetric bandwidth and to eliminate invalid dependent work, and that the reshaped system remains distribution-faithful. On the communication side, the common acceptance path carries only the candidate token and its draft probability for each position; on the rare rejection, the cloud returns a small top-K support, and the edge decides from a computable certificate whether the truncated residual distribution is close enough to the exact residual. If the certificate fails, the protocol escalates to proposal-based exact recovery via rejection sampling and only as a last resort sends the full vocabulary. On the scheduling side, the confirmed-prefix invariant $N_{\mathrm{open}}(\rho) \le 1$ permits at most one open block per request, so the edge never drafts descendants of an unresolved block; instead it drafts other ready requests, and the cloud re-batches arrived blocks independently of arrival grouping. The paper reports that at all 18 evaluated operating points AsymSpec attains the highest output-token throughput, with geometric-mean ratios of 2.82–28.03× over the strongest baseline and a 1.9% change in geometric-mean throughput from Strong to Weak, while every baseline loses roughly 71–79% over the same range.
Load-bearing premise
The speedup assumes that, whenever a request's block is being verified, the edge has another independent request with a confirmed prefix ready to draft; with a single active request or sparse arrivals the confirmed-frontier rule forces stop-and-wait, so the headline 2.82–28.03× gain would not materialize.
Editorial extensions
If this is right
- Cloud–edge serving on cellular or contended uplinks can stop uploading per-position vocabulary distributions; only token–probability pairs travel on the common path, so the uplink cost per block becomes $O(\gamma)$ instead of $O(\gamma|V|)$.
- A verifier can absorb asymmetric bandwidth by moving correction detail to the downlink and expanding support only on rejection, so a weak uplink no longer starves the target model while downlink capacity sits unused.
- Because throughput is stable from the Strong to the Weak profile, deployments on weaker links can expect similar output-token throughput to strong links, provided enough confirmed-prefix requests keep the edge busy.
- Same-request runahead, whose invalidated work grows with draft depth and rejection probability, is replaced by cross-request overlap; the deliverable is valid committed work rather than raw edge utilization.
- Bounded correction supports up to $K=8192$ track full-vocabulary task accuracy within about half a percentage point on average while compressing the correction payload, so the fidelity–communication tradeoff is acceptable in the tested range.
Reading between the lines
- Inference: the paper's own ablation shows that replacing the confirmed-prefix pipeline with same-request runahead keeps only about 76% of full throughput, so a natural extension is an adaptive policy that relaxes $N_{\mathrm{open}}\le 1$ only when the ready pool is empty, using runahead with an explicit invalidation budget; the paper does not propose such a policy.
- Inference: the per-event certificate could be reused as a control signal for choosing $K$ by predicted acceptance probability rather than by fixed escalation, potentially cutting downlink traffic further; no such predictor appears in the paper.
- Inference: the confirmed-prefix scheduling idea is not specific to LLM decoding — any two-stage verify-after-propose pipeline with many independent jobs and a constrained upload link could adopt the same invariant, for example edge inference pipelines with checkpoints; this extension goes beyond the paper's claims.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents AsymSpec, a cloud-edge speculative decoding system designed for asymmetric networks. The system uses an acceptance-sufficient uplink (token--probability pairs rather than full distributions), a certified progressive-correction protocol that returns target top-K support on the downlink and expands it or falls back to exact recovery based on a total-variation certificate, and a confirmed-prefix pipeline that forbids same-request runahead while overlapping independent requests. The authors evaluate AsymSpec on three draft/target model pairs, two workloads, and three network profiles, reporting 2.82--28.03x output-token throughput over the strongest baseline across 18 operating points. The protocol math is clearly derived: Proposition 1's residual-TV bound follows from Z_tail <= delta_p, Corollary 1 is a standard coupling argument, and Eq. (10) correctly instantiates rejection sampling as a two-endpoint correction protocol.
Significance. The paper addresses a real and increasingly relevant bottleneck: deploying speculative decoding across an asymmetric edge-cloud link. Its conceptual contribution is a clean separation between acceptance-sufficient uploads and rejection-only downlink corrections, plus a scheduling invariant that eliminates invalidated dependent work. The theoretical core is sound, self-contained, and does not rely on fitted constants; the design is well motivated by the latency breakdown in Section III-A and the payload imbalance in Table I. If the claimed throughputs are reproducible, AsymSpec would be a useful addition to the cloud-edge serving literature. The main weakness is empirical: the headline speedup is demonstrated only under a single, saturating high-concurrency workload, which is exactly the regime where the confirmed-prefix pipeline is advantaged. The paper is transparent about this mechanism, but it does not test the low-concurrency regime in which the invariant could hurt.
major comments (3)
- [V-B, IV-C1] The headline throughput claim (2.82--28.03x, abstract and Section V-B) is load-bearing and depends on a continuously nonempty ready pool of confirmed-prefix requests. As Section IV-C1 states, under invariant (11) the edge waits when no other request is ready; same-request runahead is forbidden. The evaluation in Section V uses only 100 requests at 4 requests/s, a saturating workload in which dozens of requests are concurrently active, so the ready pool is almost always populated. No low-concurrency, single-stream, or bursty-arrival experiments are reported. Under low concurrency AsymSpec degenerates closer to stop-and-wait, and a baseline such as PipeInfer that performs optimistic same-request runahead could plausibly output more tokens. The paper should either add experiments across request arrival rates and concurrency levels or explicitly qualify the throughput claim as limited to the saturated regime.
- [V-A, IV-B] Key protocol parameters that determine the communication/fidelity tradeoff are not reported: the request-level TV budget epsilon_req (Corollary 1), the initial target support size K and its expansion schedule, the proposal cap Nprop in the exact-recovery path, the cloud re-batching queue thresholds mentioned in Section IV-C2, and the draft depth gamma used in the experiments. Without these values the component analysis in Table II and the sensitivity experiments in Figures 6--7 cannot be reproduced, and the claim that the certificate-based selection avoids a workload-specific threshold (Section IV-B2) is not substantiated. Please report the actual values and any tuning procedure.
- [V-D, IV-B2] The evaluation does not directly validate the certified budget mechanism of Corollary 1. The protocol dynamically selects K per rejection so that the certificate satisfies the request-level error budget, but Figures 6 and 7 evaluate fixed K settings instead. Consequently, Figure 6 reports task accuracy for each fixed K, not the end-to-end trace-level TV distance that the protocol is designed to bound. The paper should report the realized distribution of certificate values, the chosen epsilon_req, and the resulting actual residual TV, or explain why fixed-K accuracy experiments are sufficient evidence for the certified-fidelity claim.
minor comments (4)
- [V-C, Table II] Table II does not state which model--workload pair the component analysis uses; the Full throughput values (~750 tokens/s) differ substantially from the geometric-mean values across all pairs (438--446 tokens/s), so the table likely refers to a single representative pair. Please specify the pair.
- [III-B, Table I] Table I's upload payload estimates do not state the precision assumption for the probability values. For example, 4 candidates on a 151,936-token vocabulary give 1.159 MiB, which is consistent with fp16 values (4*151936*2 bytes); please make the encoding explicit so the numbers are reproducible.
- [V-B, Figure 4] No error bars, confidence intervals, or number of independent runs are reported for the throughput measurements. Since the paper makes a strong claim about AsymSpec winning at all 18 operating points and about network insensitivity, some measure of run-to-run variance is needed.
- [III-C, Eq. (5)] In Eq. (5), the off-critical-path interval is written as T_of f with inconsistent spacing; please use a consistent notation such as T_off throughout.
Circularity Check
No significant circularity: AsymSpec's correctness proofs are derived from definitions and its throughput claims are measured against external baselines under stated workload conditions.
full rationale
The paper's derivation chain is self-contained rather than circular. The residual-distribution TV certificate (Proposition 1, Eq. 8) is proven directly from the definitions of the residual r, the support-truncated approximation hat-r, and the tail-mass bound Ztail ≤ δp; no fitted constant or predicted quantity is involved. Corollary 1 (Eq. 9) is a standard coupling argument that composes per-correction TV errors into a request-level budget, again purely definitional. The exact recovery path (Eq. 10) instantiates the textbook rejection-sampling identity and preserves the target distribution by construction. The confirmed-prefix invariant N_open(rho) ≤ 1 (Eq. 11) is an explicit design choice, not a consequence of fitting data, and the paper states its cost: the edge waits when no confirmed-prefix request is ready. The headline 2.82–28.03x throughput figure is an empirical measurement against three named baselines (Standard Spec, CoSine, PipeInfer) across 18 operating points, not a number derived from the protocol's own equations; the protocol parameters epsilon_req and K are adaptive runtime choices, not parameters fitted to reproduce the reported curves. The only notable caveat is workload scope: the evaluation uses 100 requests at 4 requests/s, a saturating high-concurrency regime, so the ready pool is usually populated. That is a validity limitation on the empirical claim, not a circular step, and the paper's own invariant makes the dependency explicit rather than concealing it. No self-citation is load-bearing, no uniqueness theorem is imported from the authors' prior work, and no known result is merely renamed. The correctness and fidelity claims therefore stand on their own derivations, and the performance claim stands or falls on the reported experiments, not on the protocol's structure.
Assumptions & free parameters
free parameters (5)
- Request-level TV budget epsilon_req =
not reported
- Correction support size K =
not reported for main end-to-end run
- Proposal cap Nprop =
not reported
- Cloud re-batching queue thresholds =
not reported
- Draft depth gamma =
not reported
assumptions (5)
- standard math Standard speculative decoding acceptance and residual-correction rules preserve the target output distribution.
- domain assumption The draft and target distributions are categorical over the same vocabulary and are represented exactly by the canonical finite-precision values used for sampling at both endpoints.
- domain assumption The target model can verify all candidate positions in a draft block in a single parallel forward pass.
- domain assumption The edge can query its retained draft distribution q_i(v) for arbitrary tokens v on demand.
- domain assumption The network model assumes the uplink is constrained while the downlink is stronger, and communication plus verification waiting dominate end-to-end latency.
Cite this review
Pith. "Pith review of AsymSpec: Efficient Cloud-Edge Speculative Decoding over Asymmetric Networks." pith.science (2026). https://pith.science/paper/5GZ5N3XK
@misc{pith2026260804974,
author = {Pith},
title = {Pith review of: AsymSpec: Efficient Cloud-Edge Speculative Decoding over Asymmetric Networks},
year = {2026},
howpublished = {\url{https://pith.science/paper/5GZ5N3XK}},
note = {Machine review of arXiv:2608.04974}
}
abstract
Cloud-edge speculative decoding places a lightweight draft model at an edge gateway and a higher-quality target model in the cloud, but inserts communication into every speculative block. Under a constrained uplink, candidate messages may queue while the verifier is idle. Stop-and-wait scheduling leaves edge compute underutilized; optimistic same-request runahead can waste work when a rejection or an unexpected bonus token invalidates dependent drafts. We present AsymSpec, which addresses uplink-gated verification and invalid dependent work with two corresponding mechanisms. Its asymmetric verification protocol keeps the common-path acceptance upload compact and moves richer, rejection-only correction information to the downlink. A total-variation (TV) certificate for the residual distribution determines whether a small target top-K response suffices; if not, the protocol progressively escalates through proposal-based exact recovery before falling back to the full distribution. Its confirmed-prefix pipeline exposes only independent, valid requests to the edge scheduler and lets the cloud re-batch arrived blocks, hiding verification waits when another confirmed-prefix request is ready without using same-request runahead. Across three draft-target pairs, two workloads, and three asymmetric network profiles, our end-to-end evaluation shows that AsymSpec delivers 2.82-28.03$\times$ the output-token throughput of the strongest baseline.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
LaMDA: Language models for dialog applications,
R. Thoppilan, D. D. Freitas, J. Hall, N. Shazeer, A. Kulshreshthaet al., “LaMDA: Language models for dialog applications,”arXiv preprint arXiv:2201.08239, 2022
arXiv 2022
-
[2]
Evaluating large language models trained on code,
M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. de Oliveira Pinto, J. Kaplan, H. Edwards, Y . Burda, N. Joseph, G. Brockmanet al., “Evaluating large language models trained on code,”arXiv preprint arXiv:2107.03374, 2021
arXiv 2021
-
[3]
Language models are few-shot learners,
T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal et al., “Language models are few-shot learners,” inAdvances in Neural Information Processing Systems, vol. 33, 2020, pp. 1877–1901
work page 2020
-
[4]
Chain-of-thought prompting elicits reasoning in large language models,
J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. H. Chi, Q. V . Le, and D. Zhou, “Chain-of-thought prompting elicits reasoning in large language models,” inAdvances in Neural Information Processing Systems, vol. 35, 2022, pp. 24 824–24 837
2022
-
[5]
ReAct: Synergizing reasoning and acting in language models,
S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. R. Narasimhan, and Y . Cao, “ReAct: Synergizing reasoning and acting in language models,” inThe Eleventh International Conference on Learning Representations, 2023
work page 2023
-
[6]
Fast inference from transform- ers via speculative decoding,
Y . Leviathan, M. Kalman, and Y . Matias, “Fast inference from transform- ers via speculative decoding,” inProceedings of the 40th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 202, 2023, pp. 19 274–19 286
work page 2023
-
[7]
Accelerating large language model decoding with speculative sampling,
C. Chen, S. Borgeaud, G. Irving, J.-B. Lespiau, L. Sifre, and J. Jumper, “Accelerating large language model decoding with speculative sampling,” arXiv preprint arXiv:2302.01318, 2023
arXiv 2023
-
[8]
J. Ning, C. Zheng, and T. Yang, “DSSD: Efficient edge-device LLM deployment and collaborative inference via distributed split speculative decoding,”arXiv preprint arXiv:2507.12000, 2025
arXiv 2025
Show all 37 references
-
[9]
SpecEdge: Scalable edge-assisted serving framework for interactive LLMs,
J. Park, S. Cho, and D. Han, “SpecEdge: Scalable edge-assisted serving framework for interactive LLMs,”arXiv preprint arXiv:2505.17052, 2025
2025
-
[10]
PipeSD: An efficient cloud-edge collaborative pipeline inference frame- work with speculative decoding,
Y . Han, Y . Gao, B. Hu, M. B. Mashhadi, Y . Duan, P. Xiao, and Y . Zhang, “PipeSD: An efficient cloud-edge collaborative pipeline inference frame- work with speculative decoding,”arXiv preprint arXiv:2605.13319, 2026
2026 arXiv
-
[11]
Communication-efficient collaborative LLM inference via distributed speculative decoding,
C. Zheng and T. Yang, “Communication-efficient collaborative LLM inference via distributed speculative decoding,”arXiv preprint arXiv:2509.04576, 2025
2025
-
[12]
Fast collaborative inference via distributed speculative decoding,
C. Zheng, K. Zhang, C. Sun, W. Zhang, Q. Liu, and A. A. Tesfay, “Fast collaborative inference via distributed speculative decoding,”arXiv preprint arXiv:2512.16273, 2025
2025
-
[13]
Speculation at a distance: Where edge-cloud speculative decoding actually pays off,
Y . Lyu, B. Irukulapati, and J. P. Champati, “Speculation at a distance: Where edge-cloud speculative decoding actually pays off,”arXiv preprint arXiv:2606.25091, 2026
2026 arXiv
-
[14]
LLMCad: Fast and scalable on-device large language model inference,
D. Xu, W. Yin, X. Jin, Y . Zhang, S. Wei, M. Xu, and X. Liu, “LLMCad: Fast and scalable on-device large language model inference,”arXiv preprint arXiv:2309.04255, 2023
2023 arXiv
-
[15]
PipeInfer: Accelerating LLM inference using asynchronous pipelined speculation,
B. Butler, S. Yu, A. Mazaheri, and A. Jannesari, “PipeInfer: Accelerating LLM inference using asynchronous pipelined speculation,” inSC24: International Conference for High Performance Computing, Networking, Storage and Analysis, 2024, pp. 1–19
2024
-
[16]
AMUSD: Asynchronous multi-device speculative decoding for LLM acceleration,
B. McDanel, “AMUSD: Asynchronous multi-device speculative decoding for LLM acceleration,”arXiv preprint arXiv:2410.17375, 2024
2024 arXiv
-
[17]
PEARL: Parallel speculative decoding with adaptive draft length,
T. Liu, Y . Li, Q. Lv, K. Liu, J. Zhu, W. Hu, and X. Sun, “PEARL: Parallel speculative decoding with adaptive draft length,”arXiv preprint arXiv:2408.11850, 2024
2024 arXiv
-
[18]
CoSine: Enhancing LLM serving via collaborative and decoupled speculative inference,
L. Gao, J. Liu, X. Zhang, G. Gao, and Y . Liao, “CoSine: Enhancing LLM serving via collaborative and decoupled speculative inference,” in IEEE INFOCOM 2026 - IEEE Conference on Computer Communications, 2026, pp. 1–10
2026
-
[19]
WISP: Waste- and interference-suppressed distributed speculative LLM serving at the edge via dynamic drafting and SLO-aware batching,
X. Li, J. Fan, Q. Wang, D. Spatharakis, S. Ghafouri, H. Vandierendonck, D. John, B. Ji, A. R. Butt, and D. S. Nikolopoulos, “WISP: Waste- and interference-suppressed distributed speculative LLM serving at the edge via dynamic drafting and SLO-aware batching,”arXiv preprint arX...
2026 arXiv
-
[20]
Efficient memory management for large language model serving with PagedAttention,
W. Kwon, Z. Li, S. Zhuang, Y . Sheng, L. Zheng, C. H. Yu, J. E. Gonzalez, H. Zhang, and I. Stoica, “Efficient memory management for large language model serving with PagedAttention,” inProceedings of the 29th Symposium on Operating Systems Principles, 2023, pp. 611–626
2023
-
[21]
DistServe: Disaggregating prefill and decoding for goodput-optimized large language model serving,
Y . Zhong, S. Liu, J. Chen, J. Hu, Y . Zhu, X. Liu, X. Jin, and H. Zhang, “DistServe: Disaggregating prefill and decoding for goodput-optimized large language model serving,” in18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24). USENIX Association, ...
2024
-
[22]
Hintjens,ZeroMQ: Messaging for Many Applications
P. Hintjens,ZeroMQ: Messaging for Many Applications. O’Reilly Media, 2013
2013
-
[23]
Qwen3 technical report,
Qwen Team, “Qwen3 technical report,”arXiv preprint arXiv:2505.09388, 2025
2025 arXiv
-
[24]
ChatGLM: A family of large language models from GLM- 130B to GLM-4 All Tools,
Team GLM, “ChatGLM: A family of large language models from GLM- 130B to GLM-4 All Tools,”arXiv preprint arXiv:2406.12793, 2024
2024 arXiv
-
[25]
GLM-4-0414 model series,
Z.ai, “GLM-4-0414 model series,” Official model repository, 2025. [Online]. Available: https://github.com/zai-org/GLM-4
2025
-
[26]
The Llama 3 herd of models,
A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Vaughan, A. Yanget al., “The Llama 3 herd of models,”arXiv preprint arXiv:2407.21783, 2024
2024 arXiv
-
[27]
Training verifiers to solve math word problems,
K. Cobbe, V . Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, C. Hesse, and J. Schul- man, “Training verifiers to solve math word problems,”arXiv preprint arXiv:2110.14168, 2021
-
[28]
Network emulation with NetEm,
S. Hemminger, “Network emulation with NetEm,” inProceedings of the 6th Australian National Linux Conference (linux.conf.au 2005), Canberra, Australia, Apr. 2005
2005
-
[29]
Understanding operational 5G: A first measurement study on its coverage, performance and energy consumption,
D. Xu, A. Zhou, X. Zhang, G. Wang, X. Liu, C. An, Y . Shi, L. Liu, and H. Ma, “Understanding operational 5G: A first measurement study on its coverage, performance and energy consumption,” inProceedings of the 2020 ACM SIGCOMM Conference. ACM, 2020, pp. 479–494. [Online]. Avai...
2020
-
[30]
AccuMO: Accuracy- centric multitask offloading in edge-assisted mobile augmented reality,
Z. J. Kong, Q. Xu, J. Meng, and Y . C. Hu, “AccuMO: Accuracy- centric multitask offloading in edge-assisted mobile augmented reality,” inProceedings of the 29th Annual International Conference on Mobile Computing and Networking, 2023, pp. 1–16
2023
-
[31]
A variegated look at 5G in the wild: Performance, power, and QoE implications,
A. Narayanan, X. Zhang, R. Zhu, A. Hassan, S. Jin, X. Zhu, X. Zhang, D. Rybkin, Z. Yang, Z. M. Mao, F. Qian, and Z.-L. Zhang, “A variegated look at 5G in the wild: Performance, power, and QoE implications,” in Proceedings of the 2021 ACM SIGCOMM Conference. ACM, 2021, pp. 610–...
2021
-
[32]
SpecInfer: Accelerating large language model serving with tree-based speculative inference and verification,
X. Miao, G. Oliaro, Z. Zhang, X. Cheng, Z. Wang, Z. Zhang, R. Y . Y . Wong, A. Zhu, L. Yang, X. Shi, C. Shi, Z. Chen, D. Arfeen, R. Abhyankar, and Z. Jia, “SpecInfer: Accelerating large language model serving with tree-based speculative inference and verification,” inProceedin...
2024
-
[33]
Medusa: Simple LLM inference acceleration framework with multiple decoding heads,
T. Cai, Y . Li, Z. Geng, H. Peng, J. D. Lee, D. Chen, and T. Dao, “Medusa: Simple LLM inference acceleration framework with multiple decoding heads,” inProceedings of the 41st International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 235...
2024
-
[34]
Sequoia: Scalable, robust, and hardware-aware speculative decoding,
Z. Chen, A. May, R. Svirschevski, Y . Huang, M. Ryabinin, Z. Jia, and B. Chen, “Sequoia: Scalable, robust, and hardware-aware speculative decoding,”arXiv preprint arXiv:2402.12374, 2024
2024 arXiv
-
[35]
Conformal sparsification for bandwidth-efficient edge-cloud speculative decoding,
P. Bhattacharjee, F. Tian, M. Zhong, G. Zhang, O. Simeone, and R. Tandon, “Conformal sparsification for bandwidth-efficient edge-cloud speculative decoding,”arXiv preprint arXiv:2510.09942, 2025
2025
-
[36]
Quantize- sample-and-verify: LLM acceleration via adaptive edge-cloud speculative decoding,
G. Zhang, Y . Cai, G. Yu, P. Popovski, and O. Simeone, “Quantize- sample-and-verify: LLM acceleration via adaptive edge-cloud speculative decoding,”arXiv preprint arXiv:2507.00605, 2025
2025
-
[37]
Taming throughput-latency tradeoff in LLM inference with Sarathi-Serve,
A. Agrawal, N. Kedia, A. Panwar, J. Mohan, N. Kwatra, B. S. Gulavani, A. Tumanov, and R. Ramjee, “Taming throughput-latency tradeoff in LLM inference with Sarathi-Serve,” in18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24). USENIX Association, 2024...
2024
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.