{"id":"771c5188-4273-4445-a59f-af17b8885dec","arxiv_id":"2608.02989","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"AcceptMoE is a verifier-side expert selector that uses commitment-weighted router demand and an entropy-based set size, reducing expert traffic while keeping mean accuracy within 0.27 percentage points of EAGLE-3 with natural routing.","lead":"AcceptMoE speeds up text generation from mixture-of-experts language models by letting the verifier pick a small, self-sizing set of experts for each speculative decoding block. On three large models and four benchmarks it keeps accuracy nearly unchanged while increasing throughput by up to 2.06 times when expert weights are offloaded.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Fixed offline commitment probabilities may be miscalibrated under AcceptMoE's own masked routing, so the demand ranking and self-sized expert sets in Eqs. 3-6 may not transfer to deployment.","rationale":"The reader's weakest assumption was that offline bp_d and beta generalize to evaluation prompts and draft-tree geometry. I agree this is the central vulnerability, but I would sharpen it: even with perfect generalization across prompts, the estimates may be internally inconsistent because AcceptMoE's logit mask changes the served distribution and hence the very commitment probabilities being estimated. The paper does not disclose whether the traces come from natural or masked routing, and gives no recalibration or beta-sensitivity evidence. The rest of the paper is credible: the fixed-budget comparison is controlled via B0, the accuracy and throughput measurements are reported per pair, and the mechanism (smaller eligible expert unions reducing traffic) is plausible and consistent with the reported cache-hit improvements. Those strengths do not remove the need to verify that the fixed calibration is valid under the deployed distribution. A single re-estimation experiment on a held-out split would settle the question; until then, the conditional verdict is appropriate.","tokens_in":13199,"tokens_out":11595,"duration_ms":129053,"concrete_test":"On a held-out split, run AcceptMoE while recomputing bp_d and the selected n_er from AcceptMoE's own masked-routing traces (online or from a disjoint trace set), and sweep beta in {0.25, 0.5, 0.75, 1.0}. Compare the resulting expert-set sizes, Table 1 accuracy gaps vs Standard SD, and H2D traffic against the paper's fixed offline estimates. If n_er shifts by more than about 10% on average, or the mean accuracy gap moves by more than about 0.5 percentage points, the fixed offline calibration is load-bearing and the central transferability claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The method's core mechanism is commitment-weighted demand: Eq. 3 weights each token by bp_d^beta, with bp_d = c_d/m_d estimated from offline verification traces, and Eq. 6 derives the expert-set size from the entropy of the resulting demand. For this to be correct, bp_d must approximate the probability that a node at draft position d is committed during the actual served verification pass. But AcceptMoE changes the served distribution: the logit mask in Eq. 2 restricts routing to S, altering which tokens are accepted and therefore altering the marginal commitment probabilities. The paper states only that bp_d are 'estimated from disjoint training traces and held fixed during evaluation'; it does not state whether those traces were collected under natural routing or under AcceptMoE's masked routing, nor does it report any recalibration or sensitivity analysis for beta (default 0.5). If the traces were natural-routing traces, the weights are stale even on the same prompts and draft geometry, because the mask itself shifts the acceptance probabilities. If bp_d are wrong, both the expert ranking (Eq. 4) and the self-sized cardinality n_er (Eq. 6) are wrong, which directly threatens the claimed accuracy/traffic trade-off. This is load-bearing: removing the offline estimates removes the 'commitment-weighted' and 'self-sizing' contributions, leaving an ad hoc fixed-budget selector. The paper provides no evidence that the estimates remain valid under the distribution the method itself induces.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces AcceptMoE, a verifier-side expert-selection method for speculative decoding with mixture-of-experts targets. For each verification block and MoE layer, AcceptMoE weights target-router scores by offline-estimated per-position commitment probabilities, anchors the eligible set to the root token's natural top-k experts, sizes the non-anchor part via the effective rank (exponential of entropy) of the demand distribution, and, under expert offloading, prunes nonresident experts using a rerouting budget and a minimum-set-size constraint. The paper evaluates on 12 model-task pairs spanning Qwen3-30B-A3B, Qwen3-Coder-30B-A3B, and GPT-OSS-120B on GSM8K, MATH500, HumanEval, and MBPP. It reports mean accuracy 0.27 percentage points below EAGLE-3 speculative decoding with natural routing, 1.290x resident-memory throughput and 2.06x offloaded throughput relative to Standard SD, and 73.6-77.1% reductions in host-to-device expert-weight traffic.","tokens_in":13547,"tokens_out":7029,"duration_ms":77522,"significance":"If the reported results hold, the paper makes a useful contribution by separating activated-expert union size from token count in MoE speculative decoding, proposing a per-block self-sizing expert set that avoids a user-supplied expert budget, and making eligibility depend on cache residency rather than on learned prefetching. The paper is explicitly framed as an approximation, it evaluates on a broad set of model-task pairs, and it includes a matched-budget accuracy comparison against MoE-Spec and Count-B as well as an ablation of residency-aware pruning. The main risks are empirical rather than mathematical: the offline commitment probabilities may be miscalibrated under the method's own masked routing, and the end-to-end throughput claims are made only against Standard SD rather than against the closest verifier-side baselines. These issues are addressable with additional experiments and analysis.","major_comments":[{"comment":"The commitment probabilities bp_d = c_d/m_d are estimated from offline traces and held fixed, but the paper does not state whether those traces were collected under natural routing or under AcceptMoE's masked routing. Since Eq. (2) changes which tokens are accepted and thus changes the marginal commitment probabilities, traces collected under natural routing would be stale under the served distribution. If bp_d are miscalibrated, both the demand ranking in Eq. (4) and the self-sized cardinality n_er in Eq. (6) are wrong, which directly threatens the claimed accuracy/traffic trade-off. Please report the trace-collection policy and add a calibration or sensitivity analysis, for example recomputing bp_d under AcceptMoE's own masked routing and varying beta around the default of 0.5.","section":"Method, Eqs. (3)-(6); Experiments"},{"comment":"The end-to-end throughput and offload measurements compare AcceptMoE only with Vanilla AR and Standard SD. The closest verifier-side expert-selection methods (MoE-Spec and, under offloading, SP-MoE or MoE-SpeQ) appear only in the accuracy comparison or in related work and are not measured end to end. Because those methods also reduce the activated-expert union or prefetch nonresident weights, the reported 1.290x and 2.06x speedups over Standard SD do not establish an advantage over the closest prior art. Please add at least MoE-Spec at the matched budget B0 in the resident-throughput experiment, and ideally an offloading baseline, or explicitly scope the claim as being relative only to natural routing.","section":"Experiments, End-to-End Throughput (Figs. 3-4)"},{"comment":"The central accuracy claim is a mean difference of 0.27 percentage points below Standard SD, with per-pair differences as large as 1.22 points, yet Table 1 reports no confidence intervals or significance tests, and the throughput ratios are based on three repetitions without reported variance. Given that the self-sizing design is justified partly by avoiding a budget sweep, it is important to show that the 0.27pp gap is stable, for example with bootstrap intervals over prompts or repeated runs. Please add uncertainty estimates to Table 1 and to the reported speedup ratios.","section":"Experiments, Table 1"}],"minor_comments":[{"comment":"The budget B0 is computed from the first 50 evaluation prompts for each pair; although the text correctly notes that no correctness outcomes are used, the fixed-budget baselines are still given a size derived from evaluation prompts. Please clarify whether this could favor the matched-budget comparison and consider deriving B0 from training-split traces.","section":"Experiments, Table 1"},{"comment":"The choices beta=0.5 and the exponent -1/2 in the N_d discount are presented as defaults without sensitivity analysis or a supporting reference; a brief robustness discussion or citation would help readers judge how load-bearing these choices are.","section":"Method, Eq. (3)"},{"comment":"The sentence stating that the logit mask 'preserves the target's native post-top-k weighting' is ambiguous, because clamping logits changes the softmax distribution; please specify whether probabilities are renormalized over the eligible set S or some other weighting is applied.","section":"Method, Eq. (2)"},{"comment":"The sentence 'For GPT-OSS on GSM8K and MATH500, we extract and score the final Harmonychannel' appears truncated; please clarify the extraction and scoring procedure.","section":"Experiments"},{"comment":"Given the close relationship to XShare, the current verbal comparison is thin; a small quantitative comparison in the experiments would strengthen the positioning.","section":"Related Work"}],"recommendation":"major_revision","confidential_remarks":"I see no indication of citation manipulation or novelty problems; the paper is within scope for cs.LG. The main concerns are the missing calibration evidence for the offline commitment probabilities and the absence of end-to-end comparisons to the closest verifier-side baselines, both of which are addressable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is one of the more honest and carefully measured systems papers I have seen in this area. The core idea — compute commitment-weighted router demand per verification block, self-size the eligible expert set via effective rank, and under offloading condition eligibility on cache residency — is genuinely new relative to MoE-Spec's fixed budget and the natural-route prefetchers (SP-MoE, MoE-SpeQ). The evaluation is broad (12 model–task pairs, three targets, four benchmarks), the throughput protocol is unusually careful (fresh servers, rotated order, three repetitions), and the authors are upfront that their selector is an approximation that changes the target distribution. Credit where due: the matched-budget comparison against MoE-Spec shows the weighting alone is worth 2.45 pp, and the self-sizing ablation against swept fixed budgets is a reasonable argument for dropping the hyperparameter.\n\nNow the soft spots. The stress-test about offline commitment probabilities is legitimate and load-bearing in this specific sense: bp_d = c_d/m_d is estimated from traces, but AcceptMoE's logit mask changes what gets accepted, so if those traces are natural-routing traces, the weights are stale even on the same prompts and draft tree. The paper does not say which traces were used, and there is no sensitivity analysis for beta = 0.5. This does not sink the empirical claims, because everything is measured on held-out prompts, but it means the 'self-sizing' mechanism could be tuned to the wrong distribution in deployment. A one-paragraph calibration check and a beta sweep would address it.\n\nThe bigger omission for a systems paper is the end-to-end offloading comparison, which only includes Standard SD and Vanilla AR. SP-MoE and MoE-SpeQ are cited in related work but not run, so we have no idea whether AcceptMoE beats the actual prior offloading systems it is positioned against. The 2.06x over Standard SD is nice, but it does not establish advantage over prefetching. Also, the accuracy table has no confidence intervals; with 12 pairs and small per-task sample sizes, that is a minor but easy fix. And Eq. 6 is a heuristic, not a derivation — the ablation partially compensates, but the paper overstates the word 'derive'.\n\nBottom line: the paper deserves a serious referee. The fixes I would require: report trace provenance and beta sensitivity, add at least one prefetching baseline under offload, and add error bars. The central contribution — verifier-side, budget-free, cache-aware expert eligibility — looks real and worth citing.","headline":"A well-measured systems paper that makes expert eligibility budget-free and cache-aware; the calibration and missing-baseline issues are real but fixable.","tokens_in":14037,"tokens_out":3189,"would_cite":true,"duration_ms":33060,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A mixture-of-experts language model can be sped up during speculative decoding by choosing a small, cache-aware set of eligible experts per verification block, with almost no accuracy loss.","keywords":["speculative decoding","mixture-of-experts","expert selection","self-sizing expert set","expert offloading","commitment-weighted demand","effective rank","LLM inference acceleration"],"falsifier":"Re-run the full 12 model-task evaluation with commitment probabilities estimated from the evaluation prompts themselves, or from a second held-out split, and compare accuracy and throughput; if the gap to natural-routing speculative decoding grows well beyond the reported 0.27 percentage points, or if the self-selected expert count changes sharply, the fixed offline estimates are the fragile piece.","tokens_in":13023,"feed_emoji":"⚡","tokens_out":14596,"duration_ms":124786,"temperature":0.7,"pith_summary":"AcceptMoE claims that when a mixture-of-experts (MoE) language model verifies a speculative draft tree, the dominant cost is not the number of tokens verified but the union of experts those tokens activate, and that this union can be reduced at verification time without asking the user for an expert budget. It builds one eligible expert set per layer and verification block by weighting each draft position's router scores by the offline-estimated probability that a node at that position is committed to the output, always preserving the root token's natural top-k experts. The number of additional experts is set by the effective rank of the weighted demand, so concentrated demand yields a small set and diffuse demand a larger one. Under expert offloading, the same demand ranking prunes nonresident experts, making eligibility follow cache residency instead of predicted natural routes. The paper reports 1.290x throughput over natural-routing speculative decoding with all experts resident, 2.06x under offloading, a 73.6 to 77.1 percent cut in expert-weight traffic, and a mean accuracy change of -0.27 percentage points across 12 model-task pairs.","feed_headline":"Self-sizing expert sets speed MoE decoding 2.06x under offload","feed_subtitle":"Cache-aware expert selection cuts weight traffic up to 77% at a 0.27-point accuracy cost.","key_machinery":"The carrying object is the commitment-weighted demand utility $u_e = \\sum_{t=1}^{T} \\alpha_t r_{t,e} \\mathbf{1}[e \\in K_t]$, where $r_{t,e}$ is the target router's natural top-k probability for expert $e$ at token $t$ and $K_t$ is that token's natural top-k set. This single utility ranks which non-anchor experts enter the eligible set, its entropy-derived effective rank decides how many enter, and, after comparison with the resident cache, it chooses which nonresident experts are pruned. The root anchor $A$, the union of natural top-k sets over the committed root tokens, guarantees that the one token certain to be committed keeps its natural experts.","core_discovery":"The central claim is that expert eligibility during MoE speculative verification can be treated as a per-block, per-layer, cache-aware decision rather than a fixed global budget. AcceptMoE defines a commitment-weighted utility for each expert by summing target-router top-k probabilities over the verification block, with each token weighted by $\\alpha_t = \\hat{p}_{d_t}^{\\beta} N_{d_t}^{-1/2}$, where $\\hat{p}_{d_t}$ is the offline-estimated marginal probability that a node at draft position $d_t$ is committed to the output, $\\beta$ controls sharpness, and $N_{d_t}$ discounts wide tree levels sublinearly. Experts are ranked by this utility, the root anchor's natural top-k experts are always kept, and the number of non-anchor experts is $n_{\\mathrm{er}} = \\lceil \\exp(-\\sum_e q_e \\log q_e) \\rceil$, the effective rank of the residual demand distribution. Under offloading, the paper additionally prunes the longest prefix of nonresident experts that fits within the rerouting budget and the minimum set size, so expert-set cardinality and membership both adapt to the current block and cache state. The paper argues that this removes the need for a pre-specified expert budget and for an expert-predictor prefetching stage, and that the measured throughput gains, traffic reductions, and small accuracy change follow from that mechanism.","pith_inferences":["A natural extension the paper does not explore is to update $\\hat{p}_d$ online from the running acceptance stream, which would let the method adapt when the serving prompt distribution drifts.","The effective-rank rule is one particular self-sizing policy; replacing entropy with a cost that weights per-expert transfer byte size could improve the offloaded regime further, since the paper's own traffic numbers show byte costs differ by data type.","The root-anchor principle generalizes beyond this paper: any constrained verifier should protect coverage of the certainly-committed token first and spend all remaining capacity on the highest expected-demand experts, which suggests a direct design rule for batched MoE verification and diffusion decoding."],"forward_implications":["At a matched expert budget, commitment-weighted membership is worth 2.45 percentage points of mean accuracy over MoE-Spec's uniform router-mass aggregation, so the selection rule, not just the set size, determines quality.","Self-sizing makes deployment easier: across five budget-swept model-task pairs, the automatically chosen set lands 0.97 percentage points below the best measured fixed budget on average, and at most 1.83 points below it, removing the need for a per-workload budget sweep.","Under expert offloading, conditioning eligibility on cache residency cuts expert-weight traffic by 38.6 to 48.6 percent and raises cache hit rate by 6.9 to 8.2 percentage points relative to the same selector with pruning disabled.","End to end, the method reaches 1.290x the throughput of natural-routing speculative decoding with all experts resident and 2.06x under physical expert offloading, while keeping mean accuracy 0.27 percentage points lower."],"supporting_citations":[{"why":"It supplies the EAGLE-3 draft model and draft-tree geometry used by every method in the evaluation, defining the verification-block structure AcceptMoE operates on.","marker":"Li et al. 2025"},{"why":"It is the fixed-budget verifier-side selector whose uniform router-mass aggregation AcceptMoE replaces; matched-budget accuracy comparisons are made against it.","marker":"McDanel et al. 2026"},{"why":"It represents the prior offloading approach that predicts natural routes and prefetches weights, the contrast for AcceptMoE's residency-aware eligibility.","marker":"Chen et al. 2025b"},{"why":"It provides the motivating measurement that draft-side token reduction does not proportionally shrink the activated-expert union.","marker":"Pan et al. 2026"},{"why":"It defines the effective-rank functional that AcceptMoE uses to turn the demand distribution into a self-sized expert count.","marker":"Roy and Vetterli 2007"}],"fun_headline_variants":["AcceptMoE: Self-sizing expert sets for 2.06x MoE decoding","Cache-aware expert selection slashes traffic 77%, boosts MoE decoding","Self-sizing expert sets: 2.06x MoE speedup, 77% less traffic","AcceptMoE: Adaptive expert sets cut weight traffic 77%, speed 2.06x","No more expert budgets: AcceptMoE self-sizes for 2.06x speed"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The offline-estimated commitment probabilities and the fixed weighting exponent are computed once from training-split traces and are assumed to stay calibrated for the evaluation prompts and for the specific EAGLE-3 draft-tree geometry used at serve time; if they do not, both the expert ranking and the self-sized set size shift and the reported accuracy-throughput trade-off would not transfer.","fun_headline_variants_meta":{"raw":{"variants":["AcceptMoE: Self-sizing expert sets for 2.06x MoE decoding","Cache-aware expert selection slashes traffic 77%, boosts MoE decoding","Self-sizing expert sets: 2.06x MoE speedup, 77% less traffic","AcceptMoE: Adaptive expert sets cut weight traffic 77%, speed 2.06x","No more expert budgets: AcceptMoE self-sizes for 2.06x speed"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000445,"raw_usage":{"total_tokens":2333,"prompt_tokens":1114,"completion_tokens":1219,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":730,"completion_tokens_details":{"reasoning_tokens":1105}},"tokens_in":730,"tokens_out":1219,"duration_ms":8486,"temperature":1.0,"reasoning_tokens":1105,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T04:21:15.657512+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the full 12 model-task evaluation with commitment probabilities estimated from the evaluation prompts themselves, or from a second held-out split, and compare accuracy and throughput; if the gap to natural-routing speculative decoding grows well beyond the reported 0.27 percentage points, or if the self-selected expert count changes sharply, the fixed offline estimates are the fragile piece.","supporting_citations":[],"review_version":1}