{"id":"83dc8ced-82f0-4670-b69b-34cafec5ec88","arxiv_id":"2608.00434","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"AdaMTP uses entropy-based segmentation to adaptively mask multi-token prediction losses, improving quality and speed over fixed-horizon multi-token prediction.","lead":"AdaMTP, a training method for models that predict several future tokens at once, adapts the prediction distance to the natural chunking of text by masking losses across semantic boundaries. It reports small but consistent gains in accuracy and inference speed over standard multi-token prediction on three popular language model backbones.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. 5 assigns boundary tokens the full length of the next group, so the mask in Eq. 7 retains exactly the cross-boundary predictions the paper claims to suppress.","rationale":"The manuscript's central claim rests on the assertion that AdaMTP's dynamically masked objective suppresses loss for predictions that cross semantic boundaries. That is not what the equations do. Equation 4-5 define groups by entropy surges and then, for the last token of each group, set the adaptive depth to the full length of the next group; Eq. 7 retains all losses within that depth. The token immediately before a detected boundary is therefore given maximum cross-boundary supervision, including the offset-1 prediction that is the clearest case of crossing. This is not a question of whether entropy identifies boundaries; even granting the segmentation, the masking rule is self-contradictory. The reader's weakest assumption (entropy pattern reliability) is a reasonable but second-order concern. The concrete test — mask inspection plus a boundary-token ablation — would determine whether the experiments implemented Eq. 5 as written or some corrected rule. If Eq. 5 is literal, the paper's explanation of why AdaMTP helps is unsupported; if it is a typo, the experiments need to be redone with the intended rule. Either way, conditional acceptance with this check is appropriate.","tokens_in":13439,"tokens_out":7751,"duration_ms":72527,"concrete_test":"Inspect the actual masks produced by Eq. 5 on a sample of the training corpus: for every detected boundary at position e_k, verify whether token e_k−1 has active auxiliary losses for offsets 1..min(|G_{k+1}|−1, n−1), and report the fraction of all retained MTP losses that cross detected boundaries. Then re-run the Llama-3.1-8B GSM8K experiment with two variants: (A) Eq. 5 exactly as written, and (B) boundary tokens assigned d_t = 1 so no auxiliary loss crosses a boundary. If (A) ≈ (B), the gains are not from cross-boundary masking; if (B) > (A), the stated mechanism is supported and Eq. 5 is a typo; if (A) > (B), the paper contradicts its own motivation. Either outcome resolves whether the central mechanistic claim is valid.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The core mechanism is internally inconsistent. In Section 3.1, segmentation splits at indices where ΔE_t > τ, so the semantic boundary lies between e_k−1 and e_k. Equation 5 then sets d_t = |G_{k+1}| for t = e_k−1, the token described as 'preceding a high-entropy transition,' and Eq. 7 keeps every auxiliary loss with offset j+1 ≤ d_t. Consequently, for each boundary token, all auxiliary predictions into the next group are retained, including the offset-1 prediction that crosses the detected boundary. Since τ is calibrated so that the average group size equals n, roughly one token per group (about 25% at n=4) receives no cross-boundary masking at all. The stated mechanism — masking out predictions that cross boundaries — is therefore not what Eqs. 5–7 implement. Either the experiments used an unstated boundary-token rule, or the reported gains cannot be attributed to suppressing cross-boundary gradients. This concern is more specific than the reliability of entropy-based segmentation: even if entropy boundaries are perfectly detected, the masking rule contradicts the paper's own motivation.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes AdaMTP, an adaptive multi-token prediction training scheme for large language models. It uses token-level predictive entropy from the frozen base model to segment training sequences into variable-length groups, assigns each token an adaptive prediction depth (Eq. 5), and applies a dynamically masked MTP loss (Eq. 7) that is intended to suppress cross-boundary predictions. The authors evaluate on three backbones (Llama-3.1-8B, Qwen-2.5-7B, Gemma-3-12B) across eight benchmarks, comparing AdaMTP with standard NTP and fixed-horizon MTP in both task accuracy and inference speedup. A secondary adaptive-horizon decoding mode is also discussed.","tokens_in":13690,"tokens_out":11298,"duration_ms":91243,"significance":"If the central claim held, AdaMTP would be a low-cost training-time modification with consistent quality and speed gains over fixed-horizon MTP, and the paper's scope is appropriate: three backbones, eight benchmarks, and both performance and efficiency measures. The problem is well motivated, and the self-speculative decoding framing makes the efficiency comparison meaningful. However, the current evidence is not yet convincing: the reported effect sizes are small and unreplicated, and one of the core equations contradicts the paper's stated masking mechanism. The paper does not provide code, proofs, or a parameter-free derivation; its contribution is empirical, so the reliability of the experiments is load-bearing.","major_comments":[{"comment":"The implemented mask does not suppress cross-boundary predictions for boundary tokens. For t = e_k - 1, the token immediately preceding a detected split, Eq. (5) sets d_t = |G_{k+1}|, so Eq. (7) retains every auxiliary loss with j+1 <= |G_{k+1}|; all of these targets lie in the next group, across the detected boundary. Since tau is calibrated so that the average group size equals n, a nontrivial fraction of tokens (about 1/n, i.e., 25% at n=4) receives no masking on its auxiliary losses at all. This contradicts the Abstract's and Section 3.2's claim that AdaMTP 'suppresses the loss for predictions that cross these boundaries.' If the experiments used a different boundary-token rule, it must be stated; as written, the reported gains cannot be attributed to suppressing cross-boundary gradients.","section":"Section 3.1, Eq. (5); Section 3.2, Eq. (7)"},{"comment":"The performance differences supporting 'consistently outperforms' are small and are reported without error bars, multiple seeds, or significance tests. For example, on Llama-3.1-8B the HumanEval+ scores are identical for MTP and AdaMTP (35.98), and on Gemma-3-12B the MBPP+ scores are identical (54.50); several other cells differ by less than one point. The average gains over MTP are 0.95, 1.59, and 1.13 points on the three backbones. Without estimates of variance or paired tests across seeds, these results are compatible with training noise, so the claim of consistent superiority is not yet established.","section":"Section 4.2, Table 1"},{"comment":"The entropy-based segmentation is never ablated against a non-entropy control. The threshold tau is calibrated so that the average group size equals n, and Figure 1 illustrates the entropy pattern on a single example, but the paper does not compare against random segmentation or fixed blocks of size n under otherwise identical training. Such an ablation is necessary to attribute the gains to the entropy signal rather than to any variable-depth masking. The calibration procedure for tau, including the search range and the resulting values, is also not reported.","section":"Section 3.1; Section 4.4"},{"comment":"The adaptive-horizon decoding mode is claimed to yield 'clear throughput gains as batch size increases,' but no end-to-end throughput or latency measurement under large-batch, compute-bound serving is reported; only the average number of verified tokens per step is shown. Moreover, the statement that the adaptive strategy keeps accuracy 'statistically indistinguishable' from the fixed-horizon baseline appears without any reported statistical test. Please either provide the missing measurements or soften the claims to match the evidence.","section":"Section 4.5"}],"minor_comments":[{"comment":"The indexing of E_t is ambiguous: the text says 'entropy of the next-token distribution at each position t,' but the notation E_t = H(P(·|x_<t)) is the entropy for the token at position t. Please align the notation with the segmentation indices in Eq. (5).","section":"Eq. (4)"},{"comment":"The calibration of tau is described only as a dataset-level search; please report the search space, the criterion, and the resulting tau values for each backbone and dataset.","section":"Section 3.1"},{"comment":"The claimed 'approximately monotonically decreasing' entropy pattern is illustrated on one sentence; please quantify this pattern on the training corpus or soften the generalization.","section":"Figure 1"},{"comment":"Speedup ratios are reported as single numbers without measurement variability or a description of the evaluation protocol (number of prompts, generation length, batch size, hardware). Please add these details so the ratios are reproducible.","section":"Table 2"},{"comment":"The phrase 'consistently outperforms' should be qualified where individual benchmark cells are ties, such as HumanEval+ on Llama-3.1-8B and MBPP+ on Gemma-3-12B in Table 1.","section":"Abstract; Section 4.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is clearly written and addresses a topical problem, but the discrepancy between the stated masking mechanism and Eq. (5) is a central issue that requires a correction and additional experiments. I recommend major revision rather than rejection because the idea is defensible and the inconsistency is fixable in principle."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper proposes AdaMTP, a variant of multi-token prediction where the prediction horizon is set per token based on entropy of a frozen base model. Tokens are grouped by entropy surges, and the auxiliary loss is masked beyond a per-token depth. The idea is plausible – language is not uniformly predictable – and the paper has real empirical content: three backbones (Llama-3.1-8B, Qwen-2.5-7B, Gemma-3-12B), a standard benchmark suite, and consistent, if modest, gains over fixed-horizon MTP. The speedups also look consistent. I'd say the contribution is a useful but within-subfield improvement.\n\nThe main problem is internal. Equation 5 gives the last token of each group (the token preceding the entropy surge) a depth equal to the entire next group. So the mask in Eq. 7 keeps all of that token's auxiliary losses, including the offset-1 prediction that crosses the detected boundary. The paper's stated mechanism – suppressing cross-boundary losses – is not what the equations implement. What they actually do is reduce the horizon for tokens that are near the end of a group, while the boundary token gets the full horizon. So the claimed interpretation of the gains is unsupported. This is not a nit; it changes the story. If the mask truly removes cross-boundary predictions, they need to change the boundary-token rule (e.g., set the depth of the first token of the next group to its length, or mask offset 1 for the boundary token). If the gains come from the overall reduction in MTP supervision, they should say that and ablate against random group sizes.\n\nThe experiments also lack error bars, multiple seeds, and statistical tests. The gains over MTP are sometimes small (e.g., a few points on GSM8K, and on Qwen the overall numbers are all below the base model). I'd like to see a random-mask or fixed-shorter-horizon ablation to confirm the entropy segmentation itself is what helps. The large-batch throughput claim is not measured anywhere, only argued.\n\nOverall: the empirical comparison is a fair start, but the paper as written cannot support its causal story. With a corrected formula or a revised interpretation it could be a solid contribution. It deserves peer review, not a desk reject, because the idea is new and the experiments are head-to-head against MTP. I'd send it to a referee with a clear note to check Eq. 5 against the stated contribution.","headline":"A promising adaptive MTP idea with consistent empirical gains, but the masking formula in Eq. 5 contradicts the paper's own boundary-suppression story.","tokens_in":14245,"tokens_out":6784,"would_cite":false,"duration_ms":57388,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"AdaMTP claims that aligning multi-token prediction depth with token-level predictive entropy prevents noisy cross-boundary gradients and beats standard MTP in quality and speed.","keywords":["multi-token prediction","adaptive prediction depth","entropy-based segmentation","semantic boundaries","self-speculative decoding","representation interference","LLM fine-tuning"],"falsifier":"Compute per-token entropy deltas $\\Delta E_t$ on the training corpus and check whether they are systematically larger at clause and sentence boundaries than at mid-chunk positions; if they are not, the segmentation is arbitrary. A more direct check is to retrain AdaMTP with the same masking rate but boundaries placed at random positions: if random masking matches entropy-based masking on GSM8K and HumanEval, the entropy-surge hypothesis is not what carries the gain.","tokens_in":13259,"feed_emoji":"⚡","tokens_out":13196,"duration_ms":101564,"temperature":0.7,"pith_summary":"The paper argues that multi-token prediction (MTP) — training an LLM to forecast several future tokens at once through auxiliary heads — is undermined by a fixed prediction horizon. Natural language and code are not uniformly predictable: inside a coherent phrase or code block, uncertainty falls steadily, while it spikes at semantic boundaries. Forcing every head to predict across those spikes, the paper claims, sends noisy gradients back through the shared backbone and degrades the base model. AdaMTP detects boundaries from the base model's token-level predictive entropy, assigns each token an adaptive prediction depth, and masks any MTP loss that would cross a boundary. On Llama-3.1-8B, Qwen-2.5-7B, and Gemma-3-12B across math, code, and general benchmarks, it reports higher average scores than both next-token prediction and standard MTP, while decoding faster than standard MTP.","feed_headline":"Adaptive multi-token prediction beats fixed-horizon MTP on three LLMs","feed_subtitle":"Masking multi-token supervision across unpredictable boundaries lifts accuracy and decoding speed on 3 LLM families.","key_machinery":"The load-bearing mechanism is the entropy-based segmentation coupled with a dynamically masked MTP loss. Predictive entropy $E_t$ is computed from the unmodified base model so the uncertainty signal matches the model being trained; a calibrated threshold $\\tau$ turns entropy surges into group boundaries, with average group size tied to the number of heads $n$. Adaptive depth $d_t$ is then defined per token: tokens inside a group predict only up to the group's end, while the boundary token predicts the whole next group. The indicator mask in the MTP objective drops every term with offset $j+1 > d_t$ from the LoRA fine-tuning loss, so the model is never asked to jump a detected semantic boundary.","core_discovery":"At the center is a diagnosis and a fix. The diagnosis: standard MTP's auxiliary heads share the backbone's hidden representations, so a fixed horizon forces each head to predict across high-entropy transitions; the resulting cross-boundary gradients interfere with the main language-modeling head, which is why standard MTP underperforms NTP on average in the paper's results. The fix is an entropy-based segmentation: using the frozen base model, compute $E_t = -\\sum_{v \\in \\mathcal{V}} P(v|x_{<t}) \\log P(v|x_{<t})$ and split the sequence where $\\Delta E_t > \\tau$; then give token $x_t$ an adaptive depth $d_t$ equal to the distance to the end of its group (or the next group's length, for the boundary token), and train with the masked objective $L_{\\mathrm{MTP}} = \\sum_{j=1}^{n-1}\\sum_t \\mathbb{I}(j+1 \\le d_t)\\,\\mathrm{L_{CE}}(\\mathrm{Head}_j(h_t), x_{t+j+1})$. The paper claims this suppresses noisy supervision, preserves core abilities, and makes the MTP heads' drafts more acceptable to the verifier, which is why AdaMTP reports higher task scores and higher self-speculative speedups than standard MTP.","pith_inferences":["We infer that the entropy-surge principle is not specific to MTP: any auxiliary objective that shares backbone representations should avoid supervision across entropy surges, a hypothesis testable by applying the same mask to other multi-future-token losses.","We infer that the entropy signal could serve as a general segmentation tool outside training, for example finding discourse boundaries or guiding retrieval or cache decisions; the paper does not explore these uses.","We infer that computing depths once with the frozen base model keeps AdaMTP cheap, but online depth updates during fine-tuning might capture distribution shift; the paper does not test that variant.","We infer that a natural next test is applying the same adaptive masking during full pretraining rather than SFT retrofit, to see whether the representation-interference benefit scales beyond the fine-tuning regime."],"forward_implications":["If the central claim is correct, the fixed horizon itself—not the multi-token objective—is the source of MTP's degradation: the paper reports standard MTP average scores below NTP on all three backbones, while AdaMTP sits above NTP.","Deeper MTP becomes usable: the head-count study on Llama-3.1-8B with GSM8K shows standard MTP accuracy falling from 11.60 at $n=2$ to 9.68 at $n=6$, while AdaMTP stays above the NTP reference and peaks at $n=4$.","The speedup over standard MTP is a training effect rather than a decoding change: both use the same fixed-horizon self-speculative procedure, so AdaMTP's faster decoding must come from drafts that the verifier accepts more often.","Adaptive-horizon decoding should cut verification cost under batched serving: the paper reports fewer candidate tokens verified per step with statistically indistinguishable accuracy, with the benefit growing as inference becomes compute-bound."],"supporting_citations":[{"why":"Supplies the two-stage retrofit recipe (auxiliary-head warm-up then joint LoRA fine-tuning) and the L-MTP baseline AdaMTP is compared against.","marker":"Liu et al., 2025"},{"why":"Medusa provides the multi-head MTP retrofit and the tree-based self-speculative verification that AdaMTP uses at inference.","marker":"Cai et al., 2024"},{"why":"Establishes the multi-token prediction objective with parallel heads that AdaMTP modifies with adaptive masking.","marker":"Gloeckle et al., 2024"},{"why":"Defines speculative decoding, the lossless draft-then-verify framework that makes the reported speedups meaningful.","marker":"Leviathan et al., 2023"},{"why":"GSM8K is the mathematical-reasoning benchmark where AdaMTP shows its largest speedups and head-count tolerance.","marker":"Cobbe et al., 2021"},{"why":"HumanEval is the code-generation benchmark used for pass@1 evaluation and one of the three speedup testbeds.","marker":"Chen et al., 2021"}],"fun_headline_variants":["Adaptive MTP: entropy-guided masking boosts LLM speed and accuracy","New MTP variant adapts prediction depth, outperforms on 3 LLMs","Entropy-based masking tames multi-token prediction noise","AdaMTP: smarter multi-token training for faster, better LLMs","Adaptive horizon tuning improves multi-token prediction across LLMs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the base model's next-token entropy has a reliable signature—it falls roughly monotonically inside a coherent chunk and jumps at chunk boundaries—and that this signature still holds while the model is being fine-tuned with LoRA.","fun_headline_variants_meta":{"raw":{"variants":["Adaptive MTP: entropy-guided masking boosts LLM speed and accuracy","New MTP variant adapts prediction depth, outperforms on 3 LLMs","Entropy-based masking tames multi-token prediction noise","AdaMTP: smarter multi-token training for faster, better LLMs","Adaptive horizon tuning improves multi-token prediction across LLMs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000718,"raw_usage":{"total_tokens":3288,"prompt_tokens":1069,"completion_tokens":2219,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":685,"completion_tokens_details":{"reasoning_tokens":2125}},"tokens_in":685,"tokens_out":2219,"duration_ms":14489,"temperature":1.0,"reasoning_tokens":2125,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T15:19:10.886698+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute per-token entropy deltas $\\Delta E_t$ on the training corpus and check whether they are systematically larger at clause and sentence boundaries than at mid-chunk positions; if they are not, the segmentation is arbitrary. A more direct check is to retrain AdaMTP with the same masking rate but boundaries placed at random positions: if random masking matches entropy-based masking on GSM8K and HumanEval, the entropy-surge hypothesis is not what carries the gain.","supporting_citations":[],"review_version":2}