{"id":"7678274a-1b9f-4453-ae6f-18a8978cb37d","arxiv_id":"2601.23278","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"FOCUS evicts non-decodable tokens in diffusion LLMs using an attention-delta predictor, delivering up to 3.52× throughput without quality loss.","lead":"FOCUS is an inference system that skips likely non-decodable tokens during diffusion-LLM block generation, cutting wasted compute and raising throughput. It reports up to 3.52× higher tokens/s than LMDeploy while holding generation quality, using early-layer attention as a decodability predictor.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Throughput speedup may be inflated by the admitted 'repetition termination' in the benchmark (Appendix F.1); its impact is unquantified, yet directly supports the headline 3.52× claim.","rationale":"The reader's CONDITIONAL verdict already flags the throughput-evaluation assumption, and this stress-test identifies that as the single most load-bearing threat to the paper's headline number. The paper is honest enough to disclose the repetition-termination mechanism in the appendix, but it does not even estimate its impact. The central claim is quantitative; an uncalibrated measurement protocol is a direct threat to that claim.\n\nWhy this over the predictor-generalization concern? The predictor's robustness is partially supported by multiple benchmarks and three models; the throughput claim has only one evaluation setup, and the admitted modification could affect it in either direction. The Gaussian assumption in Appendix D is acknowledged and is not the basis of the empirical results. The quality-preservation claim is less exposed because FOCUS's quality results, while lacking error bars, are consistent across configurations.\n\nThe concrete test is feasible without new hardware: rerunning the benchmark with termination disabled and reporting termination statistics would settle whether the 2.32×/3.52× numbers are artifacts. If the speedup persists and the systems have similar termination frequencies, the concern vanishes. If instead the speedup collapses, the paper's central contribution is materially overstated. The verdict should remain CONDITIONAL until this check is performed, exactly as the reader's verdict prescribes; hence UNCHANGED.","tokens_in":25184,"tokens_out":12582,"duration_ms":154614,"concrete_test":"Re-run the Section 5.3 throughput benchmark (SDAR, ShareGPT, batch sizes 32–256) with repetition termination disabled, and separately record the termination rate and total generated tokens per system. If the FOCUS/LMDeploy throughput ratio remains at least roughly 2× for the B=32 setup and the termination rates are similar (within a few percentage points), the bias is not material. Also re-run with a fixed repetition fallback (e.g., stop after 3 identical consecutive generations) to assess sensitivity to the termination policy.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is a specific throughput number: up to 3.52× over LMDeploy (Figure 7). Appendix F.1 states: 'we integrate a mechanism within the benchmarking script to detect and terminate requests exhibiting excessive repetition.' This mechanism is not mentioned in the main text, its threshold is not defined, and its frequency is not reported. DLLM inference time is proportional to the number of denoising steps; a degenerate repetitive sequence can run arbitrarily long. Terminating such requests removes the longest-running workloads from the measurement. If FOCUS's eviction reduces repetition — the paper argues it acts as a 'safety filter' (§5.2, Table 6) — while LMDeploy is more prone to degeneracy, then the baseline's throughput is disproportionately depressed by these truncations, inflating the measured speedup. Under continuous batching, a terminated request also frees a batch slot, further raising measured throughput. The paper does not quantify termination rates, discarded tokens, or whether the mechanism is applied consistently in the Fast-dLLM v2 comparison (Appendix F.2). This is a threat to the internal validity of the throughput headline, independent of the predictor's quality.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FOCUS, a training-free inference system for block-diffusion large language models (DLLMs). The key observation is that the per-token drift in early-layer attention importance (the difference between Layer 1 and Layer 0 incoming attention scores, the 'importance delta') correlates with whether a masked token will be decoded at the current denoising step. FOCUS computes this delta after the first two layers, evicts low-delta tokens before computing the remaining layers, and dynamically sets the retention budget from historical decoding yield and within-block delta statistics. Evaluations on SDAR-8B-Chat and LLaDA2.0-mini report quality matching or exceeding baselines on GSM8K, Math500, HumanEval, MBPP, and IFEval, and throughput gains over LMDeploy up to 2.32x at block size 32 and 3.52x at block size 64.","tokens_in":25481,"tokens_out":2645,"duration_ms":35671,"significance":"If the reported results hold, FOCUS addresses a real bottleneck in DLLM serving: block-wise diffusion recomputes attention/FFN for the full block even though only a small fraction of tokens decode per step. The paper gives a clean, falsifiable hypothesis (early-layer importance delta predicts decodability), a concrete system design with custom Triton kernels, and a public implementation. The claim that a training-free eviction policy can preserve or even improve quality while cutting FLOPs is significant for the growing DLLM inference literature. The main caveats are experimental: a benchmark termination rule that can bias throughput, and limited validation of quality on open-domain chat/long-reasoning settings.","major_comments":[{"comment":"The throughput benchmark includes a mechanism that detects and terminates requests with excessive repetition, but the main text does not disclose this, the trigger threshold is not defined, and the frequency/impact is not quantified. DLLM decoding time grows with the number of denoising steps, so removing degenerate long-running requests can disproportionately raise measured tokens/s, and under continuous batching it also frees batch slots. If FOCUS reduces repetition while the LMDeploy baseline does not, the reported 3.52x speedup is inflated. Please report termination rates and discarded-token counts for each system, specify the detection rule, and provide a sensitivity analysis with the mechanism disabled or applied uniformly to all baselines, including Fast-dLLM v2.","section":"Appendix F.1, Section 5.3, Figure 7"},{"comment":"The same five benchmarks (GSM8K, Math500, HumanEval, MBPP, IFEval) are used to (a) establish the correlation in Figure 4, (b) compare fixed Top/Random/Bottom selection in Table 3, and (c) tune alpha and the confidence threshold in Table 4. This creates a circularity: the reported quality results are on the same distribution used for predictor development and hyperparameter selection. Please split the data into a calibration set and a held-out test set, or add at least one held-out benchmark family (e.g., another reasoning or instruction-following suite), and report tuned-vs-default results.","section":"Section 3.2, Table 4, Section 5.1"},{"comment":"The throughput experiments use ShareGPT and WildChat, but generation quality is only reported on the five academic benchmarks; there is no quality measurement on open-domain chat or long, multi-step reasoning. Since FOCUS physically evicts tokens, a failure of the importance-delta predictor on such distributions would silently degrade output quality while throughput numbers look good. Please measure quality (e.g., response-level metrics or human/LLM-judge scores) on a sample of ShareGPT/WildChat, and on long MATH samples or another long-chain reasoning set, so the reader can verify that the predictor's generalization is not merely assumed.","section":"Section 5.3, Section 5.2, Appendix B"}],"minor_comments":[{"comment":"The softmax in Eq. (2) is written over S^{(h)}_{i,j} without specifying the normalization axis; the surrounding text suggests a row/query-wise softmax, but this should be explicit.","section":"Eq. (2)"},{"comment":"The 'importance delta' is motivated as Common Mode Rejection, but the statistical properties of the difference (e.g., correlation between Layer 0 and Layer 1 importance) are not analyzed. A sentence quantifying the typical correlation would strengthen the claim.","section":"Section 3.2, Eq. (3)"},{"comment":"The caption of Figure 6 reports up to 2.32x while the abstract/conclusion emphasize 3.52x from Figure 7 (B=64). Consider stating the block-size dependence in the main-text figure captions to avoid apparent inconsistency.","section":"Figure 6 and Figure 7"},{"comment":"Disabling multi-loop optimization in LMDeploy is an important implementation detail; the resulting scheduler overhead should be reported quantitatively in the main text, not only in the appendix, because it affects the fairness of the speedup comparison.","section":"Appendix E.3.4"},{"comment":"The paper says results are averaged over two runs, but no variance or error bars are shown. Given that throughput measurements can be noisy, reporting min/max or standard error would help.","section":"Appendix F.1"}],"recommendation":"major_revision","confidential_remarks":"The core idea is timely and the system appears non-trivial to build; the GitHub release is a strength. However, the headline throughput result currently rests on an undisclosed repetition-termination rule, and the quality evaluation is concentrated on a small set of benchmarks that also served for tuning. Both issues are addressable with additional experiments and disclosure. I recommend major revision rather than rejection, because the central mechanism is plausible and the reported quality preservation is encouraging."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, the core idea is real and worth engaging with: the paper predicts which masked tokens in a block-diffusion LLM will decode by looking at the change in incoming attention between layers 0 and 1, then evicts the rest from the forward pass. That is a genuine departure from the prior KV-cache and approximate-caching work, and the cheap two-layer delta is a plausible, well-motivated signal. Second, the 3.52× throughput headline is not yet well supported as reported, because the benchmarking script terminates requests with excessive repetition and the paper does not quantify the termination rate or its differential effect on the two systems.\n\nWhat the paper does well: the systems work is serious. There is a real implementation on top of LMDeploy with custom Triton kernels, 4,000+ lines, and a public repo. The ablations are useful, especially the comparison showing the delta metric beats raw Layer 1 attention. The observation that only ~10% of tokens decode per step is clearly documented across models and block sizes. The design constraints around AR-context preservation and placeholder integrity make sense, and the overhead analysis is at least plausible. I do not see a load-bearing logical flaw. The importance-delta predictor is empirical, which is fine as long as it is not oversold, and the paper mostly does not oversell it.\n\nThe soft spots are real but fixable. The repetition-termination issue in Appendix F.1 is the most serious one. Under continuous batching, terminated requests free slots, so even a modest termination rate can inflate measured throughput, and if FOCUS suppresses repetition while the baseline degenerates, the comparison is systematically unfair. This needs to be quantified or the benchmark rerun without truncation. Second, all quality numbers are averages over two runs with no error bars; given the small benchmark set and the fact that α, the confidence threshold, and the MaxPool kernel size are chosen on these same benchmarks, the quality-preservation claim is weaker than the tables suggest. This is mild circularity, not a fatal one. Third, the theoretical safety bound in Appendix D is explicitly conditional on a zero-mean Gaussian model of the importance delta. The paper acknowledges the approximation and argues it is conservative, which is honest, but the bound should not be cited without that caveat. Fourth, the generalization worry is legitimate: the correlation is demonstrated on math, code, and instruction-following, not on long open-domain chat or complex multi-step reasoning. If the delta signal weakens there, eviction silently drops quality.\n\nI would send this to peer review. Ask for the termination statistics, quality confidence intervals, and at least one long-form or open-domain chat evaluation. If those hold up, this is a solid contribution to the LLM-systems literature.","headline":"Genuinely new compute-centric token eviction idea for block-diffusion LLMs, but the headline throughput claim is not safe until the benchmark's repetition-termination behavior is quantified or removed.","tokens_in":25954,"tokens_out":1935,"would_cite":true,"duration_ms":26252,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Diffusion large language models can be decoded several times faster by evicting, after two layers, tokens whose attention delta says they will not decode.","keywords":["diffusion language models","block diffusion","token eviction","attention importance delta","inference acceleration","compute-bound decoding","throughput scaling"],"falsifier":"Run the FOCUS eviction policy on long, open-domain conversations or multi-step reasoning chains and compare, token by token, the evicted set against the set the full model actually decodes at the next step. If a substantial fraction (say >5%) of evicted tokens would have been decoded, or if downstream quality degrades measurably despite the same confidence threshold, the importance-delta predictor is not robust enough to support the speedup claim.","tokens_in":25067,"feed_emoji":"⚡","tokens_out":4752,"duration_ms":51513,"temperature":0.7,"pith_summary":"Diffusion large language models (DLLMs) generate many candidate tokens in parallel, but at each denoising step only a small fraction—roughly 10%—are actually decoded; the rest of the block still consumes full attention and feed-forward compute. This paper claims that which tokens will decode is predictable before most of the network runs: the rise in attention importance between the first two transformer layers, the 'importance delta,' correlates strongly with a token's decoding probability. On that basis it builds FOCUS, a training-free inference system that computes only the first two layers for the whole block, evicts low-delta tokens, and runs the remaining layers only on the retained candidates. The paper reports that this removes 65–80% of per-step token processing, restoring throughput scaling at large batch sizes, with up to 3.52x throughput over a state-of-the-art serving engine while preserving or improving benchmark quality.","feed_headline":"Skipping undecodable tokens makes diffusion LLMs 3.5x faster","feed_subtitle":"A two-layer attention delta predicts which block tokens will decode, letting inference focus compute on the few that matter.","key_machinery":"The central object is the importance delta, ΔI_j = I_j^(Layer1) − I_j^(Layer0), where I_j is the column-wise sum of softmax attention weights received by token j from all query tokens in the block, smoothed by MaxPool1D. The subtraction acts as common-mode rejection, removing Layer 0's positional/noise baseline and isolating the semantic lift in Layer 1. FOCUS uses ΔI to select top-K candidates under a dynamic budget, then applies two structural constraints—retaining each candidate's immediate predecessor and all masked tokens before the rightmost candidate—and an intra-block KV cache with a neighbor-aware stability criterion. This mechanism converts a statistical correlation into a wall-clo","core_discovery":"FOCUS establishes that token decodability in block-diffusion language models is not random but can be read off from the change in incoming attention between Layer 0 and Layer 1. Layer 0 attention is dominated by static priors and noise; Layer 1 shows a semantic lift for tokens that are about to decode. The difference, ΔI, separates decodable from non-decodable tokens, so the system can evict the latter after computing only two layers and run the rest of the transformer on a reduced token set. Combined with a dynamic retention budget and a neighbor-aware KV cache that freezes only stabilized tokens, this brings the computation-to-generation ratio close to the 1:1 ideal of autoregressive model","pith_inferences":["Beyond the paper: if the importance-delta predictor is a universal property of masked diffusion models, it could be folded into training—models could be regularized to make decodability even more separable, or the heuristic budget could be replaced by a learned scheduler.","Beyond the paper: the reported speedup depends on a throughput benchmark that terminates repetitive requests; in open-domain chat or long reasoning where such termination is disallowed, the gains may be smaller and quality effects should be measured directly.","Beyond the paper: a natural test is to apply the same two-layer eviction rule to non-English or multimodal block-diffusion models; the Layer 0 vs Layer 1 attention delta should be re-validated before assuming the correlation transfers."],"forward_implications":["DLLM serving shifts from compute-bound saturation back to throughput scaling as batch size grows, because effective FLOPs per step drop roughly in proportion to the eviction rate.","The predictor can act as a safety filter, allowing lower confidence thresholds without the quality degradation caused by high-confidence-but-wrong tokens.","Because the technique is training-free and demonstrated across both dense and mixture-of-experts architectures, it can be applied to future block-diffusion models without retraining.","A smaller retained set also reduces the number of KV states to refresh and freeze, shrinking memory traffic and cache-management overhead, not just arithmetic.","Diffusion decoding moves closer to the 1:1 computation-to-generation ratio of autoregressive models, suggesting that DLLM serving efficiency could approach AR engine levels on the compute side."],"fun_headline_variants":["Attention delta tells DLLMs which tokens to skip","FOCUS speeds diffusion LLMs 3.5x by evicting dead tokens","Decodability predicted by layer-1 attention lift","Two-layer attention delta unlocks 3.5x DLLM speedup"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the Layer 0-to-Layer 1 attention delta reliably separates decodable from non-decodable tokens in every context, not just the tested benchmarks and models; if that correlation weakens, FOCUS evicts tokens the model would have decoded and quality silently drops.","fun_headline_variants_meta":{"raw":{"variants":["Attention delta tells DLLMs which tokens to skip","FOCUS speeds diffusion LLMs 3.5x by evicting dead tokens","Decodability predicted by layer-1 attention lift","Two-layer attention delta unlocks 3.5x DLLM speedup"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000481,"raw_usage":{"total_tokens":2202,"prompt_tokens":715,"completion_tokens":1487,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":459,"completion_tokens_details":{"reasoning_tokens":1413}},"tokens_in":459,"tokens_out":1487,"duration_ms":11236,"temperature":1.0,"reasoning_tokens":1413,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T06:06:28.504560+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the FOCUS eviction policy on long, open-domain conversations or multi-step reasoning chains and compare, token by token, the evicted set against the set the full model actually decodes at the next step. If a substantial fraction (say >5%) of evicted tokens would have been decoded, or if downstream quality degrades measurably despite the same confidence threshold, the importance-delta predictor is not robust enough to support the speedup claim.","supporting_citations":[],"review_version":1}