Pith. sign in

REVIEW 4 major objections 5 minor 3 cited by

Block-wise diffusion decoding of LLMs can be accelerated by treating most masked suffix tokens as redundant placeholders, pruning them to a small sliding window plus a trailing position token, and adapting the confidence threshold as tokens

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 08:08 UTC pith:YJHOCGBA

load-bearing objection Useful training-free acceleration for dLLMs, but the headline speedups against vanilla overstate the contribution because the baseline lacks an EOS early exit; the honest gains over Fast-dLLM are still solid. the 4 major comments →

arxiv 2601.17917 v3 pith:YJHOCGBA submitted 2026-01-25 cs.LG cs.AIcs.CLcs.CV

Streaming-dLLM: Accelerating Diffusion LLMs via Suffix Pruning and Dynamic Decoding

Zhongyu Xiao , Zhiwei Hao , Jianyuan Guo , Yong Luo , Jia Liu , Jie Xu , Han Hu This is my paper
classification cs.LG cs.AIcs.CLcs.CV
keywords diffusion language modelsinference accelerationsuffix pruningdynamic confidence thresholdearly exitblock-wise decodingtraining-freethroughput speedup
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that block-wise diffusion decoding of LLMs wastes computation in two ways: it treats the entire masked suffix of the sequence as informative, when attention actually concentrates on a few neighboring blocks and the final position, and it applies a fixed confidence threshold across all denoising steps, when confidence grows and tightens over time. Streaming-dLLM is a training-free inference recipe that prunes the suffix to a small sliding window plus one trailing positional token, adapts the acceptance threshold to the remaining mask ratio, and exits early when EOS is predicted. On Dream, LLaDA, and LLaDA-1.5 across reasoning and code benchmarks it reports throughput gains of up to 68.2x at length 512 and 225.3x at length 2048 with equal or slightly better accuracy. The speedups are large, but they are measured against a vanilla decoder that processes all L mask positions even after EOS has been generated.

Core claim

The central claim is that the redundancy in block-wise diffusion decoding is removable at inference time without retraining: the suffix region of the sequence is, for most of the decoding, only a structural placeholder, and token confidence converges at different rates within a block. By feeding the model only the current block, a small window of nearby suffix blocks, and a single token carrying the final position (via RoPE position IDs), and by accepting tokens with an adaptive threshold that decreases as the masked ratio shrinks, the authors show a diffusion LLM can decode 3.7x–225.3x faster while keeping or slightly improving benchmark accuracy. The early-exit mechanism stops decoding as

What carries the argument

The load-bearing mechanism is the suffix approximation: instead of attending to all remaining mask blocks, the model attends only to the w blocks immediately after the current block plus the terminal position token, preserving positional ordering through RoPE position IDs. This is combined with a dynamic confidence-aware acceptance rule that lowers the threshold as a block's mask ratio shrinks, plus an early exit on EOS. The KV cache of the decoded prefix is reused across iterations, so the per-step forward pass is over a much shorter sequence.

Load-bearing premise

The reported speedups assume a baseline that keeps denoising every mask position even after an end-of-sequence token has been predicted, so part of the gain comes from skipping post-EOS blocks rather than from suffix pruning or dynamic thresholds.

What would settle it

Run the same benchmarks with a baseline that stops generation as soon as EOS is predicted and uses the best fixed confidence threshold per task; if Streaming-dLLM's speedup over this baseline approaches 1x, the gains come from the early exit rather than suffix pruning.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • A training-free speedup is available for existing block-wise dLLMs without any fine-tuning or architectural change.
  • Suffix pruning can be tuned with a single sliding-window parameter; the paper shows accuracy saturates around a window of 128 tokens and throughput then falls, implying most suffix tokens are dispensable.
  • Dynamic thresholding lets high-confidence tokens finalize early, increasing parallel decoding throughput without the accuracy drop of a fixed high threshold.
  • Early exit on EOS is safe across the tested benchmarks, per the ablation results.
  • Longer generation lengths amplify the gain, up to 225.3x at 2048 tokens, because the suffix region grows with the target length while the retained window stays fixed.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The headline speedups would shrink if the baseline were allowed to stop at the EOS token; a fair ablation isolating suffix pruning from early exit would clarify how much of the 68.2x and 225.3x comes from each component.
  • Because the retained trailing position token appears necessary (dropping it costs accuracy), the method suggests that many dLLMs rely on position cues more than on the semantic content of distant masked tokens—a claim that could be tested by feeding random tokens at those positions.
  • The same attenuation logic may transfer to other block-diffusion or semi-autoregressive models, or to sliding-window attention during training, which could bake in the efficiency rather than retrofitting it at inference.
  • Accuracy gains on MBPP with one model hint that distant masked tokens act as noise; a testable extension is comparing suffix pruning against random suffix dropout to see whether the gain comes from removing interference or from the windowed prior.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Streaming-dLLM, a training-free inference acceleration framework for block-wise diffusion LLMs. It combines three mechanisms: attenuation-guided suffix modeling (pruning distant suffix mask tokens and keeping only a sliding window of nearby blocks plus a single trailing positional token), dynamic confidence-aware parallel decoding (an adaptive acceptance threshold, Eq. (10)), and an EOS early-exit that skips all remaining blocks once a high-confidence EOS is predicted. Experiments on Dream, LLaDA, and LLaDA-1.5 report accuracy and throughput on HumanEval, GSM8K, MBPP, and MATH, with headline speedups up to 68.2× over vanilla decoding and up to 225.3× at generation length 2048.

Significance. The paper identifies two plausible sources of inefficiency in block-wise dLLMs: spatial redundancy in the suffix mask region and temporal rigidity of fixed confidence thresholds. The proposed suffix-pruning idea, with the trailing positional token, is interesting and could be practically useful. The code is provided, and the authors include detailed hyperparameter tables and ablations. However, the central quantitative claim—the 68.2× and 225.3× speedups—is not supported as stated, because the vanilla baseline is apparently not allowed to use an EOS early exit. A large share of the reported gain likely comes from skipping post-EOS computation rather than from the proposed suffix pruning or adaptive threshold. The marginal gains over Fast-dLLM appear much smaller (around 2–3× in the ablation table), and need to be isolated with a fair baseline.

major comments (4)
  1. [§3.3, §4.1, Tables 2 and 5] The headline speedups are computed against a vanilla decoder that appears to decode all L mask positions even after an EOS token has been predicted. Section 3.3 introduces early exit for Streaming-dLLM, while §4.1 states that throughput counts only non-EOS tokens across the entire generated sequence. Thus the denominator for the vanilla baseline includes post-EOS blocks whose outputs are excluded from the numerator, while Streaming-dLLM skips those blocks entirely. A trivial EOS-aware baseline would capture a large fraction of the reported 68.2× and 225.3× gains. Please re-benchmark against a vanilla baseline that stops at EOS (or reports separate EOS-aware speedups) and against Fast-dLLM with the same early-exit mechanism enabled.
  2. [Table 3] The ablation baseline is not the vanilla model. In Table 3, the LLaDA-1.5 row with all modules disabled reports Accuracy 80.4 and Throughput 25.8 tok/s, which exactly matches the Fast-dLLM row in Table 2, while the vanilla LLaDA-1.5 throughput on the same task is 2.5 tok/s. Consequently, the ablation attributes Fast-dLLM's KV-cache and parallel-decoding gains to 'no modules', and the marginal contribution of the three proposed modules over Fast-dLLM is only about 2.7× on this task. Please report ablations relative to both the vanilla model and Fast-dLLM so that the reader can separate the contributions.
  3. [Eq. (10), Table 11] The dynamic threshold τ(t)=τ0(1−α(1−r_mask)) is introduced as a heuristic, and Table 11 shows that the sliding window size w and adaptation strength α are tuned separately for each benchmark (α from 0.1 to 0.7, w from 32 to 256). Because these values are selected on the same benchmarks where the headline speedups are reported, and Figures 5–6 show strong sensitivity to them, the claim that generation quality is 'maintained' is vulnerable to selection bias. Please provide validation-based hyperparameter selection, error bars, or a sensitivity analysis to support the generalization of the speed/quality trade-off.
  4. [Table 7, Section B.1] On LLaDA-Instruct MBPP, the proposed method improves accuracy from 29.4 to 42.0 at length 256 and from 14.8 to 40.8 at length 512, while pruning suffix information. This is an unusually large and unexplained improvement from an efficiency-only intervention. It may reflect a real effect (e.g., suppressing noisy masked-token attention), but it needs explicit analysis and verification that the evaluation setup is identical. As it stands, this result casts doubt on whether the accuracy numbers are directly comparable across methods.
minor comments (5)
  1. [Section 5] The conclusion states 'up to 68.3× inference speedup', which is inconsistent with the abstract's '68.2×'. Please unify.
  2. [Figure 5] The x-axis label 'no suffix windows, meansize=512' is ambiguous. It should be stated explicitly that 'no suffix window' corresponds to keeping the full suffix of length 512, and that the window size is measured in tokens.
  3. [Table 3] The caption says 'Dream uses GSM8K-CoT', but the Dream rows report throughput 41.7 tok/s for '× × ×', which does not match the Dream vanilla throughput in Table 1 (7.1 tok/s at length 512). Clarify what the Dream baseline in Table 3 actually is.
  4. [§4.1] The statement 'For throughput computation, we count only non EOS tokens across the entire generated sequence' should be clarified with respect to sequences that terminate early: does the denominator include the time spent on blocks after EOS for models without early exit? This is essential for interpreting all throughput tables.
  5. [Notation, Eq. (7)] The notation in Eq. (7) mixes sets of indices and regions (˜I and ˜S_suffix); please define the relationship explicitly to avoid confusion about whether the trailing position is a real token or a masked positional cue.

Circularity Check

0 steps flagged

No significant circularity; headline speedups are measured outcomes, with a benchmarking-fairness caveat about the EOS early-exit baseline.

full rationale

This is an empirical acceleration paper whose central claims are measured throughput, latency, and accuracy numbers, not derived predictions. No equation is defined in terms of another in a way that forces a headline result. The adaptive threshold τ(t)=τ0(1−α(1−r_mask)) (Eq. 10) is a heuristic whose parameters (τ0=0.9, α, window size w) are tuned per benchmark in Table 11; the reported speedups are measured with those settings, not predicted from them, so the fitted-input-called-prediction pattern does not apply. The suffix-pruning and confidence-decoding components are evaluated against external baselines (Dream, LLaDA, LLaDA-1.5, Fast-dLLM, dKV-Cache) in Tables 1–5 and 7–10, and the ablation in Table 3 decomposes the contributions of the three modules. The only self-citation is LGViT (Xu et al., 2023) in Section B.2, used to justify latency as a real-time deployment metric; it is not load-bearing. The reviewer-flagged concern that the headline 68.2×/225.3× speedups partly reflect the vanilla baseline decoding all L mask positions even after EOS is an experimental-comparison/fairness issue, not circularity: the speedup is a measured outcome, the early-exit mechanism is a genuine component, and the ablation partially separates its contribution. The paper does not derive its central claims from its own assumptions by construction, and no load-bearing argument reduces to a self-citation chain. Score 2 reflects the minor non-load-bearing self-citation and the baseline-fairness caveat, not a circular derivation.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 0 invented entities

The paper is an empirical engineering contribution, not a derivation from first principles. Its central claim rests on two empirical observations (attention locality and confidence evolution), a set of per-benchmark tuned hyperparameters, and a baseline-efficiency assumption. No new physical or model-level entities are introduced, but the trailing positional token is a synthetic structural cue whose value is only validated by ablations in Table 6.

free parameters (5)
  • base confidence threshold τ0 = 0.9 for all configurations
    Used in Eq. 10; fixed by hand rather than derived. Reported in Table 11.
  • adaptation strength α = 0.1–0.7 per benchmark and generation length
    Controls how quickly the confidence threshold decays with mask ratio in Eq. 10. Tuned per dataset (Table 11).
  • sliding window size w = 32–256 per benchmark and generation length
    Number of suffix blocks retained in Eq. 7; selected via ablation in Figure 5 and per-dataset in Table 11.
  • block size K = 32
    Partitions the suffix into blocks; set to 32 across all experiments (Table 11).
  • early-exit EOS confidence threshold = not reported
    Section 3.3 says EOS triggers early exit when predicted with 'high confidence', but the exact confidence threshold is never specified, so it is an additional unstated tuned constant.
axioms (4)
  • domain assumption Distant suffix masked tokens carry negligible information for generating the current block, and this attention locality holds across layers, models, and tasks.
    Figure 2 measures attention only at the final layer (Layer 31) of LLaDA-1.5 on 300 GSM8K samples, yet Eq. 7 globally prunes the suffix to w neighboring blocks plus the trailing position.
  • domain assumption Token confidence within a block evolves in a way that a mask-ratio-scaled threshold improves acceptance; i.e., Eq. 10 is a valid schedule.
    Figure 3 is based on 100 GSM8K samples; Appendix A itself reports that mean token confidence can temporarily decrease during decoding, weakening the monotonicity intuition.
  • domain assumption Once a block predicts EOS, all subsequent blocks can be safely skipped without losing required content.
    Section 3.3 introduces this early exit; no analysis is given for cases where EOS appears in a block but later blocks might contain necessary continuation tokens.
  • domain assumption Throughput defined as non-EOS tokens per second with full fixed-length decoding in the baseline is the right efficiency comparison.
    Section 4.1 defines throughput this way; the baseline is not given an EOS early exit. This assumption is what produces the 68×–225× headline speedups.

pith-pipeline@v1.3.0-alltime-deepseek · 18687 in / 13905 out tokens · 156172 ms · 2026-08-03T08:08:57.794835+00:00 · methodology

0 comments
read the original abstract

Diffusion Large Language Models (dLLMs) offer a compelling paradigm for natural language generation, leveraging parallel decoding and bidirectional attention to achieve superior global coherence compared to autoregressive models. While recent works have accelerated inference via KV cache reuse or heuristic decoding, they overlook the intrinsic inefficiencies within the block-wise diffusion process. Specifically, they suffer from spatial redundancy by modeling informative-sparse suffix regions uniformly and temporal inefficiency by applying fixed denoising schedules across all the decoding process. To address this, we propose Streaming-dLLM, a training-free framework that streamlines inference across both spatial and temporal dimensions. Spatially, we introduce attenuation guided suffix modeling to approximate the full context by pruning redundant mask tokens. Temporally, we employ a dynamic confidence aware strategy with an early exit mechanism, allowing the model to skip unnecessary iterations for converged tokens. Extensive experiments show that Streaming-dLLM achieves up to 68.2X speedup while maintaining generation quality, highlighting its effectiveness in diffusion decoding. The code is available at https://github.com/xiaoshideta/Streaming-dLLM.

Figures

Figures reproduced from arXiv: 2601.17917 by Han Hu, Jia Liu, Jianyuan Guo, Jie Xu, Yong Luo, Zhiwei Hao, Zhongyu Xiao.

Figure 1
Figure 1. Figure 1: Comparison of accuracy and throughput across different acceleration strategies. Our proposed method improves inference throughput while maintaining competitive accuracy compared to prior approaches. guage processing (Nie et al., 2025; Ye et al., 2025; Zhu et al., 2025a;b; You et al., 2025; Sahoo et al., 2024). In contrast to traditional autoregressive Large Language mod￾els (LLMs) (Achiam et al., 2023; Guo… view at source ↗
Figure 2
Figure 2. Figure 2: Attention distribution at the final layer (Layer 31) of LLaDA-1.5, showing interactions between the current block and the full input sequence. The solid line denotes the mean atten￾tion score at each diffusion step, the shaded region represents the interquartile range (IQR, 25%–75%), and gray dashed lines delin￾eate the prefix, current-generation and suffix regions. Attention is concentrated on a few neigh… view at source ↗
Figure 3
Figure 3. Figure 3: Token confidence distribution during iterative block-wise generation of LLaDA-1.5 on GSM8K with a generation length of 256, showing that mean token confidence steadily increases over iterations. The solid line denotes the mean confidence at each diffusion step and the shaded area represents the interquartile range (IQR, 25%–75%). While a high confidence threshold ensures generation quality, it can be conse… view at source ↗
Figure 4
Figure 4. Figure 4: Illustration of Attenuation-Guided Suffix Modeling. For each block, the nearest neighboring region following the current block is retained using a sliding window (red dashed box) and concatenated with the trailing position to form an approximate suffix region. step t, which is subsequently fed into the diffusion language model for prediction. x˜ (t) = S (t) prefix ∪ S (t) current ∪ S˜ (t) suffix, (8) This … view at source ↗
Figure 5
Figure 5. Figure 5: Ablation study on the sliding window size (no suffix windows, mean size = 512). 0.1 0.2 0.4 0.6 0.7 0.8 1.0 Alpha 70 75 80 GSM8K (5-shot) Accuracy No Adaptive threshold Accuracy Throughput 60 65 70 Throughput (tokens/s) No Adaptive threshold 1.21× Speedup [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Effect of the parallel decoding parameter α (no adaptive threshold, mean α = 0). 8 [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
Figure 8
Figure 8. Figure 8: Token confidence distribution of block 1. 0 5 10 15 20 25 Diffusion Step 0.2 0.4 0.6 0.8 1.0 Confidence Threshold Avg Confidence IQR (25% 75%) [PITH_FULL_IMAGE:figures/full_fig_p012_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Token confidence distribution of block 2. 0 5 10 15 Diffusion Step 0.2 0.4 0.6 0.8 1.0 Confidence Threshold Avg Confidence IQR (25% 75%) [PITH_FULL_IMAGE:figures/full_fig_p012_9.png] view at source ↗
Figure 11
Figure 11. Figure 11: Token confidence distribution of block 4. 0 5 10 15 20 Diffusion Step 0.2 0.4 0.6 0.8 1.0 Confidence Threshold Avg Confidence IQR (25% 75%) [PITH_FULL_IMAGE:figures/full_fig_p013_11.png] view at source ↗
Figure 13
Figure 13. Figure 13: Token confidence distribution of block 6. 0 5 10 15 20 Diffusion Step 0.2 0.4 0.6 0.8 1.0 Confidence Threshold Avg Confidence IQR (25% 75%) [PITH_FULL_IMAGE:figures/full_fig_p013_13.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Continuous Language Diffusion as a Decoder-Interface Problem

    cs.CL 2026-06 unverdicted novelty 7.0

    Continuous language diffusion works by entering high-margin decoder basins where frozen T5 embeddings recover 93-96% of native decisions and linear readouts reach 97.9% agreement, implying models should be evaluated a...

  2. Where and When to Commit: Candidate-Aware Decoding for Diffusion Language Models

    cs.CL 2026-07 conditional novelty 6.0

    Candidate-aware early exit (CVC) plus block-local acceleration (BWEC) give training-free DLM speedups of 2–18× within 2 pp of full-decode accuracy under one frozen hyperparameter set.

  3. TACG: Trajectory-Aware Commit Gating for Diffusion Language Model Decoding

    cs.CL 2026-07 conditional novelty 5.5

    Trajectory-aware commit gating (TILG + History Gate + capped extra promotion) improves or preserves DLLM accuracy while reducing steps and raising tokens-per-forward without retraining.

Reference graph

Works this paper leans on

27 extracted references · 16 linked inside Pith · cited by 3 Pith papers

  1. [1]

    L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al

    Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774,

    Pith/arXiv arXiv

  2. [5]

    The llama 3 herd of models.arXiv preprint arXiv:2407.21783,

    Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783,

    Pith/arXiv arXiv

  3. [7]

    Empowering diffusion models on the embedding space for text generation

    9 Streaming-dLLM: Accelerating Diffusion LLMs via Suffix Pruning and Dynamic Decoding Gao, Z., Guo, J., Tan, X., Zhu, Y ., Zhang, F., Bian, J., and Xu, L. Empowering diffusion models on the embedding space for text generation. InProceedings of the 2024 Conference of the North American Chapter of the Associ- ation for Computational Linguistics: Human Langu...

    2024

  4. [8]

    Deepseek-r1: In- centivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948,

    Guo, D., Yang, D., Zhang, H., Song, J., Zhang, R., Xu, R., Zhu, Q., Ma, S., Wang, P., Bi, X., et al. Deepseek-r1: In- centivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948,

    Pith/arXiv arXiv

  5. [9]

    Measuring mas- sive multitask language understanding.arXiv preprint arXiv:2009.03300,

    Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., and Steinhardt, J. Measuring mas- sive multitask language understanding.arXiv preprint arXiv:2009.03300,

    Pith/arXiv arXiv 2009

  6. [11]

    Wedlm: Reconciling diffusion language models with standard causal atten- tion for fast inference.arXiv preprint arXiv:2512.22737, 2025a

    Liu, A., He, M., Zeng, S., Zhang, S., Zhang, L., Wu, C., Jia, W., Liu, Y ., Zhou, X., and Zhou, J. Wedlm: Reconciling diffusion language models with standard causal atten- tion for fast inference.arXiv preprint arXiv:2512.22737, 2025a. Liu, Z., Yang, Y ., Zhang, Y ., Chen, J., Zou, C., Wei, Q., Wang, S., and Zhang, L. dllm-cache: Accelerating diffu- sion ...

    arXiv

  7. [12]

    Attention is all you need for kv cache in diffusion llms.arXiv preprint arXiv:2510.14973,

    Nguyen-Tri, Q., Ranjan, M., and Shen, Z. Attention is all you need for kv cache in diffusion llms.arXiv preprint arXiv:2510.14973,

    arXiv

  8. [13]

    Large language diffusion models.arXiv preprint arXiv:2502.09992,

    Nie, S., Zhu, F., You, Z., Zhang, X., Ou, J., Hu, J., Zhou, J., Lin, Y ., Wen, J.-R., and Li, C. Large language diffusion models.arXiv preprint arXiv:2502.09992,

    Pith/arXiv arXiv

  9. [14]

    Sparse-dllm: Accelerating diffu- sion llms with dynamic cache eviction.arXiv preprint arXiv:2508.02558, 2025a

    Song, Y ., Liu, X., Li, R., Liu, Z., Huang, Z., Guo, Q., He, Z., and Qiu, X. Sparse-dllm: Accelerating diffu- sion llms with dynamic cache eviction.arXiv preprint arXiv:2508.02558, 2025a. Song, Y ., Zhang, Z., Luo, C., Gao, P., Xia, F., Luo, H., Li, Z., Yang, Y ., Yu, H., Qu, X., et al. Seed diffusion: A large-scale diffusion language model with high-spee...

    arXiv

  10. [15]

    From next-token to next-block: A principled adaptation path for diffusion llms.arXiv preprint arXiv:2512.06776,

    Tian, Y ., Liang, Y ., Sun, J., Zhang, S., Yang, G., Shu, Y ., Fang, S., Guo, T., Han, K., Xu, C., et al. From next-token to next-block: A principled adaptation path for diffusion llms.arXiv preprint arXiv:2512.06776,

    arXiv

  11. [16]

    S., and Kuleshov, V

    Wang, G., Schiff, Y ., Sahoo, S. S., and Kuleshov, V . Re- masking discrete diffusion models with inference-time scaling.arXiv preprint arXiv:2503.00307,

    arXiv

  12. [17]

    Acceler- ating diffusion large language models with slowfast: The three golden principles.arXiv preprint arXiv:2506.10848,

    Wei, Q., Zhang, Y ., Liu, Z., Liu, D., and Zhang, L. Acceler- ating diffusion large language models with slowfast: The three golden principles.arXiv preprint arXiv:2506.10848,

    arXiv

  13. [18]

    Fast- dllm v2: Efficient block-diffusion llm.arXiv preprint arXiv:2509.26328, 2025a

    Wu, C., Zhang, H., Xue, S., Diao, S., Fu, Y ., Liu, Z., Molchanov, P., Luo, P., Han, S., and Xie, E. Fast- dllm v2: Efficient block-diffusion llm.arXiv preprint arXiv:2509.26328, 2025a. Wu, C., Zhang, H., Xue, S., Liu, Z., Diao, S., Zhu, L., Luo, P., Han, S., and Xie, E. Fast-dllm: Training-free acceler- ation of diffusion llm by enabling kv cache and par...

    arXiv

  14. [19]

    Qwen3 technical report.arXiv preprint arXiv:2505.09388,

    Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388,

    Pith/arXiv arXiv

  15. [20]

    Dream 7b: Diffusion large language models.arXiv preprint arXiv:2508.15487,

    Ye, J., Xie, Z., Zheng, L., Gao, J., Wu, Z., Jiang, X., Li, Z., and Kong, L. Dream 7b: Diffusion large language models.arXiv preprint arXiv:2508.15487,

    Pith/arXiv arXiv

  16. [21]

    Llada-v: Large language diffusion models with visual instruction tuning.arXiv preprint arXiv:2505.16933,

    You, Z., Nie, S., Zhang, X., Hu, J., Zhou, J., Lu, Z., Wen, J.-R., and Li, C. Llada-v: Large language diffusion models with visual instruction tuning.arXiv preprint arXiv:2505.16933,

    Pith/arXiv arXiv

  17. [22]

    Yu, P., Xie, S., Ma, X., Jia, B., Pang, B., Gao, R., Zhu, Y ., Zhu, S.-C., and Wu, Y . N. Latent diffusion energy-based model for interpretable text modeling.arXiv preprint arXiv:2206.05895,

    Pith/arXiv arXiv

  18. [24]

    d1: Scaling reasoning in diffusion large language models via rein- forcement learning.arXiv preprint arXiv:2504.12216,

    Zhao, S., Gupta, D., Zheng, Q., and Grover, A. d1: Scaling reasoning in diffusion large language models via rein- forcement learning.arXiv preprint arXiv:2504.12216,

    Pith/arXiv arXiv

  19. [25]

    Llada 1.5: Variance- reduced preference optimization for large language diffu- sion models.arXiv preprint arXiv:2505.19223, 2025a

    Zhu, F., Wang, R., Nie, S., Zhang, X., Wu, C., Hu, J., Zhou, J., Chen, J., Lin, Y ., Wen, J.-R., et al. Llada 1.5: Variance- reduced preference optimization for large language diffu- sion models.arXiv preprint arXiv:2505.19223, 2025a. Zhu, F., You, Z., Xing, Y ., Huang, Z., Liu, L., Zhuang, Y ., Lu, G., Wang, K., Wang, X., Wei, L., et al. Llada-moe: A spa...

    Pith/arXiv arXiv

  20. [26]

    In comparison, the second-best method, Fast-dLLM, achieves 4.4×–10.0× speedup with slightly lower accuracy

    Our method achieves 6.4×–20.6× speedup across all tasks. In comparison, the second-best method, Fast-dLLM, achieves 4.4×–10.0× speedup with slightly lower accuracy. Specifically, our method improves accuracy by 13.8%–27.0% on MBPP. We attribute this improvement to the proposed attenuation guided suffix modeling, which effectively suppresses interference f...

    2023

  21. [27]

    For all models, the block size is set to 32 across datasets, while other hyperparameters vary by benchmark and generation length

    For each benchmark and backbone (Dream, LLaDA, and LLaDA-1.5), we report the generation length, sliding window size, initial confidence threshold τ0, decay factor α, and block size used in our experiments. For all models, the block size is set to 32 across datasets, while other hyperparameters vary by benchmark and generation length. These settings are co...

    2048

  22. [2020]

    A survey on diffu- sion language models.arXiv preprint arXiv:2508.10875,

    Li, T., Chen, M., Guo, B., and Shen, Z. A survey on diffu- sion language models.arXiv preprint arXiv:2508.10875,

    Pith/arXiv arXiv

  23. [2021]

    Dlm-one: Diffusion language models for one-step sequence generation.arXiv preprint arXiv:2506.00290, 2025a

    Chen, T., Zhang, S., and Zhou, M. Dlm-one: Diffusion language models for one-step sequence generation.arXiv preprint arXiv:2506.00290, 2025a. Chen, X., Huang, S., Guo, C., Wei, C., He, Y ., Zhang, J., Li, H., Chen, Y ., et al. Dpad: Efficient diffusion language models with suffix dropout.arXiv preprint arXiv:2508.14148, 2025b. Cobbe, K., Kosaraju, V ., Ba...

    Pith/arXiv arXiv

  24. [2022]

    S., et al

    Zhang, L., Fang, L., Duan, C., He, M., Pan, L., Xiao, P., Huang, S., Zhai, Y ., Hu, X., Yu, P. S., et al. A survey on parallel text generation: From parallel decoding to diffu- sion language models.arXiv preprint arXiv:2508.08712,

    arXiv

  25. [2023]

    T., Yang, Z., Qi, Z., Han, J., Sahoo, S

    Arriola, M., Gokaslan, A., Chiu, J. T., Yang, Z., Qi, Z., Han, J., Sahoo, S. S., and Kuleshov, V . Block diffusion: Inter- polating between autoregressive and diffusion language models.arXiv preprint arXiv:2503.09573,

    Pith/arXiv arXiv

  26. [2024]

    Theoretical benefit and limitation of diffusion language model.arXiv preprint arXiv:2502.09622,

    Feng, G., Geng, Y ., Guan, J., Wu, W., Wang, L., and He, D. Theoretical benefit and limitation of diffusion language model.arXiv preprint arXiv:2502.09622,

    Pith/arXiv arXiv

  27. [2025]

    D., Ho, J., Tarlow, D., and Van Den Berg, R

    Austin, J., Johnson, D. D., Ho, J., Tarlow, D., and Van Den Berg, R. Structured denoising diffusion models in discrete state-spaces.Advances in neural information processing systems, 34:17981–17993, 2021a. Austin, J., Odena, A., Nye, M., Bosma, M., Michalewski, H., Dohan, D., Jiang, E., Cai, C., Terry, M., Le, Q., et al. Program synthesis with large langu...

    Pith/arXiv arXiv