Pith. sign in

REVIEW 2 major objections 8 minor 52 references

Block diffusion language models prefer left-to-right decoding, and a simple prefix-preserving parallel sampler beats arbitrary-order methods.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

BDLMs are training-biased toward left-to-right contexts, and PARD exploits that bias with leftmost-prefix parallel unmasking to improve quality–speed trade-offs over existing parallel DLM samplers.

T0 review reviewed 2026-07-31 challenge →

load-bearing objection Solid inference paper: PARD is a simple leftmost-prefix parallel sampler that clearly wins on current BDLMs; the “naturally AR-aligned” story is suggestive but confounded. the 2 major comments →

arxiv 2607.24306 v1 pith:BR7ZRYIX submitted 2026-07-27 cs.CL

Rethinking the Generation Order of Block Diffusion Language Models

Kai Syun Hou , James Kwok This is my paper
classification cs.CL
keywords block diffusion language modelsmasked diffusion modelsparallel decodingleft-to-right generationPARDtraining-free samplingAR bias
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Recent block diffusion language models (BDLMs) are trained and initialized in ways that make left-to-right contexts far more familiar than they are for whole-sequence masked diffusion models. The paper shows this both in how the models actually unmask tokens and in a training-context analysis: under block diffusion, AR-like mask patterns appear with non-negligible probability and higher expected alignment than under full-sequence masking. Building on that, it introduces Parallel Autoregressive Decoding (PARD), a training-free rule that takes any confidence-, margin-, or entropy-style parallel criterion and commits only the longest leftmost prefix of accepted positions. Across three recent BDLMs and six benchmarks, PARD improves generation quality over existing parallel samplers and stays close to pure left-to-right decoding while remaining substantially faster.

Core claim

BDLMs are naturally more aligned with left-to-right decoding than early MDMs, so inference should preserve AR prefix structure rather than fully arbitrary unmasking. PARD does this by unmasking the longest leftmost run of positions that pass a parallel acceptance criterion, recovering most of the quality of pure AR sampling with large throughput gains and beating prior parallel samplers on quality.

What carries the argument

Parallel Autoregressive Decoding (PARD): at each step inside a block, apply a token-level criterion (confidence, margin, or entropy) but accept only the contiguous leftmost prefix of positions that all pass the threshold, falling back to the single leftmost masked token if the prefix is empty.

Load-bearing premise

The paper treats the measured left-to-right bias from block-diffusion training contexts and AR pretraining as the real reason AR-order sampling works better at inference on current BDLMs, not just an artifact of the specific mid-size models, block size, and greedy setups tested.

What would settle it

On a larger BDLM or a different block/noise schedule, measure whether pure AR still beats strong arbitrary-order samplers on the same tasks, and whether PARD still dominates other parallel methods on the quality–throughput frontier; a reversal would undermine the claimed AR alignment story.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Default parallel decoding for BDLMs should enforce a left-to-right prefix constraint rather than free position selection.
  • Pure AR decoding remains a strong quality upper bound for these models; PARD is the practical efficiency compromise.
  • Training-context AR alignment (exact AR mask probability and expected pattern alignment) can explain why BDLMs and from-scratch MDMs want different samplers.
  • Static multi-token left-to-right commits without adaptive reliability checks collapse quality; adaptive prefix length is required.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If AR bias grows with block index as the alignment formula suggests, later blocks may tolerate more aggressive parallel prefixes than early ones.
  • Samplers designed for from-scratch MDMs may systematically mis-serve the growing class of AR-adapted BDLMs unless prefix structure is restored.
  • A natural next test is whether light sampling-aware fine-tuning on PARD rollouts closes the remaining gap to pure AR without losing speed.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 8 minor

Summary. The paper studies sampling order for block diffusion language models (BDLMs). It first shows, via decoding-order heatmaps and AR-ness@k metrics on SDAR 8B vs LLaDA 8B, that confidence-based sampling already induces a near-left-to-right order, more strongly so for the BDLM; and that a pure leftmost (AR) sampler beats arbitrary-order samplers on SDAR while losing on LLaDA. §3.3 gives two propositions under the linear schedule α_t=1−t: block-diffusion training observes exact AR mask patterns with probability 1/((L'+1)C(L',k)) (vs. astronomically smaller for full-sequence MDM), and its expected AR-pattern alignment exceeds MDM's 1/2 due to the clean prefix blocks. Motivated by this, the authors propose PARD: apply any threshold-based parallel criterion (confidence/margin/entropy) but unmask only the longest accepted leftmost prefix, falling back to the single leftmost masked token. Experiments on three AR-initialized BDLMs (Fast-dLLM v2 7B, SDAR 8B, LLaDA2.1-Mini 16B) across six benchmarks show PARD has the best average among parallel samplers on every model (66.3/73.6/83.9 avg vs 61.9/70.2/81.5 for Confidence Parallel), matches pure AR on SDAR, trails AR by ≤0.6 elsewhere, and gives 1.3×–3.6× speedups over AR. Ablations: static leftmost-k collapses; top-p sampling preserves the ranking; Dream analysis in App. D.1 attempts to disentangle AR-init vs block-diffusion training.

Significance. If the empirical claims hold — and they are well supported — this is a useful and timely contribution. Sampling for diffusion LMs is an active area, and the paper's central practical result (PARD: restrict any threshold-based parallel sampler to the longest accepted leftmost prefix) is simple, training-free, model-agnostic, and immediately deployable on public checkpoints. The evaluation is broad: three recent BDLMs (Fast-dLLM v2 7B, SDAR 8B, LLaDA2.1-Mini 16B), six external benchmarks (HumanEval/MBPP Base+Plus, GSM8K, MATH, IFEval, BBH), sequential and parallel baselines including KLASS, EB-Sampler, APD, and Hierarchy, threshold sweeps with pass@1–throughput/NFE Pareto curves, a static-leftmost ablation (Table 3) that cleanly isolates the contribution of adaptive prefix length, top-p robustness with paired t-tests, and an instructive qualitative failure case (HumanEval/111). The static-leftmost ablation in particular is a strong falsifiable check: k=2 already costs 8–20 pass@1 points, showing PARD is not merely "commit k leftmost tokens." Propositions 1–2 are correct, closed-form, and honestly derived under a stated linear schedule. The main caveat is that the causal attribution (

major comments (2)
  1. [§3.3, Propositions 1–2 and App. D.1] The paper's framing claim — that BDLMs are "naturally more aligned" with L2R decoding *because of the block-diffusion training objective* — is not established by the evidence given, and Proposition 2 in particular is close to a restatement of the architecture rather than a behavioral explanation. In the App. B.2 derivation, the expected alignment within the current block contributes exactly 1/2 (the k/2 + (L'−k−1)/2 terms), identical to MDM; the entire improvement E[ρ] = 1/2 + (b−1)L'/(2(bL'−1)) comes from the deterministic clean prefix, i.e., "conditioning on clean prefix blocks yields more prefix-like contexts," which is definitional. Meanwhile, the AR-initialization confound is real: all three evaluated BDLMs are AR-initialized, and the paper's own App. D.1 shows an AR-initialized *MDM* (Dream) already flips the ordering preference (AR 52.4 > Confidence 51.8 on HumanEval). The attribu
  2. [§5.2, Table 1] Table 1 compares PARD against parallel baselines at a single vendor-default threshold each (τc=0.9/0.9/0.7), and PARD wins the average on all three models — but the margin is uneven across tasks (e.g., on SDAR, EB-Sampler beats PARD on MBPP 78.0 vs 77.0; on Fast-dLLM v2 GSM8K, Uncode/AR beat PARD). Because the thresholds were presumably tuned by the model vendors for Confidence Parallel, not for PARD, the comparison could be more rather than less favorable to PARD — or vice versa. Given that Fig. 4–5 sweep thresholds on HumanEval only, please report the threshold-swept comparison on at least one non-code task (GSM8K or IFEval) for the main baselines, or state explicitly that per-benchmark rankings at default thresholds are sensitive to this choice. This is load-bearing for the claim "consistently outperforms existing parallel samplers in generation quality," which currently reads as univ
minor comments (8)
  1. [§3.3, Proposition 1 example] Prop 1 numerical example: at L'=32, k=15, the stated probability "about 10^{-11}" should be 1/(33·C(32,15)) ≈ 5.4×10^{-11}. Not load-bearing (the qualitative point stands), but worth correcting since the example is quoted in the main text.
  2. [Table 2] All throughput numbers are from a single A6000 at batch size 4, with absolute values as low as ~10 tokens/s (LLaDA2.1-Mini AR). The 1.29×–3.64× speedups are plausible but platform- and batch-dependent; please note this caveat near Table 2 and clarify whether throughput includes prompt processing or only generation.
  3. [§C.4 / Table 1] BBH for LLaDA2.1-Mini is an 8-task subset while the other two models use (presumably) full BBH; the Avg column in Table 1 therefore mixes incomparable quantities across models. The paper does not make cross-model claims, but a footnote in Table 1 would prevent misreading.
  4. [§3.2–3.3] Notation: the block-diffusion objective in §3.2 is written without the expectation over x_0 ∼ p_data that appears in the MDM loss; minor inconsistency. Also, k = ℓ−1−(b−1)L' is reused as "left-context length in this block" in both propositions; stating its range (0 ≤ k ≤ L'−1) once would help.
  5. [Figures 1–2] Figure 1: the two heatmaps share a colorbar but the on-diagonal masses differ (0.39 vs 0.27); consider noting the shared scale in the caption, and state the number of HumanEval generations aggregated. Figure 2's k ranges (1–8 local, 1–3 global) deserve one sentence of justification since local AR-ness@k is near-trivially small for large k.
  6. [§4] PARD's fallback rule (unmask the single leftmost masked position when the accepted prefix is empty) means PARD never unmasks fewer tokens than pure AR, but it also means a low-confidence leftmost token is forced. Table 3 shows static leftmost-k collapses, so this forcing is apparently harmless with the criterion gate, but one sentence discussing worst-case behavior (e.g., entropy spikes at decision points) would be useful.
  7. [App. D.4] The qualitative example (Fig. 14–15) is compelling; consider also reporting one case where the AR-prefix constraint hurts relative to Confidence Parallel, to preempt the impression of selection.
  8. [Throughout] Typos/formatting: "proposeParallel" (p.1); "rules.Confidenceselects", "onMarginselects", "positions.Entropyselects" (§2.2); "In this work, we study" duplicated phrasing between abstract and intro; missing space "L'=32,the". Several references cite future-dated arXiv preprints (2026) — verify these are stable at camera-ready.

Circularity Check

0 steps flagged

No significant circularity: analytical mask-pattern facts follow from the stated forward process; PARD and quality claims are external-benchmark evaluations, not fits renamed as predictions.

full rationale

The paper’s load-bearing chain is (i) empirical decoding-order/AR-ness comparisons on public checkpoints, (ii) Propositions 1–2 computing Pr(AR mask) and E[ρ] under the stated linear schedule and block factorization, and (iii) training-free PARD evaluated on HumanEval/MBPP/GSM8K/MATH/IFEval/BBH. Props 1–2 are direct integrals over the training forward process (App. B); they characterize how often AR-like contexts appear and do not fit a free parameter then re-report it as a prediction. Prop 2’s clean-prefix term is architecturally immediate, but that is transparent analysis of the objective, not a circular ‘prediction equals input’ identity. PARD thresholds are swept or taken from model defaults and scored against external pass@1/throughput—not quantities forced by construction. There is no uniqueness theorem, ansatz smuggled via overlapping-author citation, or self-citation that alone justifies the central claim. Residual AR pretraining is discussed openly (incl. Dream in App. D.1). Standard self-contained ML methods paper; circularity score 0.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 1 invented entities

The central story rests on standard discrete masked-diffusion and block-diffusion training assumptions, the linear noise schedule used in the propositions, residual AR behavior after relatively small block-diffusion adaptation budgets, and hand-chosen or default unmasking thresholds. No new physical entities are postulated; PARD is an inference rule over existing predictive distributions.

free parameters (5)
  • confidence threshold τ_c = 0.9 / 0.7 (defaults); sweep {0.7,0.75,0.8,0.85,0.9}
    Controls which tokens enter the accepted set / leftmost prefix; main benchmark runs fix τ_c=0.9 (Fast-dLLM v2, SDAR) or 0.7 (LLaDA2.1-Mini), with sweeps elsewhere.
  • margin threshold τ_m = sweep {0.7,0.8,0.9,0.95,0.97}
    Acceptance cutoff for margin-based parallel and PARD variants in trade-off curves.
  • entropy threshold τ_h = sweep {0.1..0.7}
    Acceptance cutoff for entropy-based parallel and PARD variants.
  • block size L' and Fast-dLLM sub-block size = L'=32; sub-block=8
    Architectural/inference chunking fixed to model defaults (L'=32, sub-block 8), which shapes both AR-context probabilities and achievable parallelism.
  • LLaDA2.1 editing threshold τ_edit = 0.5
    Token-editing acceptance hyperparameter used in the third model’s default stack.
axioms (5)
  • domain assumption Masked diffusion forward process with independent per-position masking and linear schedule α_t=1−t, t~U[0,1], as used in MDM/BDLM objectives and Propositions 1–2.
    Standard in the cited MDM/block-diffusion literature; proofs integrate exactly under this schedule (§3.2–3.3, App. B).
  • domain assumption Block diffusion factorizes p(x_0)=∏_b p(x^b_0|x^{<b}_0) with clean prefix conditioning and diffusion only inside the current block.
    Defines BDLMs versus full-sequence MDMs (Arriola et al.; §3.2).
  • domain assumption Recent BDLMs retain substantial left-to-right conditional behavior from AR pretraining because block-diffusion adaptation uses far fewer tokens than AR pretraining.
    Stated motivational premise in §3.3 (e.g., 18T vs ~1B tokens); supported but not isolated as the sole cause.
  • domain assumption At inference, committing argmax (or nucleus) samples for all positions in the accepted unmasking set each step is a valid ancestral-style update for the trained denoiser.
    Shared by all compared samplers; PARD only changes which set is accepted (§4).
  • standard math Beta-integral identities used to obtain closed forms Pr(AR mask) = 1/(L+1) * binom(L,ℓ−1) (MDM) and 1/(L'+1)*binom(L',k) (block).
    Appendix B.1 derivation steps.
invented entities (1)
  • Parallel Autoregressive Decoding (PARD) independent evidence
    purpose: Inference rule that restricts any score-based parallel unmasking criterion to the longest leftmost contiguous accepted prefix (with leftmost fallback).
    Core methodological object; not a latent physical entity, but the paper’s main constructed procedure. Independent evidence is the external benchmark gains versus ablations and prior samplers.

reviewed 2026-07-31 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Rethinking the Generation Order of Block Diffusion Language Models." pith.science (2026). https://pith.science/paper/BR7ZRYIX

@misc{pith2026260724306,
  author       = {Pith},
  title        = {Pith review of: Rethinking the Generation Order of Block Diffusion Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BR7ZRYIX}},
  note         = {Machine review of arXiv:2607.24306}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Diffusion language models enable flexible arbitrary-order generation, but existing sampling methods are mostly designed for early masked diffusion models (MDMs). In this work, we study sampling for recent block diffusion language models (BDLMs). We show empirically and analytically that these models are naturally more aligned with left-to-right decoding than MDMs. Based on this observation, we propose Parallel Autoregressive Decoding (PARD), a simple training-free sampling method that preserves left-to-right unmasking structure while allowing parallel token commitment. Extensive experiments show that PARD consistently outperforms existing parallel samplers in generation quality, while achieving substantial speedups over pure AR decoding with only a small quality gap.

Figures

Figures reproduced from arXiv: 2607.24306 by James Kwok, Kai Syun Hou.

Figure 1
Figure 1. Figure 1: Distributions of unmasking positions at various [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 3
Figure 3. Figure 3: HumanEval Base pass@1 for various samplers on SDAR (left) and LLaDA (right). positions at each denoising step on HumanEval. A strictly autoregressive order would place all proba￾bility mass on the diagonal. As can be seen, both models exhibit a clear concentration around the di￾agonal, indicating that Confidence already induces a near-left-to-right decoding order. Within ±2 posi￾tions of the diagonal, the … view at source ↗
Figure 4
Figure 4. Figure 4: HumanEval Base pass@1 vs. decoding throughput (token/s) for Confidence (left), Margin (middle), and Entropy (right) parallel sampling and their PARD variants on Fast-dLLM v2 7B (top row) and SDAR 8B (bottom row). 100 120 140 160 180 Throughput (token/s) 52.5 55 57.5 60 62.5 65 67.5 70 pass@1 (%) Confidence (PARD) Confidence KLASS EB-Sampler APD Hierarchy 50 60 70 80 90 100 110 120 Throughput (token/s) 65 6… view at source ↗
Figure 5
Figure 5. Figure 5: HumanEval Base pass@1 vs. decoding throughput (token/s) for confidence-based PARD, EB￾Sampler, KLASS, APD, and Hierarchy on Fast-dLLM v2 7B (left) and SDAR 8B (right). 5 Experiments In this section, we first evaluate the speed-quality trade-off by varying the parallel decoding thresh￾olds. Next, we focus on the generation qualities and evaluate across various tasks and models. We evaluate three recent BDLM… view at source ↗
Figure 6
Figure 6. Figure 6: Local (left) and global (right) AR-ness@ [PITH_FULL_IMAGE:figures/full_fig_p014_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: HumanEval pass@1 vs. Number of Function Evaluations (NFE) for Confidence (left), Margin (middle), and Entropy (right) parallel sampling and their PARD variants on Fast-dLLM v2 7B (top row) and SDAR 8B (bottom row). 12000 14000 16000 18000 20000 NFE 52.5 55 57.5 60 62.5 65 67.5 70 pass@1 (%) Confidence (PARD) Confidence KLASS EB-Sampler APD Hierarchy 8000 9000 10000 11000 12000 13000 14000 15000 NFE 65 67.5… view at source ↗
Figure 8
Figure 8. Figure 8: HumanEval Base pass@1 vs. Number of Function Evaluations (NFE) for confidence-based PARD, EB-Sampler, KLASS, and APD on Fast-dLLM v2 7B (left) and SDAR 8B (right). Figures 9 and 10 show the corresponding MBPP pass@1–throughput and pass@1–NFE trade-offs for Confidence, Margin, and Entropy, along with their PARD variants. Figures 11 and 12 show the same MBPP trade-offs for confidence-based PARD, EB￾Sampler, … view at source ↗
Figure 9
Figure 9. Figure 9: MBPP pass@1 vs. decoding throughput for Confidence (left), Margin (middle), and Entropy (right) parallel sampling and their PARD variants on Fast-dLLM v2 7B (top row) and SDAR 8B (bottom row). 22.5 24.0 25.5 27.0 NFE ×10 3 60 62 64 66 68 70 pass@1 (%) Confidence Confidence (PARD) Confidence 24 26 28 30 32 NFE ×10 3 58 60 62 64 66 Margin Margin (PARD) Margin 24 25 26 27 NFE ×10 3 64 64.5 65 65.5 66 66.5 67 … view at source ↗
Figure 10
Figure 10. Figure 10: MBPP pass@1 vs. NFE for Confidence (left), Margin (middle), and Entropy (right) parallel sampling and their PARD variants on Fast-dLLM v2 7B (top row) and SDAR 8B (bottom row). 16 [PITH_FULL_IMAGE:figures/full_fig_p016_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: MBPP Base pass@1 vs. decoding throughput for confidence-based PARD, EB-Sampler, and KLASS on Fast-dLLM v2 7B (left) and SDAR 8B (right). 22000 24000 26000 28000 30000 32000 34000 36000 NFE 58 60 62 64 66 68 70 pass@1 (%) Confidence (PARD) Confidence KLASS EB-Sampler APD Hierarchy 18000 20000 22000 24000 26000 28000 30000 32000 NFE 68 70 72 74 76 78 Confidence (PARD) Confidence KLASS EB-Sampler APD Hierarc… view at source ↗
Figure 12
Figure 12. Figure 12: MBPP Base pass@1 vs. NFE for confidence-based PARD, EB-Sampler, and KLASS on Fast-dLLM v2 7B (left) and SDAR 8B (right). 17 [PITH_FULL_IMAGE:figures/full_fig_p017_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Number of tokens unmasked per step by confidence-based PARD and Confidence Parallel on [PITH_FULL_IMAGE:figures/full_fig_p018_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Generations of Confidence Parallel and PARD on [PITH_FULL_IMAGE:figures/full_fig_p019_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Tokens unmasked at the first few decoding step by Confidence Parallel and PARD for the example in [PITH_FULL_IMAGE:figures/full_fig_p020_15.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

52 extracted references · 21 linked inside Pith

  1. [1]

    Chengyue Wu and Hao Zhang and Shuchen Xue and Shizhe Diao and Yonggan Fu and Zhijian Liu and Pavlo Molchanov and Ping Luo and Song Han and Enze Xie , booktitle=. Fast-d. 2026 , url=

    2026

  2. [2]

    arXiv preprint arXiv:2510.06303 , year=

    SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation , author=. arXiv preprint arXiv:2510.06303 , year=

    arXiv

  3. [3]

    arXiv preprint arXiv:2502.09992 , year=

    Large Language Diffusion Models , author=. arXiv preprint arXiv:2502.09992 , year=

    Pith/arXiv arXiv

  4. [4]

    arXiv preprint arXiv:2508.15487 , year=

    Dream 7B: Diffusion Large Language Models , author=. arXiv preprint arXiv:2508.15487 , year=

    Pith/arXiv arXiv

  5. [5]

    Zhu, Fengqi and Wang, Rongzhen and Nie, Shen and Zhang, Xiaolu and Wu, Chunwei and others , journal=

  6. [6]

    Bie, Tiwei and Cao, Maosong and Chen, Kun and Du, Lun and Gong, Mingliang and others , journal=

  7. [7]

    Bie, Tiwei and Cao, Maosong and Cao, Xiang and Chen, Bingsen and Chen, Fuyuan and others , journal=

  8. [8]

    From Next-Token to Next-Block: A Principled Adaptation Path for Diffusion

    Tian, Yuchuan and Liang, Yuchen and Zhang, Shuo and Shu, Yingte and Yang, Guangwen and others , journal=. From Next-Token to Next-Block: A Principled Adaptation Path for Diffusion

  9. [9]

    The Fourteenth International Conference on Learning Representations , year=

    DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation , author=. The Fourteenth International Conference on Learning Representations , year=

  10. [10]

    arXiv preprint arXiv:2506.20639 , year=

    DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation , author=. arXiv preprint arXiv:2506.20639 , year=

    Pith/arXiv arXiv

  11. [11]

    arXiv preprint arXiv:2505.09388 , year=

    Qwen3 technical report , author=. arXiv preprint arXiv:2505.09388 , year=

    Pith/arXiv arXiv

  12. [12]

    arXiv preprint arXiv:2412.15115 , year=

    Qwen2.5 Technical Report , author=. arXiv preprint arXiv:2412.15115 , year=

    Pith/arXiv arXiv

  13. [13]

    arXiv preprint arXiv:2604.06832 , year=

    Fast-dVLM: Efficient Block-Diffusion VLM via Direct Conversion from Autoregressive VLM , author=. arXiv preprint arXiv:2604.06832 , year=

    Pith/arXiv arXiv

  14. [14]

    Kim, Seo Hyun and Hong, Sunwoo and Jung, Hojung and Park, Youngrok and Yun, Se-Young , journal=

  15. [15]

    arXiv preprint arXiv:2505.24857 , year=

    Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmasking , author=. arXiv preprint arXiv:2505.24857 , year=

    Pith/arXiv arXiv

  16. [16]

    International Conference on Machine Learning , pages=

    Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions , author=. International Conference on Machine Learning , pages=. 2025 , organization=

    2025

  17. [17]

    arXiv preprint arXiv:2505.22618 , year=

    Fast-dllm: Training-free acceleration of diffusion llm by enabling kv cache and parallel decoding , author=. arXiv preprint arXiv:2505.22618 , year=

    Pith/arXiv arXiv

  18. [18]

    arXiv preprint arXiv:2604.08564 , year=

    Attention-Based Sampler for Diffusion Language Models , author=. arXiv preprint arXiv:2604.08564 , year=

    Pith/arXiv arXiv

  19. [19]

    Accelerating Diffusion LLMs via Adaptive Parallel Decoding , url =

    Israel, Daniel and Van den Broeck, Guy and Grover, Aditya , booktitle =. Accelerating Diffusion LLMs via Adaptive Parallel Decoding , url =

  20. [20]

    The Fourteenth International Conference on Learning Representations , year=

    Hierarchy decoding: A training-free parallel decoding strategy for diffusion large language models , author=. The Fourteenth International Conference on Learning Representations , year=

  21. [21]

    arXiv preprint arXiv:2107.03374 , year=

    Evaluating Large Language Models Trained on Code , author=. arXiv preprint arXiv:2107.03374 , year=

    Pith/arXiv arXiv

  22. [22]

    arXiv preprint arXiv:2108.07732 , year=

    Program Synthesis with Large Language Models , author=. arXiv preprint arXiv:2108.07732 , year=

    Pith/arXiv arXiv

  23. [23]

    arXiv preprint arXiv:2110.14168 , year=

    Training Verifiers to Solve Math Word Problems , author=. arXiv preprint arXiv:2110.14168 , year=

    Pith/arXiv arXiv

  24. [24]

    Measuring Mathematical Problem Solving with the

    Hendrycks, Dan and Burns, Collin and Kadavath, Saurav and Arora, Akul and Basart, Steven and Tang, Eric and Song, Dawn and Steinhardt, Jacob , journal=. Measuring Mathematical Problem Solving with the

  25. [25]

    arXiv preprint arXiv:2311.07911 , year=

    Instruction-Following Evaluation for Large Language Models , author=. arXiv preprint arXiv:2311.07911 , year=

    Pith/arXiv arXiv

  26. [26]

    Is Your Code Generated by

    Liu, Jiawei and Xia, Chunqiu Steven and Wang, Yuyao and Zhang, Lingming , booktitle=. Is Your Code Generated by

  27. [27]

    A framework for few-shot language model evaluation , author=

  28. [28]

    arXiv preprint arXiv:2210.09261 , year=

    Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them , author=. arXiv preprint arXiv:2210.09261 , year=

    Pith/arXiv arXiv

  29. [29]

    International conference on machine learning , pages=

    Deep unsupervised learning using nonequilibrium thermodynamics , author=. International conference on machine learning , pages=. 2015 , organization=

    2015

  30. [30]

    ACM computing surveys , volume=

    Diffusion models: A comprehensive survey of methods and applications , author=. ACM computing surveys , volume=. 2023 , publisher=

    2023

  31. [31]

    arXiv preprint arXiv:2009.09761 , year=

    Diffwave: A versatile diffusion model for audio synthesis , author=. arXiv preprint arXiv:2009.09761 , year=

    Pith/arXiv arXiv 2009

  32. [32]

    Advances in neural information processing systems , volume=

    Argmax flows and multinomial diffusion: Learning categorical distributions , author=. Advances in neural information processing systems , volume=

  33. [33]

    arXiv preprint arXiv:2310.16834 , year=

    Discrete diffusion modeling by estimating the ratios of the data distribution , author=. arXiv preprint arXiv:2310.16834 , year=

    Pith/arXiv arXiv

  34. [34]

    arXiv preprint arXiv:2601.15593 , year=

    Parallelism and Generation Order in Masked Diffusion Language Models: Limits Today, Potential Tomorrow , author=. arXiv preprint arXiv:2601.15593 , year=

    Pith/arXiv arXiv

  35. [35]

    The Thirteenth International Conference on Learning Representations , year=

    Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models , author=. The Thirteenth International Conference on Learning Representations , year=

  36. [36]

    Advances in Neural Information Processing Systems , volume=

    Simple and effective masked diffusion language models , author=. Advances in Neural Information Processing Systems , volume=

  37. [37]

    Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL) , year=

    Empirical Analysis of Decoding Biases in Masked Diffusion Models , author=. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL) , year=

  38. [38]

    Advances in neural information processing systems , volume=

    Structured denoising diffusion models in discrete state-spaces , author=. Advances in neural information processing systems , volume=

  39. [39]

    International Conference on Learning Representations , volume=

    Your absorbing discrete diffusion secretly models the conditional distributions of clean data , author=. International Conference on Learning Representations , volume=

  40. [40]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Maskgit: Masked generative image transformer , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  41. [41]

    arXiv preprint arXiv:2505.16990 , year=

    Dimple: Discrete diffusion multimodal large language model with parallel decoding , author=. arXiv preprint arXiv:2505.16990 , year=

    Pith/arXiv arXiv

  42. [42]

    Advances in neural information processing systems , volume=

    Denoising diffusion probabilistic models , author=. Advances in neural information processing systems , volume=

  43. [43]

    Advances in neural information processing systems , volume=

    Simplified and generalized masked diffusion for discrete data , author=. Advances in neural information processing systems , volume=

  44. [44]

    Advances in Neural Information Processing Systems , volume=

    A continuous time framework for discrete denoising models , author=. Advances in Neural Information Processing Systems , volume=

  45. [45]

    arXiv preprint arXiv:2508.10875 , year=

    A survey on diffusion language models , author=. arXiv preprint arXiv:2508.10875 , year=

    Pith/arXiv arXiv

  46. [46]

    arXiv preprint arXiv:2508.02193 , year=

    Seed diffusion: A large-scale diffusion language model with high-speed inference , author=. arXiv preprint arXiv:2508.02193 , year=

    Pith/arXiv arXiv

  47. [47]

    A is B” fail to learn “B is A

    The Reversal Curse: LLMs trained on “A is B” fail to learn “B is A” , author=. International Conference on Learning Representations , volume=

  48. [48]

    arXiv e-prints , pages=

    Mercury: Ultra-fast language models based on diffusion , author=. arXiv e-prints , pages=

  49. [49]

    arXiv preprint arXiv:2601.15165 , year=

    The Flexibility Trap: Why Arbitrary Order Limits Reasoning Potential in Diffusion Language Models , author=. arXiv preprint arXiv:2601.15165 , year=

    Pith/arXiv arXiv

  50. [50]

    arXiv preprint arXiv:2602.23225 , year=

    Why Diffusion Language Models Struggle with Truly Parallel (Non-Autoregressive) Decoding? , author=. arXiv preprint arXiv:2602.23225 , year=

    arXiv

  51. [51]

    Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    Are Your LLMs Capable of Stable Reasoning? , author=. Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    2025

  52. [52]

    Forty-third International Conference on Machine Learning , year=

    Entropy-Aware On-Policy Distillation of Language Models , author=. Forty-third International Conference on Machine Learning , year=

This paper was first reviewed by grok-4.5 on July 31, 2026.