LightningRL: Breaking the Accuracy-Parallelism Trade-off of Block-wise dLLMs via Reinforcement Learning

Kai Yu; Pengfei Liu; Yanzhe Hu; Yijie Jin; Zhijie Deng

REVIEW 4 major objections 6 minor 6 cited by

Post-training a block-wise diffusion LLM with reinforcement learning more than doubles its parallel decoding speed without lowering task accuracy.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 02:33 UTC pith:QUQXZ645

load-bearing objection A practical RL recipe that plausibly improves the speed-accuracy frontier of block-wise dLLMs, but the TPF-aware filtering creates a train/eval gap that needs explicit measurement before the 'breaking the trade-off' headline can be taken at face value. the 4 major comments →

arxiv 2603.13319 v2 pith:QUQXZ645 submitted 2026-03-04 cs.LG

LightningRL: Breaking the Accuracy-Parallelism Trade-off of Block-wise dLLMs via Reinforcement Learning

Yanzhe Hu , Yijie Jin , Pengfei Liu , Kai Yu , Zhijie Deng This is my paper

classification cs.LG

keywords diffusion language modelsblock-wise decodingreinforcement learningparallel token generationaccuracy-parallelism trade-offmulti-objective optimizationtokens per forwardspeed-quality Pareto frontier

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that the accuracy–parallelism trade-off in block-wise diffusion language models is not fixed: it can be broken by treating parallelism as a training objective rather than an inference-time knob. LightningRL is a reinforcement-learning post-training method that rewards both correctness and the number of tokens decoded per forward step, steering the model toward trajectories that are simultaneously fast and verifier-correct. On math and coding benchmarks, the trained model keeps accuracy at the base model's level while raising average parallelism from about 3 to more than 7 tokens per forward, and a combined speed–quality score from about 189 to 498. If the claim holds, it means high-throughput parallel decoding can be made reliable through targeted training instead of accepting a quality penalty.

Core claim

The central claim is that reinforcement learning can reshape the decoding policy of a pre-trained block-wise diffusion LLM so that high-parallelism trajectories become both probable and correct. LightningRL optimizes the policy against two rewards—task accuracy and tokens per forward—using a group-relative advantage estimate, with three stabilizing modifications: each reward is normalized separately before being combined; a token-level negative-log-likelihood loss anchors the policy to verifier-correct trajectories; and a dynamic sampler keeps only prompt groups that contain at least one correct rollout and a sufficient spread of parallelism. The reported result is an average of 7.32 tokens

What carries the argument

The policy is the conditional denoising distribution over masked token positions in each block. LightningRL optimizes it with a group-relative policy-gradient objective whose advantage is the sum of independently normalized accuracy and speed rewards, so the discrete correctness signal cannot drown out the fine-grained parallelism signal. A token-level negative-log-likelihood term on correct trajectories supplies dense gradient anchoring, and a TPF-aware filter rejects prompt groups with no correct rollout or with a near-tied speed spread, keeping the policy-gradient signal dense and stable.

Load-bearing premise

The training filter keeps only prompt groups with at least one correct rollout and a spread of decoding speeds, while evaluation happens on the full unfiltered benchmark sets; if optimizing on that filtered subset does not transfer to the full distribution, the reported accuracy and speed gains would not hold.

What would settle it

Compute the discard rate of the TPF-aware filter and compare per-prompt accuracy on prompts the filter would accept versus prompts it would reject: if accuracy on rejected prompts is substantially lower than on accepted prompts, the claim that the method generalizes to the unfiltered evaluation distribution is falsified.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

The trained model keeps 87.5% accuracy even at an aggressive 7 tokens per forward, where the base model drops to about 71%, making high-parallelism decoding usable on reasoning tasks.
Decoding terminates far earlier for most samples, compressing a heavy-tail distribution that previously stretched beyond 850 steps into roughly 100 steps.
The speed–quality gains appear across model scales from 1.7B to 8B parameters, suggesting the method transfers with model capacity.
Because parallelism is now a first-class training objective, the method offers a path to lower inference cost per token without sacrificing benchmark accuracy.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Editorial inference: if the filtered-training transfer holds, the same recipe should work on other verifier-rich domains, such as theorem proving or automated code repair, where correctness can be checked automatically.
Editorial inference: the TPF-spread threshold is fixed at 0.01; annealing it over training could be a testable way to push parallelism further than the reported average of 7.32.
Editorial inference: the paper does not report the prompt discard rate from its accept–reject sampling; measuring that rate and the accuracy gap between accepted and rejected prompts would let readers judge how much of the gain comes from curriculum selection versus policy learning.
Editorial inference: the token-level NLL anchor weights all correct tokens equally; weighting by rollout length or by confidence in the verifier signal is an unexplored variant that could sharpen or overcommit the anchoring.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

A practical RL recipe that plausibly improves the speed-accuracy frontier of block-wise dLLMs, but the TPF-aware filtering creates a train/eval gap that needs explicit measurement before the 'breaking the trade-off' headline can be taken at face value.

read the letter

What you should know: this is a genuine empirical advance. The authors take SDAR, a block-wise dLLM, and apply GRPO with three modifications: per-reward decoupled normalization, NLL anchoring on correct trajectories, and TPF-aware filtering. They report average TPF 7.32 vs 3.12 for the SDAR baseline at essentially equal accuracy across GSM8K, MATH500, MBPP, and HumanEval. If those numbers hold, it is a real practical win for dLLM deployment.

The three pieces are not individually novel — decoupled normalization is a standard multi-objective trick, NLL anchoring is self-imitation, and TPF-aware filtering is a form of reward-based sample selection — but the combination is new and the ablations show each component contributes. The paper also ships code, which deserves credit. The training curves and step-wise decoding analysis support the internal story.

The main soft spot is the TPF-aware filtering itself. Equation 11 and Algorithm 1 reject any prompt-group where no rollout is correct or where the within-group TPF spread is below 0.01. That means the policy is trained only on prompts for which the current behavior can already produce a correct answer. Evaluation, though, is on the full unfiltered benchmarks. If hard prompts are disproportionately rejected, the reported average gains could come from easy prompts while the model silently degrades on harder ones. The paper does not report the rejection rate or any difficulty-stratified results. That is a load-bearing gap, because the claim is about the full distribution, not the filtered one. It is fixable, but it must be fixed.

Second, there is an internal contradiction: Section 3.1 says they deliberately do not normalize the accuracy reward, but Algorithm 1 normalizes both accuracy and TPF rewards with group statistics. This needs reconciliation before the method is reproducible as written.

Third, the main results appear to be selected at the best epoch on AUP (Table 3's caption says so for the ablation, and the main tables presumably follow the same protocol), and there are no error bars or multiple seeds. For an RL paper with this much variance, that is a real limitation.

On circularity: TPF is part of the training reward, so higher TPF is expected, but the AUP gain is not circular — it comes from keeping accuracy while raising TPF. The overclaim is 'breaking' the trade-off; the evidence supports 'improving' the frontier.

Who this is for: people working on dLLM inference acceleration or RL for non-autoregressive LLMs. The paper deserves a serious referee: the method is clearly described, the ablation study is honest, and the results are impressive if they hold. Recommend peer review, with the requirement that the authors report the filtering discard rate, evaluate on hard subsets, fix the normalization contradiction, and provide error bars.

Referee Report

4 major / 6 minor

Summary. The paper introduces LightningRL, a GRPO-based reinforcement-learning post-training framework for block-wise diffusion LLMs (dLLMs), instantiated on the SDAR architecture. The method optimizes a multi-objective reward combining task accuracy and tokens-per-forward (TPF), with three declared components: per-reward decoupled normalization, a token-level NLL regularizer on verifier-correct trajectories, and dynamic sampling with TPF-aware filtering. Experiments on GSM8K, MATH500, MBPP, and HumanEval report that LightningRL-8B-b32 attains an average TPF of 7.32 and an average AUP of 497.9 at 71.1% accuracy, versus SDAR-8B-b32's 3.12 TPF and 189.2 AUP at 71.0% accuracy. Ablations on GSM8K are provided for each component.

Significance. If the reported results are robust, the paper makes a practically useful contribution: it demonstrates that RL post-training can improve the speed-accuracy trade-off of a block-wise dLLM, moving beyond inference-only acceleration. The paper's strengths include released code, a clear experimental comparison across four benchmarks, wall-clock TPS measurements in the appendix, and ablations isolating the proposed components. However, the central empirical claim is not yet established because the training distribution is altered by TPF-aware filtering (Sec. 3.3) and the evaluation is performed on the full unfiltered benchmarks; the paper neither quantifies the rejection rate nor demonstrates transfer to the discarded, typically harder, prompts. In addition, the main results have no error bars or seed variance, Table 3 selects checkpoints by AUP, and the description of reward normalization is internally inconsistent. These issues must be resolved before the headline accuracy-parallelism claim can be accepted.

major comments (4)

[Sec. 3.3 (Eq. 11) and Algorithm 1] The dynamic-sampling filter rejects every prompt-group with no verifier-correct rollout or with TPF spread below δ=0.01. Training therefore optimizes an expected reward over the conditional distribution of prompts on which the current policy already produces a correct rollout; the hardest prompts are never updated. Evaluation, however, is on the full unfiltered GSM8K/MATH500/MBPP/HumanEval sets, yet no rejection rate, no difficulty-stratified results, and no comparison to a variant trained on all groups are reported. The central claim that LightningRL improves TPF at equal accuracy on the full distribution is unsupported: gains may be concentrated on easy, high-TPF prompts while hard prompts degrade. Please report acceptance rates, per-difficulty-decile accuracy/TPF, and evaluate at least one control without filtering (or provide an explicit transfer analysis).
[Sec. 3.1 vs Algorithm 1 (lines 25–28)] The text states 'we do not normalize the accuracy reward to prevent reward shifts' and then says the composite advantage is derived by 'summing these normalized components.' But Eq. (5) and Algorithm 1 normalize the accuracy reward exactly like every other component (A_acc ← (r_acc−μ_acc)/(σ_acc+ε)). This is an internal contradiction in the specification of the first contribution. Please state exactly which rewards are normalized; if accuracy is not normalized, update Eq. (5) and Algorithm 1 and verify that the ablation in Table 3 still holds.
[Sec. 4.6, Table 3; Sec. 4.2] Table 3 reports results 'at the highest AUP during 20 training epochs,' i.e., model selection on an evaluation metric. The main results in Tables 1–2 and Sec. 4.5 do not state their checkpoint-selection rule, and no standard errors or multiple-seed results appear anywhere. Because the method's objective directly includes TPF, checkpoint selection on AUP can inflate reported gains. Please specify the checkpoint-selection protocol for all tables and report mean±std over at least three seeds, or justify why this is infeasible.
[Sec. 3.2 (Eq. 9) and Sec. 4.6] The NLL anchor is defined as a token-level denoising likelihood, yet Sec. 4.6 concedes, citing Ou et al. (2025), that diffusion LLMs lack tractable token-level conditional likelihoods and that token-level losses are heuristic proxies with known bias. Since this component is one of the three stated contributions and its removal causes a 9.6-point accuracy drop in Table 3, the paper should specify the exact proxy used in Eq. (9), discuss the bias it introduces, and provide evidence that the anchoring effect is robust to the choice of NLL weight μ.

minor comments (6)

[Sec. 4.5] The figure reference is empty: 'An illustration of the comparison ... is provided in Fig. .' Please insert the correct figure number.
[Table 2] The row 'd3LLM-LLaDA (Chen et al., 2025)' appears to be mis-cited; the text and reference list attribute d3LLM to Qian et al. (2026).
[Sec. 4.1] AUP is a headline metric but is never defined in the paper. Provide the formula or a precise definition, rather than only citing Qian et al. (2026).
[Sec. 3.1 / Fig. 3a] The 'Collapse Ratio' is defined as the proportion of within-group pairs with advantage differences below ε, but ε is already used as a numerical stabilizer in Eqs. (5) and (7). Please disambiguate the two uses.
[Sec. 4.1] Evaluation settings are under-specified. The text mentions 4-shot MATH and 3-shot for LLaDA-based models on MBPP and says 'all other evaluations' are zero-shot, but it should explicitly state the shot setting for SDAR and LightningRL on each benchmark.
[Appendix A] The claim that value-model approximation errors 'can dominate and flip the sign of Â_t' is plausible but entirely unquantified. Either support it with diagnostics or remove it.

Circularity Check

0 steps flagged

No significant circularity: LightningRL is an empirical RL method whose reported gains are evaluation outcomes, not quantities equivalent to its inputs by construction.

full rationale

The paper makes no first-principles prediction or derivation that reduces to its inputs. Its central claim is an empirical one: after RL post-training with a multi-objective reward containing accuracy and TPF signals, the model achieves higher TPF at comparable accuracy on held-out benchmarks. That TPF appears in the reward does not make the reported TPF increase circular, because the paper does not present the increase as a predicted consequence of the reward definition; it is a measured outcome of optimization, evaluated on GSM8K, MATH500, MBPP, and HumanEval, and compared against external baselines such as EAGLE-3, Fast-dLLM-v2, and Qwen-2.5. The AUP metric is attributed to Qian et al. (2026), which shares an author with this work, but AUP is a simple summary computed from the separately reported accuracy and TPF numbers and is not used to derive any result. The TPF-aware filtering in Sec. 3.3 (Eq. 11) is a training-distribution selection rule: it discards groups that are all-wrong or have near-tied TPF. This is a potential external-validity or transfer concern, since the paper does not report the discard rate or a difficulty-stratified evaluation, but it is not circularity: no reported evaluation number is defined in terms of the filter, and the final evaluation is on the full unfiltered benchmarks. The ablations (Tab. 3) provide independent evidence that each component contributes to the held-out result. There is no self-citation chain invoked to forbid alternatives and no uniqueness theorem imported from the authors' prior work. The paper is self-contained as an empirical systems contribution, so the appropriate circularity score is 0.

Axiom & Free-Parameter Ledger

6 free parameters · 4 axioms · 0 invented entities

The central contribution is empirical; the method relies on standard RL assumptions and hand-chosen hyperparameters rather than new theoretical entities.

free parameters (6)

confidence threshold φ = 0.9
Used for low-confidence dynamic remasking (B.1); hand-chosen threshold controlling how many tokens can be accepted per forward.
TPF spread filter threshold δ = 0.01
Eq 11; groups with max-min TPF below δ are discarded during training; hand-chosen.
KL penalty coefficient β = 0.01
B.1; weight of KL divergence against reference in Eq 10.
NLL loss weight μ = 0.1
B.1; weight of token-level NLL on correct trajectories in Eq 10.
group size G = 32
Number of rollouts per prompt in GRPO (Sec 4.1).
batch size = 128
Tasks per training iteration (Sec 4.1).

axioms (4)

domain assumption GRPO advantage estimator (Eq 2) provides a valid policy-gradient signal for dLLM denoising transitions
The paper maps the policy to the denoising distribution and assumes the score-function update (Eq 8) is unbiased despite the non-autoregressive masked-token structure.
domain assumption Accuracy reward (binary answer-match / unit-test pass fraction) is a faithful proxy for generation quality
Math verifies via answer equivalence, code via test pass proportion; both can be gamed or incomplete.
domain assumption TPF is a valid speed proxy and AUP = Acc×TPF summarizes the speed-quality frontier
AUP from Qian et al. 2026 is treated as the objective metric; TPF does not account for per-step compute cost.
ad hoc to paper Token-level NLL on correct trajectories is a valid anchoring regularizer despite the acknowledged mismatch between AR likelihoods and diffusion processes
The paper cites Ou et al. 2025 for the mismatch and still uses token-level NLL as a heuristic (Sec 3.2, Eq 9).

pith-pipeline@v1.3.0-alltime-deepseek · 16953 in / 12092 out tokens · 113060 ms · 2026-08-03T02:33:02.196932+00:00 · methodology

0 comments

read the original abstract

Diffusion Large Language Models (dLLMs) have emerged as a promising paradigm for parallel token generation, with block-wise variants garnering significant research interest. Despite their potential, existing dLLMs typically suffer from a rigid accuracy-parallelism trade-off: increasing the number of tokens per forward (TPF) via aggressive parallel decoding often leads to performance degradation and increased generation instability. We identify that this limitation stems from the model's inability to navigate high-parallelism regimes where approximation errors and local corruptions accumulate, ultimately undermining the reliability of parallel generation. To address this, we propose LightningRL, a post-training framework designed to directly optimize the speed-quality Pareto frontier of pre-trained dLLMs. Instead of forcing uniform parallelization, our approach leverages reinforcement learning to identify and reinforce high-parallelism trajectories that maintain generation accuracy. Built upon the Group Relative Policy Optimization (GRPO) framework, LightningRL introduces several enhancements tailored for dLLMs: (1) stabilized training via per-reward decoupled normalization; (2) token-level negative log-likelihood (NLL) regularization on correct trajectories to anchor model performance; and (3) a dynamic sampling strategy with TPF-aware filtering to enhance training efficiency. Experimental results across mathematical and coding benchmarks demonstrate that LightningRL consistently advances the Pareto frontier, achieving competitive task accuracy while significantly increasing parallelism, reaching an average TPF of 7.32 (with a peak of 11.10 on the MBPP dataset). Our code is available at https://github.com/SJTU-DENG-Lab/LightningRL.

Figures

Figures reproduced from arXiv: 2603.13319 by Kai Yu, Pengfei Liu, Yanzhe Hu, Yijie Jin, Zhijie Deng.

**Figure 1.** Figure 1: Evaluation results of LightningRL and the baselines. LightningRL achieves superior accuracy on math and code benchmarks while accelerating parallel decoding to an average of 7.32 tokens per forward (TPF) and 497.9 accuracy under parallelism (AUP) (Qian et al., 2026), significantly outperforming baselines including Eagle-3 (Li et al., 2025) and Fast-dLLM-v2 (Wu et al., 2025a). Abstract Diffusion Large Langu… view at source ↗

**Figure 2.** Figure 2: Overview of LightningRL. LightningRL samples a group of decoding trajectories per prompt, applies per-reward decoupled normalization to preserve within-group ranking under heterogeneous scales. The policy is optimized with a GRPO-style objective plus a token-level NLL anchor. The bottom panel shows the resulting shift toward the fastest correct trajectory, improving TPF without degrading accuracy. Specific… view at source ↗

**Figure 3.** Figure 3: Per-reward decoupled normalization improves training stability. It reduces signal collapse (a) and yields more stable reward optimization (b) under the same training setup. illustrated in [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

**Figure 4.** Figure 4: Token-level NLL loss anchors the accuracy objective on GSM8K. Compared with training without the token-level NLL term, it maintains a higher accuracy reward and mitigates latestage drift in the accuracy signal under the same setup. 0 30 60 90 120 150 180 Step 0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 Total Reward Total Reward over Training Steps (GSM8K) w/o Dynamic Sample w/ Dynamic Sample [PITH_FULL_IMAGE… view at source ↗

**Figure 6.** Figure 6: Total Reward curves over training steps. LightningRL outperforms baseline methods by suggesting a more reliable optimization landscape. 0 100 200 300 400 500 600 700 800 900 Decoding Step 0 5 10 15 20 25 30 35 Avg. Parallel Token Count Avg. Parallel Token Count over Decoding Steps (GSM8K) SDAR-8B-b32 (smooth) LightningRL-8B-b32 (smooth) SDAR-8B-b32 Incomplete Samples LightningRL-8B-b32 Incomplete Samples … view at source ↗

**Figure 8.** Figure 8: Training dynamics and Accuracy-Parallism Trad-off on GSM8K. LightningRL avoids the objective drift and reward collapse observed in TraceRL, while sustaining higher decoding throughput and more synchronized termination. C.4. Accuracy–Parallelism Trade-off We further investigate the inherent trade-off between decoding parallelism and reasoning accuracy on the GSM8K benchmark. In non-autoregressive or draftin… view at source ↗

discussion (0)

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

TAD: Temporal-Aware Trajectory Self-Distillation for Fast and Accurate Diffusion LLM
cs.CL 2026-05 unverdicted novelty 7.0

TAD improves the accuracy-parallelism trade-off in diffusion LLMs via temporal-aware self-distillation that applies hard labels to soon-to-be-decoded tokens and soft supervision to future tokens.
DMax: Aggressive Parallel Decoding for dLLMs
cs.LG 2026-04 conditional novelty 7.0

DMax uses On-Policy Uniform Training and Soft Parallel Decoding to enable aggressive parallelism in dLLMs, raising TPF on GSM8K from 2.04 to 5.47 and on MBPP from 2.71 to 5.86 while preserving accuracy.
SLIM-RL: Risk-Budgeted Random-Masking RL for Diffusion LLMs Without Trajectory Slicing
cs.CL 2026-06 unverdicted novelty 6.0

SLIM-RL matches or exceeds TraceRL performance on MATH500, GSM8K, MBPP and HumanEval for diffusion LLMs by risk-budgeted random-masking RL without trajectory slicing.
Multi-Block Diffusion Language Models
cs.LG 2026-06 unverdicted novelty 6.0

MBD-LMs post-train BD-LMs using MultiTF on bounded noise-groups with randomized schedulers and Block Buffer decoding to increase average TPF from 3.47 to 6.19 with accuracy rising to 81.03%.
Multi-Block Diffusion Language Models
cs.LG 2026-06 unverdicted novelty 6.0

MBD-LMs raise average tokens per forward pass from 3.47 to 6.19 (and to 9.34 with DMax) via multi-block teacher forcing and optimized parallel decoding while holding or slightly improving accuracy on math and code tasks.
DMax: Aggressive Parallel Decoding for dLLMs
cs.LG 2026-04 unverdicted novelty 5.0

DMax enables faster parallel decoding in diffusion language models by using on-policy training to recover from errors and soft embedding interpolations for iterative revision, boosting tokens per forward pass roughly ...

Reference graph

Works this paper leans on

22 extracted references · 10 linked inside Pith · cited by 4 Pith papers

[2]

Chen, M., Tworek, J., Jun, H., Yuan, Q., de Oliveira Pinto, H

URL https: //arxiv.org/abs/2509.25188. Chen, M., Tworek, J., Jun, H., Yuan, Q., de Oliveira Pinto, H. P., Kaplan, J., Edwards, H., Burda, Y ., Joseph, N., Brockman, G., Ray, A., Puri, R., Krueger, G., Petrov, M., Khlaaf, H., Sastry, G., Mishkin, P., Chan, B., Gray, S., Ryder, N., Pavlov, M., Power, A., Kaiser, L., Bavar- ian, M., Winter, C., Tillet, P., S...

arXiv
[4]

URL https: //arxiv.org/abs/2510.06303. 9 LightningRL: Breaking the Accuracy–Parallelism Trade-off of Block-wise dLLMs via Reinforcement Learning Cobbe, K., Kosaraju, V ., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., Hesse, C., and Schulman, J. Training verifiers to solve math word problems.arXiv preprint a...

arXiv
[5]

Gulrajani, I

URL https://arxiv.org/abs/2506.20639. Gulrajani, I. and Hashimoto, T. B. Likelihood-based diffu- sion language models,

Pith/arXiv arXiv
[7]

Hendrycks, D., Burns, C., Kadavath, S., Arora, A., Basart, S., Tang, E., Song, D., and Steinhardt, J

URL https://arxiv.org/ abs/2508.13148. Hendrycks, D., Burns, C., Kadavath, S., Arora, A., Basart, S., Tang, E., Song, D., and Steinhardt, J. Measuring math- ematical problem solving with the math dataset.arXiv preprint arXiv:2103.03874,

arXiv
[8]

Kou, S., Hu, L., He, Z., Deng, Z., and Zhang, H

URL https://arxiv.org/ abs/2505.10446. Kou, S., Hu, L., He, Z., Deng, Z., and Zhang, H. Cllms: Consistency large language models. InForty-first Inter- national Conference on Machine Learning,

arXiv
[10]

Ma, Y ., Melnychuk, V ., Schweisthal, J., and Feuerriegel, S

URL https://arxiv.org/ abs/2503.01840. Ma, Y ., Melnychuk, V ., Schweisthal, J., and Feuerriegel, S. Diffpo: A causal diffusion model for learning dis- tributions of potential outcomes,

Pith/arXiv arXiv
[12]

Ou, J., Han, J., Xu, M., Xu, S., Xie, J., Ermon, S., Wu, Y ., and Li, C

URL https://arxiv.org/ abs/2502.09992. Ou, J., Han, J., Xu, M., Xu, S., Xie, J., Ermon, S., Wu, Y ., and Li, C. Principled rl for diffusion llms emerges from a sequence-level perspective,

Pith/arXiv arXiv
[13]

Qian, Y .-Y ., Su, J., Hu, L., Zhang, P., Deng, Z., Zhao, P., and Zhang, H

URL https: //arxiv.org/abs/2512.03759. Qian, Y .-Y ., Su, J., Hu, L., Zhang, P., Deng, Z., Zhao, P., and Zhang, H. d3llm: Ultra-fast diffusion llm using pseudo- trajectory distillation,

arXiv
[15]

Sahoo, S

URL https: //arxiv.org/abs/2412.15115. Sahoo, S. S., Arriola, M., Schiff, Y ., Gokaslan, A., Marro- quin, E., Chiu, J. T., Rush, A., and Kuleshov, V . Simple and effective masked diffusion language models, 2024a. URLhttps://arxiv.org/abs/2406.07524. Sahoo, S. S., Arriola, M., Schiff, Y ., Gokaslan, A., Marro- quin, E., Chiu, J. T., Rush, A., and Kuleshov,...

Pith/arXiv arXiv
[16]

URL https://arxiv.org/abs/2402.03300. Team, P. I., Senghaas, M., Obeid, F., Jaghouar, S., Brown, W., Ong, J. M., Auras, D., Sirovatka, M., Straube, J., Baker, A., M ¨uller, S., Mattern, J., Basra, M., Ismail, A., Scherm, D., Miller, C., Patel, A., Kirsten, S., Sieg, M., Reetz, C., Erdem, K., Weisser, V ., and Hagemann, J. Intellect-3: Technical report,

Pith/arXiv arXiv
[17]

Wang, X., Xu, C., Jin, Y ., Jin, J., Zhang, H., and Deng, Z

URL https: //arxiv.org/abs/2512.16144. Wang, X., Xu, C., Jin, Y ., Jin, J., Zhang, H., and Deng, Z. Diffusion llms can do faster-than-ar inference via dis- crete diffusion forcing, 2025a. URL https://arxiv. org/abs/2508.09192. Wang, Y ., Yang, L., Li, B., Tian, Y ., Shen, K., and Wang, M. Revolutionizing reinforcement learning framework for diffusion larg...

arXiv
[18]

Ye, J., Xie, Z., Zheng, L., Gao, J., Wu, Z., Jiang, X., Li, Z., and Kong, L

URL https://arxiv.org/abs/2512.16229. Ye, J., Xie, Z., Zheng, L., Gao, J., Wu, Z., Jiang, X., Li, Z., and Kong, L. Dream 7b: Diffusion large language mod- els,

arXiv
[19]

org/abs/2504.12216

URL https://arxiv. org/abs/2504.12216. Zhou, R., Ni, Z., Chen, T., Liu, Z., Yue, Y ., Wang, Y ., Wang, Y ., Liu, J., and Huang, G. Co-grpo: Co-optimized group relative policy optimization for masked diffu- sion model,

Pith/arXiv arXiv
[20]

Zhu, Y ., Wan, J., Liu, X., He, S., Wang, Q., Guo, X., Liang, T., Huang, Z., He, Z., and Qiu, X

URL https://arxiv.org/abs/ 2512.22288. Zhu, Y ., Wan, J., Liu, X., He, S., Wang, Q., Guo, X., Liang, T., Huang, Z., He, Z., and Qiu, X. Dirl: An efficient post- training framework for diffusion language models,

arXiv
[21]

11 LightningRL: Breaking the Accuracy–Parallelism Trade-off of Block-wise dLLMs via Reinforcement Learning A

URLhttps://arxiv.org/abs/2512.22234. 11 LightningRL: Breaking the Accuracy–Parallelism Trade-off of Block-wise dLLMs via Reinforcement Learning A. Discussion on Value Model Incorporation Method Acc(%)TPF AUP w/ Value Model 80.7 5.14 408.1 LightningRL 90.3 5.58 492.4 Table 5.Comparison of Performance with and without Value Model Incorporation on GSM8K. To ...

arXiv
[22]

Notably, LightningRL achieves exceptional inference speed, significantly outperforming the baselines

Since prior works are not yet natively supported by SGLang, their reported results from the original publication are cited for comparison (Qian et al., 2026). Notably, LightningRL achieves exceptional inference speed, significantly outperforming the baselines. 14 LightningRL: Breaking the Accuracy–Parallelism Trade-off of Block-wise dLLMs via Reinforcemen...

2026
[2021]

dparallel: Learnable parallel decoding for dllms.arXiv preprint arXiv:2509.26488,

Chen, Z., Fang, G., Ma, X., Yu, R., and Wang, X. dparallel: Learnable parallel decoding for dllms.arXiv preprint arXiv:2509.26488,

arXiv
[2022]

Li, Y ., Wei, F., Zhang, C., and Zhang, H

URL https://arxiv.org/ abs/2205.14217. Li, Y ., Wei, F., Zhang, C., and Zhang, H. Eagle-3: Scaling up inference acceleration of large language models via training-time test,

Pith/arXiv arXiv
[2023]

org/abs/2305.18619

URL https://arxiv. org/abs/2305.18619. He, H., Renz, K., Cao, Y ., and Geiger, A. Mdpo: Over- coming the training-inference divide of masked diffusion language models,

Pith/arXiv arXiv
[2024]

Nie, S., Zhu, F., You, Z., Zhang, X., Ou, J., Hu, J., Zhou, J., Lin, Y ., Wen, J.-R., and Li, C

URL https: //arxiv.org/abs/2410.08924. Nie, S., Zhu, F., You, Z., Zhang, X., Ou, J., Hu, J., Zhou, J., Lin, Y ., Wen, J.-R., and Li, C. Large language dif- fusion models,

Pith/arXiv arXiv
[2025]

Austin, J., Odena, A., Nye, M., Bosma, M., Michalewski, H., Dohan, D., Jiang, E., Cai, C., Terry, M., Le, Q., et al

URL https://arxiv.org/ abs/2503.09573. Austin, J., Odena, A., Nye, M., Bosma, M., Michalewski, H., Dohan, D., Jiang, E., Cai, C., Terry, M., Le, Q., et al. Program synthesis with large language models.arXiv preprint arXiv:2108.07732,

Pith/arXiv arXiv
[2026]

org/abs/2601.07568

URL https://arxiv. org/abs/2601.07568. Qwen, :, Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., Huang, F., Wei, H., Lin, H., Yang, J., Tu, J., Zhang, J., Yang, J., Yang, J., Zhou, J., Lin, J., Dang, K., Lu, K., Bao, K., Yang, K., Yu, L., Li, M., Xue, M., Zhang, P., Zhu, Q., Men, R., Lin, R., Li, T., Tang, T., Xia, T., Ren, X., ...

arXiv

[1] [2]

Chen, M., Tworek, J., Jun, H., Yuan, Q., de Oliveira Pinto, H

URL https: //arxiv.org/abs/2509.25188. Chen, M., Tworek, J., Jun, H., Yuan, Q., de Oliveira Pinto, H. P., Kaplan, J., Edwards, H., Burda, Y ., Joseph, N., Brockman, G., Ray, A., Puri, R., Krueger, G., Petrov, M., Khlaaf, H., Sastry, G., Mishkin, P., Chan, B., Gray, S., Ryder, N., Pavlov, M., Power, A., Kaiser, L., Bavar- ian, M., Winter, C., Tillet, P., S...

arXiv

[2] [4]

URL https: //arxiv.org/abs/2510.06303. 9 LightningRL: Breaking the Accuracy–Parallelism Trade-off of Block-wise dLLMs via Reinforcement Learning Cobbe, K., Kosaraju, V ., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., Hesse, C., and Schulman, J. Training verifiers to solve math word problems.arXiv preprint a...

arXiv

[3] [5]

Gulrajani, I

URL https://arxiv.org/abs/2506.20639. Gulrajani, I. and Hashimoto, T. B. Likelihood-based diffu- sion language models,

Pith/arXiv arXiv

[4] [7]

Hendrycks, D., Burns, C., Kadavath, S., Arora, A., Basart, S., Tang, E., Song, D., and Steinhardt, J

URL https://arxiv.org/ abs/2508.13148. Hendrycks, D., Burns, C., Kadavath, S., Arora, A., Basart, S., Tang, E., Song, D., and Steinhardt, J. Measuring math- ematical problem solving with the math dataset.arXiv preprint arXiv:2103.03874,

arXiv

[5] [8]

Kou, S., Hu, L., He, Z., Deng, Z., and Zhang, H

URL https://arxiv.org/ abs/2505.10446. Kou, S., Hu, L., He, Z., Deng, Z., and Zhang, H. Cllms: Consistency large language models. InForty-first Inter- national Conference on Machine Learning,

arXiv

[6] [10]

Ma, Y ., Melnychuk, V ., Schweisthal, J., and Feuerriegel, S

URL https://arxiv.org/ abs/2503.01840. Ma, Y ., Melnychuk, V ., Schweisthal, J., and Feuerriegel, S. Diffpo: A causal diffusion model for learning dis- tributions of potential outcomes,

Pith/arXiv arXiv

[7] [12]

Ou, J., Han, J., Xu, M., Xu, S., Xie, J., Ermon, S., Wu, Y ., and Li, C

URL https://arxiv.org/ abs/2502.09992. Ou, J., Han, J., Xu, M., Xu, S., Xie, J., Ermon, S., Wu, Y ., and Li, C. Principled rl for diffusion llms emerges from a sequence-level perspective,

Pith/arXiv arXiv

[8] [13]

Qian, Y .-Y ., Su, J., Hu, L., Zhang, P., Deng, Z., Zhao, P., and Zhang, H

URL https: //arxiv.org/abs/2512.03759. Qian, Y .-Y ., Su, J., Hu, L., Zhang, P., Deng, Z., Zhao, P., and Zhang, H. d3llm: Ultra-fast diffusion llm using pseudo- trajectory distillation,

arXiv

[9] [15]

Sahoo, S

URL https: //arxiv.org/abs/2412.15115. Sahoo, S. S., Arriola, M., Schiff, Y ., Gokaslan, A., Marro- quin, E., Chiu, J. T., Rush, A., and Kuleshov, V . Simple and effective masked diffusion language models, 2024a. URLhttps://arxiv.org/abs/2406.07524. Sahoo, S. S., Arriola, M., Schiff, Y ., Gokaslan, A., Marro- quin, E., Chiu, J. T., Rush, A., and Kuleshov,...

Pith/arXiv arXiv

[10] [16]

URL https://arxiv.org/abs/2402.03300. Team, P. I., Senghaas, M., Obeid, F., Jaghouar, S., Brown, W., Ong, J. M., Auras, D., Sirovatka, M., Straube, J., Baker, A., M ¨uller, S., Mattern, J., Basra, M., Ismail, A., Scherm, D., Miller, C., Patel, A., Kirsten, S., Sieg, M., Reetz, C., Erdem, K., Weisser, V ., and Hagemann, J. Intellect-3: Technical report,

Pith/arXiv arXiv

[11] [17]

Wang, X., Xu, C., Jin, Y ., Jin, J., Zhang, H., and Deng, Z

URL https: //arxiv.org/abs/2512.16144. Wang, X., Xu, C., Jin, Y ., Jin, J., Zhang, H., and Deng, Z. Diffusion llms can do faster-than-ar inference via dis- crete diffusion forcing, 2025a. URL https://arxiv. org/abs/2508.09192. Wang, Y ., Yang, L., Li, B., Tian, Y ., Shen, K., and Wang, M. Revolutionizing reinforcement learning framework for diffusion larg...

arXiv

[12] [18]

Ye, J., Xie, Z., Zheng, L., Gao, J., Wu, Z., Jiang, X., Li, Z., and Kong, L

URL https://arxiv.org/abs/2512.16229. Ye, J., Xie, Z., Zheng, L., Gao, J., Wu, Z., Jiang, X., Li, Z., and Kong, L. Dream 7b: Diffusion large language mod- els,

arXiv

[13] [19]

org/abs/2504.12216

URL https://arxiv. org/abs/2504.12216. Zhou, R., Ni, Z., Chen, T., Liu, Z., Yue, Y ., Wang, Y ., Wang, Y ., Liu, J., and Huang, G. Co-grpo: Co-optimized group relative policy optimization for masked diffu- sion model,

Pith/arXiv arXiv

[14] [20]

Zhu, Y ., Wan, J., Liu, X., He, S., Wang, Q., Guo, X., Liang, T., Huang, Z., He, Z., and Qiu, X

URL https://arxiv.org/abs/ 2512.22288. Zhu, Y ., Wan, J., Liu, X., He, S., Wang, Q., Guo, X., Liang, T., Huang, Z., He, Z., and Qiu, X. Dirl: An efficient post- training framework for diffusion language models,

arXiv

[15] [21]

11 LightningRL: Breaking the Accuracy–Parallelism Trade-off of Block-wise dLLMs via Reinforcement Learning A

URLhttps://arxiv.org/abs/2512.22234. 11 LightningRL: Breaking the Accuracy–Parallelism Trade-off of Block-wise dLLMs via Reinforcement Learning A. Discussion on Value Model Incorporation Method Acc(%)TPF AUP w/ Value Model 80.7 5.14 408.1 LightningRL 90.3 5.58 492.4 Table 5.Comparison of Performance with and without Value Model Incorporation on GSM8K. To ...

arXiv

[16] [22]

Notably, LightningRL achieves exceptional inference speed, significantly outperforming the baselines

Since prior works are not yet natively supported by SGLang, their reported results from the original publication are cited for comparison (Qian et al., 2026). Notably, LightningRL achieves exceptional inference speed, significantly outperforming the baselines. 14 LightningRL: Breaking the Accuracy–Parallelism Trade-off of Block-wise dLLMs via Reinforcemen...

2026

[17] [2021]

dparallel: Learnable parallel decoding for dllms.arXiv preprint arXiv:2509.26488,

Chen, Z., Fang, G., Ma, X., Yu, R., and Wang, X. dparallel: Learnable parallel decoding for dllms.arXiv preprint arXiv:2509.26488,

arXiv

[18] [2022]

Li, Y ., Wei, F., Zhang, C., and Zhang, H

URL https://arxiv.org/ abs/2205.14217. Li, Y ., Wei, F., Zhang, C., and Zhang, H. Eagle-3: Scaling up inference acceleration of large language models via training-time test,

Pith/arXiv arXiv

[19] [2023]

org/abs/2305.18619

URL https://arxiv. org/abs/2305.18619. He, H., Renz, K., Cao, Y ., and Geiger, A. Mdpo: Over- coming the training-inference divide of masked diffusion language models,

Pith/arXiv arXiv

[20] [2024]

Nie, S., Zhu, F., You, Z., Zhang, X., Ou, J., Hu, J., Zhou, J., Lin, Y ., Wen, J.-R., and Li, C

URL https: //arxiv.org/abs/2410.08924. Nie, S., Zhu, F., You, Z., Zhang, X., Ou, J., Hu, J., Zhou, J., Lin, Y ., Wen, J.-R., and Li, C. Large language dif- fusion models,

Pith/arXiv arXiv

[21] [2025]

Austin, J., Odena, A., Nye, M., Bosma, M., Michalewski, H., Dohan, D., Jiang, E., Cai, C., Terry, M., Le, Q., et al

URL https://arxiv.org/ abs/2503.09573. Austin, J., Odena, A., Nye, M., Bosma, M., Michalewski, H., Dohan, D., Jiang, E., Cai, C., Terry, M., Le, Q., et al. Program synthesis with large language models.arXiv preprint arXiv:2108.07732,

Pith/arXiv arXiv

[22] [2026]

org/abs/2601.07568

URL https://arxiv. org/abs/2601.07568. Qwen, :, Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., Huang, F., Wei, H., Lin, H., Yang, J., Tu, J., Zhang, J., Yang, J., Yang, J., Zhou, J., Lin, J., Dang, K., Lu, K., Bao, K., Yang, K., Yu, L., Li, M., Xue, M., Zhang, P., Zhu, Q., Men, R., Lin, R., Li, T., Tang, T., Xia, T., Ren, X., ...

arXiv