Pith. sign in

REVIEW 4 major objections 4 minor 2 cited by

AGRPO adapts GRPO to diffusion language models by rewriting the token-level objective as an expectation over denoising steps, then estimating it with Monte Carlo sampling — a move that claims to make policy-gradient RL both tractable and un

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 11:32 UTC pith:NA4RM74X

load-bearing objection AGRPO's timestep-MC idea is promising and the empirical gains look real, but the paper's central unbiasedness claim is underspecified and the abstract numbers don't match the text — fixable, but not as written. the 4 major comments →

arxiv 2510.04019 v3 pith:NA4RM74X submitted 2025-10-05 cs.LG cs.AIcs.CL

Simple Policy Gradients for Reasoning with Diffusion Language Models

Anthony Zhan This is my paper
classification cs.LG cs.AIcs.CL
keywords AGRPOdiffusion language modelspolicy gradient methodsGRPOreinforcement learningMonte Carlo estimationreasoning benchmarkslow-discrepancy sampling
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Diffusion language models (dLLMs) generate text by iteratively unmasking tokens, but this multi-step process prevented the standard GRPO policy-gradient objective from being applied: computing exact token probabilities for every token would cost one forward pass per token. AGRPO's core move is to view the GRPO inner sum over tokens as an expectation over the m denoising steps, then estimate it by sampling a small number of steps and computing exact probabilities for tokens unmasked at those steps. Because the estimate is unbiased, the paper claims it is the first principled and tractable RL algorithm for dLLMs. Experiments on GSM8K, MATH, and Countdown report consistent gains over the LLaDA-8B-Instruct baseline and over previous heuristic dLLM-RL methods, and show the gains persist when fewer sampling steps are used at inference time. A careful reader should care because post-training is the missing piece that could make diffusion language models competitive with autoregressive models on reasoning tasks.

Core claim

The paper claims to settle the question of how to apply GRPO to diffusion language models: instead of approximating token-level probabilities with a single masked pass (as diffu-GRPO and UniGRPO do), AGRPO keeps the exact probabilities but amortizes the computation over timesteps. The inner sum (1/|o_i|)Σ_{t=1}^{|o_i|} ρ_i_t A_i in GRPO is replaced by (1/m)Σ_{t=1}^{m} ρ_i_t A_i = E_{t∼U{1..m}}[ρ_i_t A_i], where m is the number of denoising steps. This converts the intractable per-token sum into an expectation that can be Monte Carlo estimated with k≪m steps, using low-discrepancy sampling over step indices and caching the unmasking order to reconstruct exact contexts. The paper states that A

What carries the argument

The central identity is Eq. (3): replacing the GRPO sum over the |o_i| tokens with a sum over the m denoising steps and reading that sum as an expectation under a uniform distribution over step indices. This converts an O(|o_i|)-forward-pass computation into a Monte Carlo estimation problem with O(k) forward passes, where k is a small number of sampled steps. Low-discrepancy sampling (stratifying a uniform random variable into k bins) ensures the sampled step indices cover {1,...,m} while preserving marginals, so as k→m the objective recovers the original GRPO objective. Caching the unmasking order at generation time lets the algorithm reconstruct the exact partially-masked state at a sample

Load-bearing premise

The derivation in Eq. (3) implicitly assumes one token is unmasked per denoising step, so that the sum over m steps equals the sum over |o_i| tokens; the experiments unmask multiple tokens per step, so the claimed equivalence—and therefore the unbiasedness—is not established in the settings actually evaluated.

What would settle it

Take a trained rollout with n=384, m=192, and compute both (1/384)Σ_{tokens} ρ_i_t A_i and (1/192)Σ_{steps} ρ_i_t A_i. The paper's Eq. (3) asserts these are equal; if the step-sum expectation differs from the token-sum expectation (as it does whenever n≠m), the central unbiasedness claim fails, and the experiment would show the magnitude of that bias.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • AGRPO yields an unbiased policy-gradient estimate for dLLMs, removing the need for the biased one-step approximations used by diffu-GRPO and UniGRPO.
  • As k approaches m, the sampled objective converges exactly to the original GRPO objective, so the algorithm interpolates between a cheap noisy update and a full faithful one.
  • Models trained with AGRPO retain accuracy under evaluation with fewer sampling steps, matching baseline accuracy with 4x fewer steps on GSM8K.
  • Across GSM8K, MATH, and Countdown, AGRPO outperforms the LLaDA-8B-Instruct baseline and previous dLLM RL methods with no task-specific modifications.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Because the experiments fix n/m at 2–3 tokens per step, the equality in Eq. (3) is not exact; the paper's 'unbiased' claim would only be literally true if each step unmasked exactly one token. Checking the bias numerically on real rollouts would settle this.
  • The arXiv abstract reports a Sudoku gain (+69.7%) that the full text does not include; if that experiment was run, its omission is a reproducibility gap, and if it was not, the abstract overstates the results.
  • The time-axis amortization is a natural fit for continuous-time diffusion and flow matching, where sampling a timestep is already the standard training procedure; this could unify dLLM RL with flow-style GRPO methods.
  • A more faithful KL penalty would sum over tokens rather than steps; the paper applies a step-level KL inside the expectation, which is not the same as GRPO's sequence-level KL for multi-token unmasking.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces Amortized Group Relative Policy Optimization (AGRPO), an RL algorithm for diffusion LLMs. AGRPO replaces the token-level inner sum of GRPO with an expectation over denoising timesteps, estimates this expectation via Monte Carlo (low-discrepancy) sampling, and computes exact probability ratios by caching the unmasking order. The authors claim this yields the first tractable, unbiased, and faithful adaptation of GRPO to dLLMs, and report experiments on GSM8K, MATH, and Countdown using LLaDA-8B-Instruct, with gains over diffu-GRPO and the base model.

Significance. If the theoretical claim is made precise and correct, the paper addresses a real gap: RL post-training for diffusion LLMs is currently limited by biased or intractable probability estimates. The use of cached partially masked states to obtain exact token probabilities is a practical and sensible idea, and the code-release and pseudocode support reproducibility. However, the central unbiasedness claim, as written, is not established because the equality in Eq. (3) conflates timesteps with tokens in settings where several tokens are unmasked per step. The empirical section also lacks variance information, and the abstract contains results not present in the full text. These issues are load-bearing for the paper's main contribution.

major comments (4)
  1. [§4.1, Eq. (3); §4.2] The derivation replaces the token-level GRPO inner sum 1/|o_i| Σ_{t=1}^{|o_i|} ρ_i^t A_i with 1/m Σ_{t=1}^m ρ_i^t A_i. This is an identity only if m = |o_i|. In all reported experiments m = n/2 (GSM8K: n=384, m=192; MATH/Countdown: n=256, m=128), so the MC estimator in Eq. (4) is unbiased for a step-level objective, not for the GRPO objective in Eq. (2). The statement in §4.2 that 'as k approaches m, we fully recreate the original GRPO objective' is consequently false. The footnote citing Dr. GRPO's length-bias correction acknowledges the denominator change but is not reconciled with the abstract's claims of 'faithful' and 'unbiased' adaptation of GRPO. The authors should either define the exact objective being optimized, state clearly that it is a modified GRPO variant, or correct the estimator to be unbiased for the original token-level sum (e.g., by sampling tokens within a step).
  2. [Eq. (4); Algorithm 1; §4.3.1] The quantity ρ_i^t is not defined for steps that unmask more than one token. Eq. (4) and Algorithm 1 write ρ_i^t as a single ratio π_θ(o_i^t | q, o_i^{<t}) / π_old(o_i^t | q, o_i^{<t}), but if o_i^t is a token, the notation is ambiguous about which token among the several unmasked at step t is used. Section 4.3.1 states that 'each token is associated with a timestep,' suggesting a per-token computation, but the pseudocode samples one timestep and computes one ratio. If ρ_i^t is instead intended as a product or sum over the set of tokens unmasked at step t, that is not stated and would not equal the GRPO sum. This underspecification is not a minor clarity issue: the unbiasedness claim depends entirely on what ρ_i^t means and how tokens are selected. A formal definition and a corrected Algorithm 1 are needed.
  3. [§5, Table 1] Table 1 reports a single accuracy number per method and task, with no error bars, confidence intervals, or number of seeds. RL training is stochastic (group sampling, LoRA, temperature), and the reported gains over diffu-GRPO on GSM8K (87.3 vs. 79.8) and MATH (40.8 vs. 39.2) could be within run-to-run noise. Without variance estimates, the empirical claim of '1.3x performance gains over comparable RL methods' is not substantiated. Please provide multiple seeds or at least standard errors, and report the per-task hyperparameter settings (the appendix lists ranges such as k∈{16,24,32} but does not map values to tasks).
  4. [Abstract vs. §5.1] The arXiv metadata abstract reports '+59.4% on Countdown and +69.7% on Sudoku over the base LLaDA model.' The full text reports only three tasks and gives a +29.6% absolute gain on Countdown (40.0 vs. 10.4 in Table 1); Sudoku does not appear anywhere in the full text. These discrepancies must be resolved before publication. If the abstract is from a different version of the paper, the metadata needs to match the submitted manuscript.
minor comments (4)
  1. [§2.2 and §5] The text in §5 states 'we fix the response length to be n=384 for GSM8K + MATH and n=256 for Countdown,' but then says evaluation uses 'n=512, m=256 for MATH and n=256, m=128 for GSM8K + Countdown.' Please clarify which numbers refer to training and which to evaluation, and why MATH evaluation uses a longer response than training.
  2. [Appendix A.2, Table 2] The sample response contains apparent LaTeX artifacts: 'f rac14' in the m=192 trace and '<thinkthink>' in the m=48 trace. Please proofread the appendix examples.
  3. [Appendix A.1.1] Hyperparameters are given as ranges across tasks, but the mapping from each task to its specific k, G, and temperature is missing. This makes the experiments hard to reproduce despite the code link.
  4. [Eq. (2)] Equation (2) omits the min/clip structure for clarity and Eq. (4) later includes it, but the relationship between the two is not explained. A short sentence noting that Eq. (2) shows the unclipped form would avoid confusion.

Circularity Check

1 steps flagged

No fitted-input or self-citation circularity; one self-definitional step in the Eq. (3) derivation limits the theoretical claim.

specific steps
  1. self definitional [Section 4.1, Eq. (3) and footnote 3; echoed in Section 4.2; training config in Section 5]
    "Note that |o_i| has been replaced by m, the number of sampling steps (which may be < |o_i| if multiple tokens are unmasked per step). ... Low-discrepancy sampling induces the desirable property that as k approaches m, we fully recreate the original GRPO objective."

    Unbiasedness is built into the definition: once the objective is written as an expectation over m sampling steps, MC averaging over those steps is unbiased for that step-level quantity by construction. The assertion that k→m 'fully recreates the original GRPO objective' relies on the same replacement of |o_i| by m; it is not a derived equivalence with Eq. (2). With m=192,n=384 and m=128,n=256 in all reported runs, the step-level sum is not the token-level GRPO sum unless the per-step ratio aggregates multiple tokens, which is never defined. The 'faithful GRPO/unbiased' claim is therefore a restatement of the paper's redefinition rather than an independent derivation.

full rationale

This is a single definitional conflation, not a fitted-input prediction or self-citation chain. No parameter is fitted to data and then renamed as a prediction: the Table 1 accuracy gains are external benchmark results, so the empirical claims are not circular. No load-bearing self-citation exists: the low-discrepancy sampler is cited to Zheng et al. and Dr. GRPO's unnormalized advantages to Liu et al., both external. The abstract/full-text discrepancies (Sudoku absent, Countdown +59.4% vs +29.6%) are integrity/reporting concerns, not circularity. Because the central theoretical guarantee ('unbiased, faithful to GRPO') is partly definitional under the paper's own Eq. (3) substitution, a small nonzero score is appropriate, but the derivation is otherwise self-contained.

Axiom & Free-Parameter Ledger

7 free parameters · 5 axioms · 0 invented entities

The central claim rests on the step/token equivalence assumption, which is not satisfied in the experiments; hyperparameters ε and β are omitted from the appendix despite being promised.

free parameters (7)
  • k (number of MC samples) = 16, 24, or 32; task-specific values not specified per task
    Chosen by hand; controls variance/compute tradeoff.
  • G (group size) = 8 or 16
    Chosen by hand; GRPO group size.
  • temperature = 0.5 or 0.9
    Chosen by hand; sampling temperature.
  • ε (clip range) = not reported
    Promised in Appendix A.1.1 but missing.
  • β (KL coefficient) = not reported
    Promised in Appendix A.1.1 but missing.
  • learning rate = 5e-6
    Standard optimizer hyperparameter.
  • LoRA rank/alpha = 64
    Standard LoRA hyperparameter.
axioms (5)
  • domain assumption The dLLM policy factorizes over tokens within a denoising step, allowing exact token probabilities from a single forward pass
    Needed to compute ρ_i_t; not explicitly stated in Section 4.1.
  • ad hoc to paper The expectation in Eq (3) over uniform timesteps with m equal to the number of sampling steps is equivalent to the GRPO token-level objective
    This is the central algebraic step; it only holds when one token is unmasked per step, which contradicts the experimental setup (n=384, m=192).
  • domain assumption Rewards are outcome-based and advantage normalization is valid
    Standard RLVR; used in Section 3.2.
  • standard math The Schulman (2020) unbiased KL estimator applies to the per-timestep KL
    Cited; used in Eq (4).
  • domain assumption Random remasking during training does not affect the policy gradient (selection probability cancels in the ratio)
    Stated in Section 5 but not derived; if remasking distribution depends on the model's confidence, it may not cancel.

pith-pipeline@v1.3.0-alltime-deepseek · 13958 in / 16720 out tokens · 122734 ms · 2026-08-04T11:32:10.824952+00:00 · methodology

0 comments
read the original abstract

Diffusion large language models (dLLMs) represent a promising alternative to autoregressive LLMs; however, the lack of effective post-training techniques, including reinforcement learning (RL), remains a key challenge for dLLMs, especially for downstream applications. Existing approaches often rely on a sequence-level view that requires biased likelihood approximations. In this work, we propose Amortized Group Relative Policy Optimization (AGRPO), a policy gradient algorithm that leverages the Markovian nature of dLLMs, optimizing individual denoising steps rather than full sequences. Our approach improves alignment between the trained policy and the inference process and also admits efficient, unbiased gradient updates via a novel timestep estimation scheme. We demonstrate AGRPO's effectiveness on different math and reasoning tasks, achieving absolute accuracy gains of +59.4\% and +69.7\% on Countdown and Sudoku over the base LLaDA model, exceeding comparable methods such as diffu-GRPO. Code is available at https://github.com/probablyabot/agrpo.

Figures

Figures reproduced from arXiv: 2510.04019 by Anthony Zhan.

Figure 1
Figure 1. Figure 1: A comparison of different RL post-training algorithms for dLLMs. Existing algorithms [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Models under the autoregressive and diffusion paradigms are trained on different objec [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The inference compute/quality frontier for GSM8K across different configurations. Lines [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Back on Track: Aligning Rewards and States for Reasoning in Diffusion Large Language Models

    cs.CL 2026-06 unverdicted novelty 6.0

    PAPO improves reasoning performance in diffusion LLMs by converting sparse terminal rewards into dense step-wise credit and replaying real high-uncertainty trajectories, reporting gains up to 42.2% on Countdown.

  2. Sketch Then Paint: Hierarchical Reinforcement Learning for Diffusion Multi-Modal Large Language Models

    cs.AI 2026-05 unverdicted novelty 6.0

    Proposes HT-GRPO with sketch-then-paint staged updates, prompt-conditioned importance ratios, and hierarchical credit assignment for dMLLMs, reporting gains on GenEval and DPG plus quality metrics.

Reference graph

Works this paper leans on

33 extracted references · 18 linked inside Pith · cited by 2 Pith papers

  1. [1]

    Block diffusion: Interpolating between autoregressive and diffusion language models

    Marianne Arriola, Subham Sekhar Sahoo, Aaron Gokaslan, Zhihan Yang, Zhixuan Qi, Jiaqi Han, Justin T Chiu, and Volodymyr Kuleshov. Block diffusion: Interpolating between autoregressive and diffusion language models. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=tyEyYT267x

    2025

  2. [2]

    Johnson, Jonathan Ho, Daniel Tarlow, and Rianne van den Berg

    Jacob Austin, Daniel D. Johnson, Jonathan Ho, Daniel Tarlow, and Rianne van den Berg. Structured denoising diffusion models in discrete state-spaces. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, 2021. URL https://openreview.net/forum?id=h7-XixPCAL

    2021

  3. [3]

    Training diffusion models with reinforcement learning

    Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. Training diffusion models with reinforcement learning. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=YCWjhGrJFD

    2024

  4. [4]

    Training verifiers to solve math word problems

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021

    Pith/arXiv arXiv 2021

  5. [5]

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai D...

    Pith/arXiv arXiv 2025

  6. [6]

    Measuring mathematical problem solving with the math dataset

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. NeurIPS, 2021

    2021

  7. [7]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models, 2021. URL https://arxiv.org/abs/2106.09685

    Pith/arXiv arXiv 2021

  8. [8]

    Variational diffusion models

    Diederik Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, volume 34, pp.\ 21696--21707. Curran Associates, Inc., 2021. URL https://proceedings.neurips.cc/paper_files/paper/2021/file/b578f2a...

    2021

  9. [9]

    Gemini achieves gold-level performance at the international collegiate programming contest world finals

    Hanzhao (Maggie) Lin and Heng‐Tze Cheng. Gemini achieves gold-level performance at the international collegiate programming contest world finals. https://deepmind.google/discover/blog/gemini-achieves-gold-level-performance-at-the-international-collegiate-programming-contest-world-finals/, September 2025. Accessed:

    2025

  10. [10]

    Flow-grpo: Training flow matching models via online rl, 2025 a

    Jie Liu, Gongye Liu, Jiajun Liang, Yangguang Li, Jiaheng Liu, Xintao Wang, Pengfei Wan, Di Zhang, and Wanli Ouyang. Flow-grpo: Training flow matching models via online rl, 2025 a . URL https://arxiv.org/abs/2505.05470

    Pith/arXiv arXiv 2025

  11. [11]

    Understanding r1-zero-like training: A critical perspective, 2025 b

    Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding r1-zero-like training: A critical perspective, 2025 b . URL https://arxiv.org/abs/2503.20783

    Pith/arXiv arXiv 2025

  12. [12]

    Discrete diffusion modeling by estimating the ratios of the data distribution

    Aaron Lou, Chenlin Meng, and Stefano Ermon. Discrete diffusion modeling by estimating the ratios of the data distribution. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Ma...

    2024

  13. [13]

    Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct, 2025

    Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jianguang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, Yansong Tang, and Dongmei Zhang. Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct, 2025. URL https://arxiv.org/abs/2308.09583

    Pith/arXiv arXiv 2025

  14. [14]

    Advanced version of gemini with deep think officially achieves gold-medal standard at the international mathematical olympiad

    Thang Luong and Edward Lockhart. Advanced version of gemini with deep think officially achieves gold-medal standard at the international mathematical olympiad. https://deepmind.google/discover/blog/advanced-version-of-gemini-with-deep-think-officially-achieves-gold-medal-standard-at-the-international-mathematical-olympiad/, July 2025. Accessed:

    2025

  15. [15]

    dkv-cache: The cache for diffusion language models, 2025

    Xinyin Ma, Runpeng Yu, Gongfan Fang, and Xinchao Wang. dkv-cache: The cache for diffusion language models, 2025. URL https://arxiv.org/abs/2505.15781

    Pith/arXiv arXiv 2025

  16. [16]

    Large language diffusion models, 2025

    Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. Large language diffusion models, 2025. URL https://arxiv.org/abs/2502.09992

    Pith/arXiv arXiv 2025

  17. [17]

    OpenAI, :, Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, Alex Iftimie, Alex Karpenko, Alex Tachard Passos, Alexander Neitz, Alexander Prokofiev, Alexander Wei, Allison Tam, Ally Bennett, Ananya Kumar, Andre Saraiva, Andrea Vallone, Andrew Duberstein, Andrew Kondri...

    Pith/arXiv arXiv 2024

  18. [18]

    Your absorbing discrete diffusion secretly models the conditional distributions of clean data

    Jingyang Ou, Shen Nie, Kaiwen Xue, Fengqi Zhu, Jiacheng Sun, Zhenguo Li, and Chongxuan Li. Your absorbing discrete diffusion secretly models the conditional distributions of clean data. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=sMyXP8Tanm

    2025

  19. [19]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. arXiv preprint arXiv:2212.09748, 2022

    Pith/arXiv arXiv 2022

  20. [20]

    Simple and effective masked diffusion language models

    Subham Sekhar Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin T Chiu, Alexander Rush, and Volodymyr Kuleshov. Simple and effective masked diffusion language models. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (eds.), Advances in Neural Information Processing Systems, volume 37, pp.\ 13...

    2024

  21. [21]

    Approximating kl divergence

    John Schulman. Approximating kl divergence. http://joschu.net/blog/kl-approx.html, 2020. Blog post

    2020

  22. [22]

    Proximal policy optimization algorithms, 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms, 2017. URL https://arxiv.org/abs/1707.06347

    Pith/arXiv arXiv 2017

  23. [23]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024. URL https://arxiv.org/abs/2402.03300

    Pith/arXiv arXiv 2024

  24. [24]

    Openmathinstruct-2: Accelerating ai for math with massive open-source instruction data

    Shubham Toshniwal, Wei Du, Ivan Moshkov, Branislav Kisacanin, Alexan Ayrapetyan, and Igor Gitman. Openmathinstruct-2: Accelerating ai for math with massive open-source instruction data. arXiv preprint arXiv:2410.01560, 2024

    Pith/arXiv arXiv 2024

  25. [25]

    Do large language model benchmarks test reliability?, 2025

    Joshua Vendrow, Edward Vendrow, Sara Beery, and Aleksander Madry. Do large language model benchmarks test reliability?, 2025. URL https://arxiv.org/abs/2502.03461

    Pith/arXiv arXiv 2025

  26. [26]

    Fast-dllm: Training-free acceleration of diffusion llm by enabling kv cache and parallel decoding, 2025

    Chengyue Wu, Hao Zhang, Shuchen Xue, Zhijian Liu, Shizhe Diao, Ligeng Zhu, Ping Luo, Song Han, and Enze Xie. Fast-dllm: Training-free acceleration of diffusion llm by enabling kv cache and parallel decoding, 2025. URL https://arxiv.org/abs/2505.22618

    Pith/arXiv arXiv 2025

  27. [27]

    A minimalist approach to llm reasoning: from rejection sampling to reinforce, 2025

    Wei Xiong, Jiarui Yao, Yuhui Xu, Bo Pang, Lei Wang, Doyen Sahoo, Junnan Li, Nan Jiang, Tong Zhang, Caiming Xiong, and Hanze Dong. A minimalist approach to llm reasoning: from rejection sampling to reinforce, 2025. URL https://arxiv.org/abs/2504.11343

    Pith/arXiv arXiv 2025

  28. [28]

    Mmada: Multimodal large diffusion language models, 2025

    Ling Yang, Ye Tian, Bowen Li, Xinchen Zhang, Ke Shen, Yunhai Tong, and Mengdi Wang. Mmada: Multimodal large diffusion language models, 2025. URL https://arxiv.org/abs/2505.15809

    Pith/arXiv arXiv 2025

  29. [29]

    d1: Scaling reasoning in diffusion large language models via reinforcement learning, 2025

    Siyan Zhao, Devaansh Gupta, Qinqing Zheng, and Aditya Grover. d1: Scaling reasoning in diffusion large language models via reinforcement learning, 2025. URL https://arxiv.org/abs/2504.12216

    Pith/arXiv arXiv 2025

  30. [30]

    Masked diffusion models are secretly time-agnostic masked models and exploit inaccurate categorical sampling

    Kaiwen Zheng, Yongxin Chen, Hanzi Mao, Ming-Yu Liu, Jun Zhu, and Qinsheng Zhang. Masked diffusion models are secretly time-agnostic masked models and exploit inaccurate categorical sampling. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=CTC7CmirNr

    2025

  31. [31]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should not add it explicitly Type <Return> for now, but then later remove the command n...

  32. [32]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@first@sw \@firstoftwo \@ifundefined NAT@b*@#2 \@firstoftwo @num @NAT@ctr \@secondoft...

  33. [33]

    D + E5Z0w8vn ` Tk ] [fJ( S = H VBḯ ?|܍ߎwh[nZir 39 ]g!]8 lmI[ y*F+:;2_ݮ S VD@=

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibsetup #1 @NAT@ctr @ @openbib .11em \@plus.33em \@minus.07em 4000 4000 `\.\@m @bibit...

    arXiv 2000