Pith. sign in

REVIEW 4 major objections 6 minor 5 cited by

IGPO injects partial ground-truth reasoning hints through diffusion inpainting, turning zero-advantage GRPO failures into usable gradients and new state-of-the-art math scores for masked diffusion LLMs.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 17:51 UTC pith:JG5IS435

load-bearing objection The inpainting mechanism is genuinely clever and the internal ablations look sound, but the headline SOTA claims rest on an apparent train/eval overlap plus test-set tuning; the paper needs a clear split statement and code release before the main numbers can be trusted. the 4 major comments →

arxiv 2509.10396 v1 pith:JG5IS435 submitted 2025-09-12 cs.LG

Inpainting-Guided Policy Optimization for Diffusion Large Language Models

Siyan Zhao , Mengchen Liu , Jing Huang , Miao Liu , Chenyu Wang , Bo Liu , Yuandong Tian , Guan Pang
show 3 more authors
Sean Bell Aditya Grover Feiyu Chen
This is my paper
classification cs.LG
keywords diffusion language modelsmasked diffusionreinforcement learningGRPOinpaintingmathematical reasoningpolicy optimizationlength-aligned SFT
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper is trying to establish that masked diffusion LLMs' inpainting ability can be turned into a reinforcement-learning exploration mechanism. In group-relative policy optimization (GRPO), when every sampled answer to a hard question is wrong, all advantages collapse to zero and the gradient vanishes; IGPO detects this, overwrites selected masked positions with chunks of the ground-truth reasoning trace, lets the model inpaint the rest, keeps only verified-correct completions, and uses them to create reward variance. The paper reports that this cuts all-wrong groups by about 60%, produces non-degenerate gradients, and, combined with length-aligned SFT on rewritten concise traces, achieves new best results among full-attention masked diffusion LLMs on GSM8K, MATH500, and AMC. A sympathetic reader should care because it shows a native architectural capability—conditional inpainting—can be used to guide exploration rather than merely to generate text.

Core claim

The central discovery claimed is that the zero-advantage dilemma in group-based policy optimization for diffusion LLMs is avoidable. If all G responses sampled for a prompt are incorrect, GRPO assigns every response advantage zero and the clipped surrogate objective contributes no gradient. IGPO replaces part of this failure: it segments the ground-truth reasoning trace into chunks, fixes a random subset (excluding the final answer) into masked positions, generates additional completions by bidirectional inpainting, verifies whether they reach the correct answer, and swaps up to a fraction of the wrong responses with verified-correct inpainted responses. The modified group then has reward va

What carries the argument

The load-bearing mechanism is the inpainting step of a full-attention masked diffusion LLM. At denoising step 0, the continuation is fully masked; hint injection fixes selected positions to ground-truth tokens, and the model's bidirectional unmasking fills the rest while the fixed tokens stay unchanged. IGPO randomizes hint injection ratio and chunk size, excludes final answer tokens, and verifies each inpainted completion before admitting it to the group, so the only new training signal comes from completions that actually reach the correct answer. A secondary mechanism is entropy-based gradient filtering: on hint-token positions, updates are restricted to the top-percentile highest-entropy

Load-bearing premise

The load-bearing assumption is that the single-pass mean-field estimator used for token-level importance ratios and sequence-level KL is accurate enough that IGPO's updates are valid policy gradients; the paper states this estimator is biased relative to the exact diffusion policy but does not quantify the bias.

What would settle it

Compare IGPO against a control where the same verified-correct ground-truth completions are inserted into the group without inpainting; if accuracy gains disappear, the effect comes from extra supervised signal, not from inpainting-guided exploration. A second check is to estimate the bias of the mean-field likelihood estimator on the 8B diffusion model with 128 denoising steps and test whether an unbiased estimator changes the gradient sign on held-out all-wrong groups.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If IGPO is correct, hard problems no longer produce empty learning steps: an all-wrong group is converted into a group containing verified correct completions, so policy-gradient updates continue.
  • The reported ~60% drop in all-wrong groups implies that IGPO changes RL dynamics early in training, not just the final checkpoint; this should reduce sample waste and improve convergence speed.
  • Length-aligned SFT on rewritten concise traces is claimed to be a stronger initialization than SFT on original verbose traces; combined with IGPO it produces cumulative gains on all three benchmarks.
  • Partial hint injection outperforming full ground-truth injection indicates that self-generated reasoning, not maximal supervision, is what makes the policy update effective.
  • The recipe establishes a new state-of-the-art point for full-attention masked diffusion LLMs on GSM8K, MATH500, and AMC.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Because IGPO needs ground-truth reasoning traces as hints, its reach is limited to domains where such traces exist; an untested extension is bootstrapping from the model's own high-reward rollouts once it starts solving some problems, turning hints into a self-produced curriculum.
  • The entropy-filtering result suggests hint tokens act like a curriculum over decision boundaries; an editor could test whether an adaptive threshold that anneals over training further stabilizes learning.
  • The same verified-correct-inpainted-completions-create-reward-variance mechanism should transfer to other group-based RL objectives beyond GRPO and to verifiable non-math tasks, since nothing in the mechanism is math-specific.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes IGPO (Inpainting-Guided Policy Optimization), an RL method for masked diffusion LLMs that injects partial ground-truth reasoning traces as fixed hints during online sampling. When a GRPO group is all-wrong, the model inpaints completions conditioned on these hints, keeps only verified-correct inpainted responses, and replaces some incorrect responses to create nonzero group-relative advantages. The authors also propose Length-Aligned SFT on rewritten concise traces and entropy-based gradient filtering for hint tokens. They report state-of-the-art results for full-attention masked dLLMs on GSM8K, MATH500, and AMC, and attribute the gains to reduced all-wrong groups, restored gradients, and improved sample efficiency.

Significance. If the results are sound, the central idea is novel and potentially impactful: it leverages the inpainting capability of masked diffusion LLMs to guide exploration without full supervision, and the partial-hint vs. full-hint ablation (Figure 4) is a genuinely informative control. The method also addresses a real failure mode of group-based RL (zero advantages). However, the headline evaluation is compromised by a likely train/eval overlap and by test-set-driven hyperparameter selection. The absence of code or released prompt IDs prevents verification. The work is therefore significant only conditionally on resolving these evaluation concerns.

major comments (4)
  1. [§4.2 vs. Table 1] The RL training set is described as 'combining questions from both GSM8K and MATH500' (12,794 examples after deduplication), and Table 1 evaluates on GSM8K and MATH500. No train/eval split, prompt IDs, or exclusion procedure is provided. If evaluation questions appear in the RL prompt set, all GSM8K and MATH500 numbers in Table 1, including the 'new state-of-the-art' claim, may reflect memorization rather than generalization. The authors must either document an explicit non-overlap (e.g., release the exact prompt set and show zero intersection with the evaluation instances) or re-run the experiments on a disjoint evaluation set.
  2. [§4.4, Figure 5, Table 1] The entropy filtering threshold tau is selected by experiments run on GSM8K (Figure 5 caption: 'This results is run on GSM8K') and then used for the final GSM8K result in Table 1. No held-out validation split is described. Tuning tau on the reported test benchmark is a form of test-set leakage for that benchmark. The authors should either tune on a validation subset and report all metrics on held-out test questions, or show that the chosen tau is not sensitive and report results for other tau on a separate validation set.
  3. [§3.1, Eq. (7)] The augmented group is constructed by injecting ground-truth reasoning chunks, generating inpainted responses, retaining only those with reward 1, and then optimizing the policy to increase their likelihood with positive advantages. This is heavily supervised: the policy is trained on correct responses that contain substantial ground-truth tokens. The improvement over GRPO may therefore partly be due to extra supervised signal rather than to a better RL algorithm. The paper should add a control that gives GRPO or a behavior-cloning baseline the same information and compute (e.g., SFT on inpainted correct completions or on the same hint-conditioned samples) to isolate the RL-specific benefit.
  4. [§2.2, §3.2] The mean-field estimator of DiffuGRPO is adopted for token-level importance ratios and sequence-level KL, and the paper states that it 'introduces bias relative to the exact diffusion policy' but does not quantify this bias. Since the exact diffusion policy is intractable, the 'policy gradient' interpretation of Eq. (7) is approximate. The comparisons to DiffuGRPO remain fair because both use the same estimator, but the paper should provide some empirical or theoretical characterization of the bias, at least for the 128-step LLaDA-8B setup, to support the claim that the gradients are meaningful policy gradients.
minor comments (6)
  1. [General] No code, data, or trained model checkpoints are released. Given the evaluation concerns, releasing the prompt set and training/eval split is essential for reproducibility.
  2. [Figure 1] Figure 1(c) is referenced in the caption and text as 'Comparison with Prior Methods', but no actual plot or numbers are visible in the caption; please provide the figure or remove the reference.
  3. [§4.4] The sentence 'restricting gradient updates to high-entropy positions prevents the destabilizing effects of large gradients on high-entropy positions' appears circular. It should clarify why high-entropy positions lead to unstable gradients and how the filtering mitigates this.
  4. [Table 1] Report standard errors or confidence intervals for the final Table 1 numbers. Figure 3 reports averages over 3 seeds, but Table 1 does not indicate whether the reported numbers are seed-averaged or from a single run.
  5. [Captions] Typographical errors: 'This results is run on GSM8K' appears in both Figure 5 and Figure 6 captions; should be 'This result is run...'.
  6. [§4.4] The description of entropy percentile is ambiguous: 'top tau percentile' could mean top tau% or top (1-tau)% depending on convention. Clarify that tau=0.2 means the top 20% highest-entropy positions.

Circularity Check

1 steps flagged

RL training prompts are stated to combine GSM8K and MATH500 questions, the same benchmarks in Table 1, with no stated exclusion; the headline SOTA is therefore contaminated. The IGPO mechanism itself is a transparent guided-exploration design, not a circular derivation, and the DiffuGRPO mean-field adoption is a disclosed self-citation rather than a logical circle.

specific steps
  1. fitted input called prediction [Section 4.2 (Experimental Setup) and Table 1]
    "For reinforcement learning, we train on the MetaMathQA dataset, specifically using the “Answer Augmentation” split and combining questions from both GSM8K and MATH500. After deduplicating identical questions, we obtain 12,794 unique training examples. ... We evaluate our approach on three mathematics benchmarks: GSM8K, MATH500 and AMC."

    The RL prompt set is explicitly built from GSM8K and MATH500, and Table 1 evaluates on exactly GSM8K and MATH500. Taking the manuscript literally, the evaluation questions are among the RL training examples; no exclusion of evaluation questions, prompt IDs, or split code is provided. The reported pass@1 gains on these two benchmarks are therefore measurements on the training distribution rather than held-out predictions. The 'new state-of-the-art' result reduces to fitting the exact benchmark prompts, which is the fitted-input-called-prediction pattern and invalidates the headline generalization claim for two of the three benchmarks.

full rationale

The core IGPO mechanism is not circular: it explicitly injects partial ground-truth hints, verifies the resulting completions, and keeps only correct ones. The resulting non-zero advantages are a designed property of the algorithm, and the paper's empirical claims (e.g., the ~60% reduction in all-wrong groups, and the relative IGPO-vs-GRPO training curves) have independent content. The adoption of DiffuGRPO's mean-field estimator from Zhao et al. (2025) is a self-citation with overlapping authors, but it is disclosed as an approximation and is not used to forbid alternatives, so it is a correctness/bias risk rather than a circular reduction. The main circularity concern is the benchmark overlap described in Section 4.2: if the RL prompts literally include GSM8K and MATH500 questions, the central SOTA numbers on those benchmarks are not independent predictions. Because AMC is not listed among the RL prompt sources, the AMC improvement remains held-out, which is why the score is 6 rather than higher.

Axiom & Free-Parameter Ledger

5 free parameters · 3 axioms · 0 invented entities

No new physical or architectural entities are introduced. The method's headroom comes from injected ground-truth tokens and several hand-set hyperparameters, and it inherits the mean-field probability estimator from a prior paper by overlapping authors. The central result therefore rests on the validity of the mean-field approximation and the exclusion of evaluation data from training, which the paper does not guarantee.

free parameters (5)
  • Hint injection ratio range eta ~ U[0.2, 0.6] = U[0.2, 0.6]
    Selected as the best range in Figure 4 on the same benchmark suite; controls how much ground truth is revealed.
  • Chunk size range c_j ~ U[5, 10] = U[5, 10]
    Chosen by hand to segment ground-truth reasoning traces; no sensitivity analysis is reported.
  • Replacement fraction lambda = 0.5
    Controls how many wrong responses are replaced by verified correct inpainted responses; set without ablation.
  • Entropy filter threshold tau = 0.2
    Tuned in Figure 5 on GSM8K, which is also one of the main evaluation benchmarks; top-20% highest-entropy hint positions are updated.
  • SFT epochs = 100
    The paper notes dLLMs benefit from longer epochs, but the exact epoch count is chosen by hand and is not fully ablated.
axioms (3)
  • domain assumption The single-pass mean-field estimator of DiffuGRPO yields token-level importance ratios and sequence-level KL terms accurate enough for policy-gradient updates.
    Adopted in Section 3.2; the paper states this approximation introduces bias relative to the exact diffusion policy, yet Eq. (7) depends on it for every gradient.
  • domain assumption Injected ground-truth hint tokens can be treated as fixed conditioning while the rest of the response is generated, and the resulting response can be scored under pi_theta as if it were a policy rollout.
    Used in Section 3.1; the fixed tokens are not sampled from the policy, so the log-probability of the full concatenated response is a mixed object.
  • domain assumption LLaMA-4-Maverick's rewriting of verbose reasoning traces preserves the logical content and does not introduce systematic errors.
    The rewriting procedure in Appendix D is validated only by one qualitative example, not by a systematic check for introduced errors.

pith-pipeline@v1.3.0-alltime-deepseek · 17910 in / 13683 out tokens · 153996 ms · 2026-08-04T17:51:21.644427+00:00 · methodology

0 comments
read the original abstract

Masked diffusion large language models (dLLMs) are emerging as promising alternatives to autoregressive LLMs, offering competitive performance while supporting unique generation capabilities such as inpainting. We explore how inpainting can inform RL algorithm design for dLLMs. Aligning LLMs with reinforcement learning faces an exploration challenge: sparse reward signals and sample waste when models fail to discover correct solutions. While this inefficiency affects LLMs broadly, dLLMs offer a distinctive opportunity--their inpainting ability can guide exploration. We introduce IGPO (Inpainting Guided Policy Optimization), an RL framework that strategically inserts partial ground-truth reasoning traces during online sampling. Unlike providing full solutions, inpainting steers exploration toward promising trajectory spaces while preserving self-generated reasoning, bridging supervised fine-tuning and reinforcement learning. We apply IGPO to group-based optimization methods such as GRPO, where exploration failures cause zero advantages and gradients. IGPO restores meaningful gradients while improving sample efficiency. We also propose supervised fine-tuning on synthetically rewritten concise traces that better align with dLLM generation patterns. With additional techniques including entropy-based filtering, our training recipe yields substantial gains across three mathematical benchmarks--GSM8K, Math500, and AMC--achieving new state-of-the-art results for full-attention masked dLLMs.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. JAGG: Jacobian-Aggregated Group Gradient for Efficient GRPO Training of Diffusion Models

    cs.LG 2026-07 conditional novelty 7.0

    JAGG replaces per-step gradient backpropagation in diffusion GRPO with two endpoint backward passes joined by timestep-weighted interpolation, giving ~2x backward-pass savings at modest quality cost.

  2. Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models

    cs.LG 2026-01 unverdicted novelty 7.0

    A single LLM improves its own reasoning by self-distilling from privileged verified traces as teacher to its question-only student policy, outperforming off-policy distillation and RL on math benchmarks with better to...

  3. Beyond On-Policy Exploration: Integrating External Policy Rollouts for Reinforcement Learning in Diffusion Language Models

    cs.LG 2026-08 conditional novelty 6.0

    ERILS mixes length-controlled external rollouts from a stronger policy with on-policy rollouts and processes their rewards separately, substantially improving dLLM reinforcement learning on reasoning tasks.

  4. Self-Supervised On-Policy Distillation for Reasoning Language Models

    cs.LG 2026-05 unverdicted novelty 6.0

    SSOPD converts intra-group correct-wrong contrast into process supervision by distilling a teacher distribution from the shortest correct completion into prefixes of the longest wrong completion, improving GRPO on AIM...

  5. Boundary-Guided Policy Optimization for Memory-efficient RL of Diffusion Large Language Models

    cs.LG 2025-10 conditional novelty 4.0

    BGPO makes RL for diffusion LLMs memory-efficient by decomposing the ELBO-based objective into per-sample terms, enabling larger Monte Carlo sample sizes and better performance.

Reference graph

Works this paper leans on

51 extracted references · 26 linked inside Pith · cited by 5 Pith papers

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  3. [3]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should not add it explicitly Type <Return> for now, but then later remove the command n...

  4. [4]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@first@sw \@firstoftwo \@ifundefined NAT@b*@#2 \@firstoftwo @num @NAT@ctr \@secondoft...

  5. [5]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibsetup #1 @NAT@ctr @ @openbib .11em \@plus.33em \@minus.07em 4000 4000 `\.\@m @bibit...

  6. [6]

    Back to basics: Revisiting reinforce style optimization for learning from human feedback in llms

    Arash Ahmadian, Chris Cremer, Matthias Gall \'e , Marzieh Fadaee, Julia Kreutzer, Olivier Pietquin, Ahmet \"U st \"u n, and Sara Hooker. Back to basics: Revisiting reinforce style optimization for learning from human feedback in llms. arXiv preprint arXiv:2402.14740, 2024

    Pith/arXiv arXiv 2024

  7. [7]

    Llama 3 model card

    AI@Meta. Llama 3 model card. 2024. https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md

    2024

  8. [8]

    Block diffusion: Interpolating between autoregressive and diffusion language models

    Marianne Arriola, Aaron Gokaslan, Justin T Chiu, Zhihan Yang, Zhixuan Qi, Jiaqi Han, Subham Sekhar Sahoo, and Volodymyr Kuleshov. Block diffusion: Interpolating between autoregressive and diffusion language models. In The Thirteenth International Conference on Learning Representations, 2025

    2025

  9. [9]

    Structured denoising diffusion models in discrete state-spaces

    Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne Van Den Berg. Structured denoising diffusion models in discrete state-spaces. Advances in neural information processing systems, 34: 0 17981--17993, 2021

    2021

  10. [10]

    Training a helpful and harmless assistant with reinforcement learning from human feedback

    Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862, 2022

    Pith/arXiv arXiv 2022

  11. [11]

    Ting Chen, Ruixiang Zhang, and Geoffrey E. Hinton. Analog bits: Generating discrete data using diffusion models with self-conditioning. ArXiv, abs/2208.04202, 2022

    Pith/arXiv arXiv 2022

  12. [12]

    Gemini diffusion, 2025

    DeepMind. Gemini diffusion, 2025. https://deepmind.google/models/gemini-diffusion/

    2025

  13. [13]

    DiffuSeq : Sequence to sequence text generation with diffusion models

    Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu, and Lingpeng Kong. DiffuSeq : Sequence to sequence text generation with diffusion models. In International Conference on Learning Representations, ICLR, 2023

    2023

  14. [14]

    Scaling diffusion language models via adaptation from autoregressive models

    Shansan Gong, Shivam Agarwal, Yizhe Zhang, Jiacheng Ye, Lin Zheng, Mukai Li, Chenxin An, Peilin Zhao, Wei Bi, Jiawei Han, Hao Peng, and Lingpeng Kong. Scaling diffusion language models via adaptation from autoregressive models. In The Thirteenth International Conference on Learning Representations, 2025 a

    2025

  15. [15]

    Diffucoder: Understanding and improving masked diffusion models for code generation

    Shansan Gong, Huangjie Zheng Ruixiang Zhang, Jiatao Gu, Navdeep Jaitly, Lingpeng Kong, and Yizhe Zhang. Diffucoder: Understanding and improving masked diffusion models for code generation. 2025 b . https://arxiv.org/abs/2506.20639

    Pith/arXiv arXiv 2025

  16. [16]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    Pith/arXiv arXiv 2025

  17. [17]

    Discrete diffusion trajectory alignment via stepwise decomposition, 2025

    Jiaqi Han, Austin Wang, Minkai Xu, Wenda Chu, Meihua Dang, Yisong Yue, and Stefano Ermon. Discrete diffusion trajectory alignment via stepwise decomposition, 2025. https://arxiv.org/abs/2507.04832

    arXiv 2025

  18. [18]

    Abdelfattah, Jae sun Seo, Zhiru Zhang, and Udit Gupta

    Zhanqiu Hu, Jian Meng, Yash Akhauri, Mohamed S. Abdelfattah, Jae sun Seo, Zhiru Zhang, and Udit Gupta. Accelerating diffusion language model inference via efficient kv caching and guided diffusion, 2025. https://arxiv.org/abs/2505.21467

    arXiv 2025

  19. [19]

    Ponti, and Ivan Titov

    Zeyu Huang, Tianhao Cheng, Zihan Qiu, Zili Wang, Yinghui Xu, Edoardo M. Ponti, and Ivan Titov. Blending supervised and reinforcement fine-tuning with prefix sampling, 2025. https://arxiv.org/abs/2507.01679

    Pith/arXiv arXiv 2025

  20. [20]

    Mercury: Ultra-fast language models based on diffusion

    Inception Labs , Samar Khanna, Siddhant Kharbanda, Shufan Li, Harshit Varma, Eric Wang, Sawyer Birnbaum, Ziyang Luo, Yanis Miraoui, Akash Palrecha, Stefano Ermon, Aditya Grover, and Volodymyr Kuleshov. Mercury: Ultra-fast language models based on diffusion. 2025. https://arxiv.org/abs/2506.17298

    Pith/arXiv arXiv 2025

  21. [21]

    Accelerating diffusion llms via adaptive parallel decoding, 2025

    Daniel Israel, Guy Van den Broeck, and Aditya Grover. Accelerating diffusion llms via adaptive parallel decoding, 2025. https://arxiv.org/abs/2506.00413

    arXiv 2025

  22. [22]

    Lavida: A large diffusion language model for multimodal understanding

    Shufan Li, Konstantinos Kallidromitis, Hritik Bansal, Akash Gokul, Yusuke Kato, Kazuki Kozuka, Jason Kuen, Zhe Lin, Kai-Wei Chang, and Aditya Grover. Lavida: A large diffusion language model for multimodal understanding. ArXiv preprint, abs/2505.16839, 2025. https://arxiv.org/abs/2505.16839

    Pith/arXiv arXiv 2025

  23. [23]

    Diffusion-lm improves controllable text generation

    Xiang Lisa Li, John Thickstun, Ishaan Gulrajani, Percy Liang, and Tatsunori Hashimoto. Diffusion-lm improves controllable text generation. ArXiv, abs/2205.14217, 2022

    Pith/arXiv arXiv 2022

  24. [24]

    Remax: A simple, effective, and efficient reinforcement learning method for aligning large language models

    Ziniu Li, Tian Xu, Yushun Zhang, Zhihang Lin, Yang Yu, Ruoyu Sun, and Zhi-Quan Luo. Remax: A simple, effective, and efficient reinforcement learning method for aligning large language models. arXiv preprint arXiv:2310.10505, 2023

    Pith/arXiv arXiv 2023

  25. [25]

    dllm-cache: Accelerating diffusion large language models with adaptive caching

    Zhiyuan Liu, Yicun Yang, Yaojie Zhang, Junjie Chen, Chang Zou, Qingyuan Wei, Shaobo Wang, and Linfeng Zhang. dllm-cache: Accelerating diffusion large language models with adaptive caching. arXiv preprint arXiv:2506.06295, 2025 a

    Pith/arXiv arXiv 2025

  26. [26]

    Understanding r1-zero-like training: A critical perspective

    Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding r1-zero-like training: A critical perspective. arXiv preprint arXiv:2503.20783, 2025 b

    Pith/arXiv arXiv 2025

  27. [27]

    Discrete diffusion modeling by estimating the ratios of the data distribution

    Aaron Lou, Chenlin Meng, and Stefano Ermon. Discrete diffusion modeling by estimating the ratios of the data distribution. In Forty-first International Conference on Machine Learning, 2024

    2024

  28. [28]

    dkv-cache: The cache for diffusion language models, 2025

    Xinyin Ma, Runpeng Yu, Gongfan Fang, and Xinchao Wang. dkv-cache: The cache for diffusion language models, 2025. https://arxiv.org/abs/2505.15781

    Pith/arXiv arXiv 2025

  29. [29]

    The llama 4 herd: The beginning of a new era of natively multimodal ai innovation

    AI Meta. The llama 4 herd: The beginning of a new era of natively multimodal ai innovation. https://ai. meta. com/blog/llama-4-multimodal-intelligence/, 2025

    2025

  30. [30]

    Diffusion language models are super data learners

    Jinjie Ni and the team. Diffusion language models are super data learners. https://jinjieni.notion.site/Diffusion-Language-Models-are-Super-Data-Learners-239d8f03a866800ab196e49928c019ac, 2025. Notion Blog

    2025

  31. [31]

    Scaling up masked diffusion models on text

    Shen Nie, Fengqi Zhu, Chao Du, Tianyu Pang, Qian Liu, Guangtao Zeng, Min Lin, and Chongxuan Li. Scaling up masked diffusion models on text. arXiv preprint arXiv:2410.18514, 2024

    Pith/arXiv arXiv 2024

  32. [32]

    Large language diffusion models, 2025

    Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. Large language diffusion models, 2025. https://arxiv.org/abs/2502.09992

    Pith/arXiv arXiv 2025

  33. [33]

    Your absorbing discrete diffusion secretly models the conditional distributions of clean data

    Jingyang Ou, Shen Nie, Kaiwen Xue, Fengqi Zhu, Jiacheng Sun, Zhenguo Li, and Chongxuan Li. Your absorbing discrete diffusion secretly models the conditional distributions of clean data. arXiv preprint arXiv:2406.03736, 2024

    Pith/arXiv arXiv 2024

  34. [34]

    Training language models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35: 0 27730--27744, 2022

    2022

  35. [35]

    Diffusion beats autoregressive in data-constrained settings, 2025

    Mihir Prabhudesai, Mengning Wu, Amir Zadeh, Katerina Fragkiadaki, and Deepak Pathak. Diffusion beats autoregressive in data-constrained settings, 2025. https://arxiv.org/abs/2507.15857

    arXiv 2025

  36. [36]

    Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin T

    Subham S. Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin T. Chiu, Alexander Rush, and Volodymyr Kuleshov. Simple and effective masked diffusion language models. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomczak, and Cheng Zhang, editors, Advances in Neural Information Processin...

    2024

  37. [37]

    Esoteric language models, 2025

    Subham Sekhar Sahoo, Zhihan Yang, Yash Akhauri, Johnna Liu, Deepansha Singh, Zhoujun Cheng, Zhengzhong Liu, Eric Xing, John Thickstun, and Arash Vahdat. Esoteric language models, 2025. https://arxiv.org/abs/2506.01928

    Pith/arXiv arXiv 2025

  38. [38]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    Pith/arXiv arXiv 2017

  39. [39]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024

    Pith/arXiv arXiv 2024

  40. [40]

    Jiaxin Shi, Kehang Han, Zhe Wang, Arnaud Doucet, and Michalis K. Titsias. Simplified and generalized masked diffusion for discrete data. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomczak, and Cheng Zhang, editors, Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information P...

    2024

  41. [41]

    wd1: Weighted policy optimization for reasoning in diffusion language models, 2025

    Xiaohang Tang, Rares Dolga, Sangwoong Yoon, and Ilija Bogunovic. wd1: Weighted policy optimization for reasoning in diffusion language models, 2025. https://arxiv.org/abs/2507.08838

    arXiv 2025

  42. [42]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599, 2025

    Pith/arXiv arXiv 2025

  43. [43]

    Qwen2.5: A party of foundation models, September 2024

    Qwen Team. Qwen2.5: A party of foundation models, September 2024. https://qwenlm.github.io/blog/qwen2.5/

    2024

  44. [44]

    Fast-dllm: Training-free acceleration of diffusion llm by enabling kv cache and parallel decoding, 2025

    Chengyue Wu, Hao Zhang, Shuchen Xue, Zhijian Liu, Shizhe Diao, Ligeng Zhu, Ping Luo, Song Han, and Enze Xie. Fast-dllm: Training-free acceleration of diffusion llm by enabling kv cache and parallel decoding, 2025. https://arxiv.org/abs/2505.22618

    Pith/arXiv arXiv 2025

  45. [45]

    Mmada: Multimodal large diffusion language models

    Ling Yang, Ye Tian, Bowen Li, Xinchen Zhang, Ke Shen, Yunhai Tong, and Mengdi Wang. Mmada: Multimodal large diffusion language models. ArXiv preprint, abs/2505.15809, 2025. https://arxiv.org/abs/2505.15809

    Pith/arXiv arXiv 2025

  46. [46]

    Dream 7b, 2025

    Jiacheng Ye, Zhihui Xie, Lin Zheng, Jiahui Gao, Zirui Wu, Xin Jiang, Zhenguo Li, and Lingpeng Kong. Dream 7b, 2025. https://hkunlp.github.io/blog/2025/dream

    2025

  47. [47]

    Llada-v: Large language diffusion models with visual instruction tuning

    Zebin You, Shen Nie, Xiaolu Zhang, Jun Hu, Jun Zhou, Zhiwu Lu, Ji-Rong Wen, and Chongxuan Li. Llada-v: Large language diffusion models with visual instruction tuning. arXiv preprint arXiv:2505.16933, 2025

    Pith/arXiv arXiv 2025

  48. [48]

    On-policy rl meets off-policy experts: Harmonizing supervised fine-tuning and reinforcement learning via dynamic weighting, 2025

    Wenhao Zhang, Yuexiang Xie, Yuchang Sun, Yanxi Chen, Guoyin Wang, Yaliang Li, Bolin Ding, and Jingren Zhou. On-policy rl meets off-policy experts: Harmonizing supervised fine-tuning and reinforcement learning via dynamic weighting, 2025. https://arxiv.org/abs/2508.11408

    arXiv 2025

  49. [49]

    d1: Scaling reasoning in diffusion large language models via reinforcement learning

    Siyan Zhao, Devaansh Gupta, Qinqing Zheng, and Aditya Grover. d1: Scaling reasoning in diffusion large language models via reinforcement learning. arXiv preprint arXiv:2504.12216, 2025

    Pith/arXiv arXiv 2025

  50. [50]

    Group sequence policy optimization

    Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, et al. Group sequence policy optimization. arXiv preprint arXiv:2507.18071, 2025

    Pith/arXiv arXiv 2025

  51. [51]

    Llada 1.5: Variance-reduced preference optimization for large language diffusion models

    Fengqi Zhu, Rongzhen Wang, Shen Nie, Xiaolu Zhang, Chunwei Wu, Jun Hu, Jun Zhou, Jianfei Chen, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. Llada 1.5: Variance-reduced preference optimization for large language diffusion models. ArXiv preprint, abs/2505.19223, 2025. https://arxiv.org/abs/2505.19223

    Pith/arXiv arXiv 2025