Pith. sign in

REVIEW 4 cited by

Generalized Interpolating Discrete Diffusion

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2503.04482 v2 pith:KDOWQNOX submitted 2025-03-06 cs.CL cs.AIcs.LG

Dimitri von Rütte , Janis Fluri , Yuhui Ding , Antonio Orvieto , Bernhard Schölkopf , Thomas Hofmann This is my paper
classification cs.CLcs.AIcs.LG
keywords diffusiondiscretegiddachieveflexibilityinabilityinterpolatinglanguage
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

While state-of-the-art language models achieve impressive results through next-token prediction, they have inherent limitations such as the inability to revise already generated tokens. This has prompted exploration of alternative approaches such as discrete diffusion. However, masked diffusion, which has emerged as a popular choice due to its simplicity and effectiveness, reintroduces this inability to revise words. To overcome this, we generalize masked diffusion, deriving a new family of general interpolating discrete diffusion (GIDD) which offers greater flexibility in the design of the noising processes. Leveraging a novel diffusion ELBO, we achieve compute-matched state-of-the-art performance in diffusion language modeling. Exploiting GIDD's flexibility, we explore a hybrid approach combining masking and uniform noise, leading to improved sample quality and unlocking the ability for the model to correct its own mistakes, an area where autoregressive models notoriously have struggled. Code: https://github.com/dvruette/gidd/

Discussion (0). Sign in to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Scalable Non-Equivariant 3D Molecule Generation via Rotational Alignment

    cs.LG 2025-06 conditional novelty 7.0 of 10

    A rotationally aligned latent space lets non-equivariant diffusion models match equivariant model quality on 3D molecule generation.

  2. WAM-Diff2: Hierarchical AR-to-Diffusion Distillation for Highly Efficient Autonomous Driving VLA

    cs.RO 2026-08 conditional novelty 6.0 of 10

    A three-stage distillation converts an autoregressive driving VLA into a block-causal masked diffusion model, preserving planning accuracy while decoding 2.8x faster (15.1x with optimized kernels).

  3. Hierarchical Domain Generalization

    cs.LG 2026-07 conditional novelty 6.0 of 10

    Over infinite domains, hierarchy-uniform domain generalization is impossible for every nontrivial hypothesis class; a length-generalization bound is a property of the length hierarchy, not a hierarchy-free guarantee.

  4. Fine-Tuning Masked Diffusion for Provable Self-Correction

    cs.LG 2025-10 conditional novelty 6.0 of 10

    PRISM fine-tunes any masked diffusion model with a binary-cross-entropy loss so its new head provably estimates per-token quality p(x_i=y_i|y⊕m_i) and can remask low-quality tokens at inference.

Pith tools