Pith. sign in

REVIEW 4 major objections 5 minor 5 cited by

Adaptive Classifier-Free Guidance via Dynamic Low-Confidence Masking

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that re-masking the tokens a masked diffusion language model is least confident about, and using that as the unconditional input in classifier-free guidance, improves generation quality and reasoning accuracy more than…

desk verdict A plausible CFG tweak for masked diffusion LMs with consistent gains, but the adaptive mechanism is unvalidated—needs a random-re-masking control before I'd trust the claim. read the letter →

arxiv 2505.20199 v1 pith:LBTHEC6Q submitted 2025-05-26 cs.CL

Pengxiang Li , Shilin Yan , Joey Tsai , Renrui Zhang , Ruichuan An , Ziyu Guo , Xiaowei Gao This is my paper · ORCID
classification cs.CL
keywords adaptiveclassifier-freeguidancemaskeddiffusionlanguagemodelslow-confidencemaskingdynamicunconditionalinputmodelconfidenceLLaDAiterativegenerationscale
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that classifier-free guidance for iterative masked diffusion language models is held back by its static unconditional input, and that the unconditional input should instead be rebuilt at every generation step from the model's own uncertainty. It proposes A-CFG, which takes the tokens the model is currently least confident about, re-masks them to [MASK], and uses that modified sequence as the unconditional input in the standard CFG interpolation. On the LLaDA 8B model this consistently beats both no guidance and standard CFG, with a 3.9-point gain on GPQA and an 8.0-point gain on Sudoku, and the same recipe improves Dream-7B. The authors' central claim is that confidence-aware, position-specific unconditioning focuses CFG's correction exactly where the model is ambiguous, making guidance more effective than a one-size-fits-all unconditional state.

What carries the argument

The load-bearing object is the dynamic unconditional input $x^{(k)}_{\mathrm{uncond}}$ constructed at each step. Its construction uses the token-level confidence score $c^{(k)}_j = \max_v \mathrm{softmax}(L^{(k)}_{\mathrm{cond}})_{j,v}$ for each non-masked position, selects the lowest-confidence fraction $\rho$, replaces exactly those tokens with [MASK], and feeds the result to the same model to obtain $L^{(k)}_{\mathrm{uncond}}$. The standard CFG formula $L^{(k)}_{\mathrm{guided}} = L^{(k)}_{\mathrm{uncond}} + (w+1)\,(L^{(k)}_{\mathrm{cond}} - L^{(k)}_{\mathrm{uncond}})$ then amplifies the difference between full context and uncertainty-erased context. What makes the mechanism work is that the guidance direction is localized: only positions the model itself flags as ambiguous are erased, so the corrective signal is concentrated rather than smeared over a fully masked sequence.

What would settle it

Run A-CFG on LLaDA 8B and replace the low-confidence selection with random selection of the same number of tokens each step; if random re-masking matches A-CFG's GPQA and Sudoku scores, the confidence ranking is not what drives the gains.

Watch

Extended reading notes

Core claim

The central claim is that the unconditional prediction in CFG for masked language diffusion can be made adaptive at no extra training cost, and that this adaptation is what drives better conditional generation. At each denoising step the model computes conditional logits, scores every non-masked token by the maximum softmax probability, re-masks the lowest-scoring fraction rho of those tokens, and runs the model again on this locally 'unconditional' input before applying the usual CFG interpolation. The paper reports that this one change consistently outperforms standard static-mask CFG on LLaDA 8B across general, math, science, and planning benchmarks, and that it transfers to Dream-7B. The mechanism is a direct comparison of the model's confident predictions against its own predictions under erased uncertainty, rather than against a generic null context.

Load-bearing premise

The method assumes the model's maximum-softmax confidence on already-visible tokens is a reliable signal for which tokens actually need reconsidering, even though the model was trained to predict masked positions.

Editorial extensions

If this is right

  • A-CFG turns CFG from a fixed interpolation into a per-step, per-token intervention, so guidance strength can be spent where the model is currently unsure.
  • The reported gains appear without modifying or retraining the underlying diffusion model, only by changing how its own logits select the unconditional input.
  • The same recipe transfers to a second masked diffusion model, Dream-7B, suggesting the mechanism is not specific to LLaDA.
  • With $\rho=0.7$ as the best re-masking proportion on ARC-C, the method's benefit saturates and then degrades at $\rho=0.9$, so there is a usable operating range for the intervention.
  • On Sudoku and GPQA, the margin over standard CFG is large enough to change how diffusion language models compare with autoregressive baselines on reasoning-style tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If visible-position logits are only weakly calibrated, the confidence score may be selecting tokens for reasons other than true error; a random re-masking control at matched $\rho$ would separate the confidence signal from the simple act of perturbing the input.
  • The same dynamic-unconditional principle could apply to other discrete diffusion families or to continuous diffusion, where the 'unconditional' input is usually a fixed null embedding.
  • Entropy of the predicted distribution, or a lower-bound estimate of token-level error from multiple sampling passes, could replace maximum softmax probability and may improve selection quality at later steps.
  • Since re-masking consumes an extra forward pass, the cost is roughly one additional model evaluation per step, and this overhead may be reducible by reusing the conditional computation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Adaptive Classifier-Free Guidance (A-CFG) for iterative masked diffusion language models. At each denoising step, A-CFG computes the model's confidence on already-visible (non-mask) tokens, re-masks the lowest-confidence fraction rho of those tokens, and uses this dynamically constructed sequence as the unconditional input in the standard CFG interpolation formula. The method is integrated into LLaDA 8B and Dream-7B and evaluated on general understanding, mathematics/science reasoning, and planning benchmarks. The main claim is that A-CFG consistently outperforms both no-guidance generation and standard static CFG, with headline gains such as +3.9 on GPQA and +8.0 on Sudoku for LLaDA.

Significance. If the empirical claims are validated, A-CFG is a simple and potentially useful contribution: it replaces a static CFG unconditioning term with a cheap, token-local dynamic input, and it could apply to any iterative masked language model. The paper ships code, evaluates on a broad benchmark suite, and includes ablation studies over rho and w. However, the central causal claim is not yet established. The confidence signal at visible tokens is not validated, no control compares confidence-based re-masking with random re-masking, and the benchmark table reports single runs with several differences that are likely within noise. The idea is worth pursuing, but the evidence in this version is insufficient to support the stated conclusion.

major comments (4)
  1. [Section 3.2.1, Algorithm 1 lines 6-8] The confidence score c_j = max_v softmax(L_cond)_{j,v} is computed at non-mask token positions, but LLaDA is trained with a masked-language-modeling loss that supervises only [MASK] positions. The logits at visible positions receive no training signal, so the paper's assertion that a low c_j indicates uncertainty about the visible token is unsupported. This is load-bearing because the adaptive selection is the entire claimed contribution. Please add a control that re-masks a uniformly random set of rho non-mask tokens under the same guidance scale and evaluation protocol, and, if possible, compare against an oracle selection based on actual generation errors. If random re-masking matches A-CFG's gains, the 'adaptive' mechanism is not causal.
  2. [Table 1 and Section 4.2] All benchmark numbers are reported as single runs with no error bars, no number of seeds, and no significance tests. Several improvements over Std CFG are small, e.g., MMLU +0.3 (66.1 vs 65.8), Dream-7B ARC-C +1.0 (60.8 vs 59.8), and LLaDA WinoGrande +0.8 (75.9 vs 75.1). Given the per-task hyperparameter selection described in Section 4.1.3, these differences may reflect tuning or evaluation noise. Please report at least three seeds with means and standard deviations, or use paired bootstrap or permutation tests over benchmark examples for the key Std-CFG-versus-A-CFG comparisons.
  3. [Section 4.1.3] The hyperparameter protocol is self-contradictory. The text states that w is 'selected from {0.5,1.0,1.5,2.0} based on performance on the validation set of each respective task' and then immediately states that 'the same w is kept fixed across all downstream benchmarks for that model.' These statements cannot both be true. This matters because if w is tuned independently per benchmark and per method, the Table 1 comparison is not a like-for-like comparison of the unconditioning mechanism. Please state the protocol unambiguously and report the selected w values for each model and benchmark.
  4. [Algorithm 1 line 4 and Section 3.2.1] The remaskable set is defined as all indices with (x^(k))_j != [MASK], and Section 3.2.1 says x^(k) includes the prompt c. Under this definition, prompt tokens are eligible for re-masking, which would erase the conditioning signal. If the implementation actually excludes prompt positions, that exclusion must be stated explicitly in Algorithm 1 and in Equation (4); otherwise the method can destroy the very condition it is meant to guide on.
minor comments (5)
  1. [Equation (1) and Table 2b] With the (w+1) convention used in Equation (1), setting w=0 gives L_guided = L_cond exactly, so adaptive re-masking has no effect on the guided logits. The text's description of w=0 as 'no CFG guidance beyond the adaptive masking' is therefore confusing and should be reconciled with the formula.
  2. [Table 2 caption] The caption says '(a) Impact of guidance scale (w). (b) Impact of adaptive re-masking proportion (rho)', but the table body labels the panels in the opposite order: (a) is Re-masking Proportion and (b) is Guidance Scale. Please correct the mismatch.
  3. [Section 4.1.3] The sentence 'The confidence for token selection in A-CFG is based on the softmax probability of the predicted token at each masked position' contradicts Algorithm 1, which computes confidence for non-masked positions. Please correct this sentence.
  4. [Section 4.1.1] MATH is listed as one of the evaluation benchmarks, but no MATH results appear anywhere in Table 1 or the surrounding text. Either add the results or remove MATH from the dataset list.
  5. [References and Table 1] Qwen2 7B scores are cited to reference [6], which is the Qwen2-Audio technical report. Please verify that this is the correct source for text-only LLM benchmark numbers, and correct the citation if necessary.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: A-CFG's confidence-based re-masking is a heuristic input to CFG, and the reported gains are empirical comparisons, not quantities derived from the method's own equations.

full rationale

The paper's claimed derivation chain is the standard CFG interpolation formula, with the unconditional input constructed by re-masking low-confidence tokens. No benchmark result is algebraically forced by the method's definitions: the confidence score c_j is computed from the model's conditional logits, and the selected tokens are then used to build a modified input, but the reported accuracy improvements are external empirical outcomes, not consequences of Equations (1)-(6) alone. The method does rely on the untested assumption that max-softmax confidence at visible tokens is meaningful in LLaDA, but that is a correctness and calibration concern, not a circularity: no fitted parameter is renamed as a prediction, and no quantity reduces by construction to another quantity it is supposed to predict. The paper contains numerous self-citations, but they appear in the related-work enumeration (e.g., [1], [10], [16], [17], [19], [22], [24], [37]-[41]) and are not load-bearing for the A-CFG mechanism; the central references for the base models (LLaDA [26], Dream-7B [43]) are external to the author set. The absence of a random-remasking control is a significant experimental limitation, but it does not make the argument circular. Overall, the paper's contribution is an empirical heuristic with an unvalidated confidence signal, not a derivation that is equivalent to its own inputs.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The method introduces no new physical or conceptual entities. Its load-bearing assumptions are about using visible-token logits as confidence and treating a partially masked sequence as an unconditional input. Two free hyperparameters (rho and w) are chosen from data.

free parameters (2)
  • re-masking proportion rho = 0.7
    Fraction of non-mask tokens re-masked in the unconditional input; set to 0.7 from ARC-C ablation and kept fixed across all benchmarks (Section 4.1.3, Table 2a).
  • guidance scale w = 0.5 for ARC-C, selected from {0.5,1.0,1.5,2.0}
    Guidance strength in the CFG formula; tuned on a validation set but per-task values are not reported (Section 4.1.3, Table 2b).
assumptions (4)
  • domain assumption The model's softmax probability at visible (non-mask) positions is a valid confidence signal for token quality.
    Algorithm 1 lines 6-8; no evidence that these logits are calibrated, since training loss only covers masked positions.
  • domain assumption Replacing low-confidence tokens with [MASK] yields an input whose model prediction can stand as the unconditional prediction in CFG.
    Section 3.2.1; the input still contains the prompt and high-confidence tokens, so it is not truly unconditional.
  • domain assumption The standard CFG formula (Eq. 1) remains applicable when the unconditional input is constructed adaptively.
    Eq. 6; no theoretical justification is given for why the interpolation should behave like CFG with a static unconditional input.
  • domain assumption The base models LLaDA and Dream-7B are fixed and used without modification.
    Section 4.1; no training or fine-tuning is performed, only inference-time guidance.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Adaptive Classifier-Free Guidance via Dynamic Low-Confidence Masking." pith.science (2026). https://pith.science/paper/LBTHEC6Q

@misc{pith2026250520199,
  author       = {Pith},
  title        = {Pith review of: Adaptive Classifier-Free Guidance via Dynamic Low-Confidence Masking},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LBTHEC6Q}},
  note         = {Machine review of arXiv:2505.20199}
}
read the original abstract

Classifier-Free Guidance (CFG) significantly enhances controllability in generative models by interpolating conditional and unconditional predictions. However, standard CFG often employs a static unconditional input, which can be suboptimal for iterative generation processes where model uncertainty varies dynamically. We introduce Adaptive Classifier-Free Guidance (A-CFG), a novel method that tailors the unconditional input by leveraging the model's instantaneous predictive confidence. At each step of an iterative (masked) diffusion language model, A-CFG identifies tokens in the currently generated sequence for which the model exhibits low confidence. These tokens are temporarily re-masked to create a dynamic, localized unconditional input. This focuses CFG's corrective influence precisely on areas of ambiguity, leading to more effective guidance. We integrate A-CFG into a state-of-the-art masked diffusion language model and demonstrate its efficacy. Experiments on diverse language generation benchmarks show that A-CFG yields substantial improvements over standard CFG, achieving, for instance, a 3.9 point gain on GPQA. Our work highlights the benefit of dynamically adapting guidance mechanisms to model uncertainty in iterative generation.

Figures

Figures reproduced from arXiv: 2505.20199 by the authors.

Figure 1
Figure 1. Overview of model confidence dynamics during iterative generation. (a) Token-level confidence heatmap across token positions and generation steps (darker shades indicate higher confidence). (b) Average and minimum confidence scores per generation step. This visualization highlights the dynamic and non-uniform nature of model confidence that A-CFG aims to leverage. However, the conventional application of CFG within … view at source ↗
Figure 2
Figure 2. Overview of (left) standard Null Prompt Classifier-Free Guidance and (right) our proposed Adaptive Classifier-Free Guidance (A-CFG) at a single generation step k. In standard CFG, the unconditional input often involves masking the entire prompt or using a null prompt. In A-CFG, after computing conditional logits from x (k) , token-level confidences for all non-[MASK] tokens in x (k) are assessed. Tokens with low con… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Rethinking Visual Autoregressive Sampling with Information-Grounding Guidance

    cs.CV 2025-09 conditional novelty 5.0 of 10

    IGG, an attention-based reweighting of classifier-free guidance, concentrates guidance on important tokens and modestly improves FID/IS in scale-wise autoregressive image generation.

  2. Images Speak Louder Than Scores: Failure Mode Escape for Enhancing Generative Quality

    cs.CV 2025-08 reject novelty 5.0 of 10

    FaME uses stored trajectories of IQA-selected low-quality images as negative guidance to improve perceptual quality of diffusion generation while keeping FID.

  3. Undress to Redress: A Training-Free Framework for Virtual Try-On

    cs.CV 2025-08 unverdicted novelty 5.0 of 10

    UR-VTON improves long-sleeve to short-sleeve virtual try-on by splitting the task into a virtual undressing step followed by dressing the bare torso.

  4. MINT-CoT: Enabling Interleaved Visual Tokens in Mathematical Chain-of-Thought Reasoning

    cs.CV 2025-06 conditional novelty 5.0 of 10

    MINT-CoT-7B interleaves fine-grained visual tokens into each math reasoning step and reports 73.70 on MathVista-Math, 64.72 on GeoQA, and 69.6 on MMStar-Math.

  5. A Survey on Diffusion Language Models

    cs.CL 2025-08 unverdicted novelty 3.0 of 10

    A comprehensive survey of diffusion language models covering taxonomy, training and inference techniques, and comparisons with autoregressive models.

Reference graph

Works this paper leans on

44 extracted references · 7 canonical work pages · cited by 5 Pith papers

  1. [1]

    Mc-llava: Multi-concept personalized vision-language model

    Ruichuan An, Sihan Yang, Ming Lu, Renrui Zhang, Kai Zeng, Yulin Luo, Jiajun Cao, Hao Liang, Ying Chen, Qi She, et al. Mc-llava: Multi-concept personalized vision-language model. arXiv preprint arXiv:2411.11706, 2024. 10

    arXiv 2024

  2. [2]

    Structured denoising diffusion models in discrete state-spaces.Advances in Neural Information Processing Systems, 34:17981–17993, 2021

    Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne Van Den Berg. Structured denoising diffusion models in discrete state-spaces.Advances in Neural Information Processing Systems, 34:17981–17993, 2021

    2021

  3. [3]

    Openflamingo: An open- source framework for training large autoregressive vision-language models.arXiv preprint arXiv:2308.01390, 2023

    Anas Awadalla, Irena Gao, Josh Gardner, Jack Hessel, Yusuf Hanafy, Wanrong Zhu, Kalyani Marathe, Yonatan Bitton, Samir Gadre, Shiori Sagawa, et al. Openflamingo: An open- source framework for training large autoregressive vision-language models.arXiv preprint arXiv:2308.01390, 2023

    arXiv 2023

  4. [4]

    Piqa: Reasoning about phys- ical commonsense in natural language

    Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. Piqa: Reasoning about phys- ical commonsense in natural language. InProceedings of the AAAI conference on artificial intelligence, 2020

    work page 2020

  5. [5]

    Language models are few-shot learners.arXiv preprint arXiv:2005.14165, 2020

    Tom B Brown. Language models are few-shot learners.arXiv preprint arXiv:2005.14165, 2020

    arXiv 2005

  6. [6]

    Qwen2-audio technical report.arXiv preprint arXiv:2407.10759, 2024

    Yunfei Chu, Jin Xu, Qian Yang, Haojie Wei, Xipin Wei, Zhifang Guo, Yichong Leng, Yuan- jun Lv, Jinzheng He, Junyang Lin, et al. Qwen2-audio technical report.arXiv preprint arXiv:2407.10759, 2024

    arXiv 2024

  7. [7]

    Think you have solved question answering? try arc, the ai2 reasoning challenge.arXiv preprint arXiv:1803.05457, 2018

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge.arXiv preprint arXiv:1803.05457, 2018

    arXiv 2018

  8. [8]

    Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    arXiv

Show all 44 references
  1. [9]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780–8794, 2021

    2021

  2. [10]

    Instructseq: Unifying vision tasks with instruction-conditioned multi-modal sequence generation.arXiv preprint arXiv:2311.18835, 2023

    Rongyao Fang, Shilin Yan, Zhaoyang Huang, Jingqiu Zhou, Hao Tian, Jifeng Dai, and Hong- sheng Li. Instructseq: Unifying vision tasks with instruction-conditioned multi-modal sequence generation.arXiv preprint arXiv:2311.18835, 2023

    2023 arXiv

  3. [11]

    Diffuseq: Sequence to sequence text generation with diffusion models.arXiv preprint arXiv:2210.08933, 2022

    Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu, and LingPeng Kong. Diffuseq: Sequence to sequence text generation with diffusion models.arXiv preprint arXiv:2210.08933, 2022

    2022 arXiv

  4. [12]

    Measuring massive multitask language understanding.arXiv preprint arXiv:2009.03300, 2020

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding.arXiv preprint arXiv:2009.03300, 2020

    2009 arXiv

  5. [13]

    Measuring mathematical problem solving with the math dataset

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874, 2021

    2021 arXiv

  6. [14]

    Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022

    2022 arXiv

  7. [15]

    Denoising diffusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    2020

  8. [16]

    Worldsense: Evaluat- ing real-world omnimodal understanding for multimodal llms.arXiv preprint arXiv:2502.04326, 2025

    Jack Hong, Shilin Yan, Jiayin Cai, Xiaolong Jiang, Yao Hu, and Weidi Xie. Worldsense: Evaluat- ing real-world omnimodal understanding for multimodal llms.arXiv preprint arXiv:2502.04326, 2025

    2025 arXiv

  9. [17]

    Onetracker: Unifying visual object tracking with foundation models and efficient tuning

    Lingyi Hong, Shilin Yan, Renrui Zhang, Wanyun Li, Xinyu Zhou, Pinxue Guo, Kaixun Jiang, Yiting Chen, Jinglun Li, Zhaoyu Chen, et al. Onetracker: Unifying visual object tracking with foundation models and efficient tuning. InProceedings of the IEEE/CVF conference on computer vi...

    2024

  10. [18]

    Mistral 7b.arXiv preprint arXiv:2310.06825, 2023

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b.arXiv preprint arXiv:2310.06825, 2023. 11

    arXiv

  11. [19]

    Llava-onevision: Easy visual task transfer.arXiv preprint arXiv:2408.03326, 2024

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer.arXiv preprint arXiv:2408.03326, 2024

    2024 arXiv

  12. [20]

    Diffusion-lm improves controllable text generation.Advances in Neural Information Processing Systems, 35:4328–4343, 2022

    Xiang Li, John Thickstun, Ishaan Gulrajani, Percy S Liang, and Tatsunori B Hashimoto. Diffusion-lm improves controllable text generation.Advances in Neural Information Processing Systems, 35:4328–4343, 2022

    2022

  13. [21]

    Truthfulqa: Measuring how models mimic human falsehoods.arXiv preprint arXiv:2109.07958, 2021

    Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human falsehoods.arXiv preprint arXiv:2109.07958, 2021

    2021 arXiv

  14. [22]

    Draw-and-understand: Leveraging visual prompts to enable MLLMs to comprehend what you want

    Weifeng Lin, Xinyu Wei, Ruichuan An, Peng Gao, Bocheng Zou, Yulin Luo, Siyuan Huang, Shanghang Zhang, and Hongsheng Li. Draw-and-understand: Leveraging visual prompts to enable MLLMs to comprehend what you want. InThe Thirteenth International Conference on Learning Representat...

    2025

  15. [23]

    Diffusion guided language modeling.arXiv preprint arXiv:2408.04220, 2024

    Justin Lovelace, Varsha Kishore, Yiwei Chen, and Kilian Q Weinberger. Diffusion guided language modeling.arXiv preprint arXiv:2408.04220, 2024

    2024 arXiv

  16. [24]

    Ee-mllm: A data-efficient and compute-efficient multi- modal large language model.arXiv preprint arXiv:2408.11795, 2024

    Feipeng Ma, Yizhou Zhou, Zheyu Zhang, Shilin Yan, Hebei Li, Zilong He, Siying Wu, Fengyun Rao, Yueyi Zhang, and Xiaoyan Sun. Ee-mllm: A data-efficient and compute-efficient multi- modal large language model.arXiv preprint arXiv:2408.11795, 2024

    2024 arXiv

  17. [25]

    Scaling up masked diffusion models on text.arXiv preprint arXiv:2410.18514, 2024

    Shen Nie, Fengqi Zhu, Chao Du, Tianyu Pang, Qian Liu, Guangtao Zeng, Min Lin, and Chongxuan Li. Scaling up masked diffusion models on text.arXiv preprint arXiv:2410.18514, 2024

    2024 arXiv

  18. [26]

    Large language diffusion models.arXiv preprint arXiv:2502.09992, 2025

    Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. Large language diffusion models.arXiv preprint arXiv:2502.09992, 2025

    2025 arXiv

  19. [27]

    Language models are unsupervised multitask learners.OpenAI blog, 1(8):9, 2019

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners.OpenAI blog, 1(8):9, 2019

    2019

  20. [28]

    Gpqa: A graduate-level google-proof q&a benchmark.arXiv preprint arXiv:2311.12022, 2023

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. Gpqa: A graduate-level google-proof q&a benchmark.arXiv preprint arXiv:2311.12022, 2023

    2023 arXiv

  21. [29]

    High- resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022

    2022

  22. [30]

    Simple and effective masked diffusion language models.arXiv preprint arXiv:2406.07524, 2024

    Subham Sekhar Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin T Chiu, Alexander Rush, and V olodymyr Kuleshov. Simple and effective masked diffusion language models.arXiv preprint arXiv:2406.07524, 2024

    2024 arXiv

  23. [31]

    Winogrande: An adversarial winograd schema challenge at scale.Communications of the ACM, 64(9):99–106, 2021

    Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale.Communications of the ACM, 64(9):99–106, 2021

    2021

  24. [32]

    Simplified and generalized masked diffusion for discrete data.arXiv preprint arXiv:2406.04329, 2024

    Jiaxin Shi, Kehang Han, Zhe Wang, Arnaud Doucet, and Michalis K Titsias. Simplified and generalized masked diffusion for discrete data.arXiv preprint arXiv:2406.04329, 2024

    2024 arXiv

  25. [33]

    Deep unsuper- vised learning using nonequilibrium thermodynamics

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsuper- vised learning using nonequilibrium thermodynamics. InInternational conference on machine learning, pages 2256–2265. PMLR, 2015

    2015

  26. [34]

    Challenging big- bench tasks and whether chain-of-thought can solve them.arXiv preprint arXiv:2210.09261, 2022

    Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V Le, Ed H Chi, Denny Zhou, et al. Challenging big- bench tasks and whether chain-of-thought can solve them.arXiv preprint arXiv:2210.09261, 2022

    arXiv

  27. [35]

    Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971, 2023

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timo- thée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971, 2023. 12

    2023 arXiv

  28. [36]

    Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288, 2023

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  29. [37]

    Dynaprompt: Dynamic test-time prompt tuning.arXiv preprint arXiv:2501.16404, 2025

    Zehao Xiao, Shilin Yan, Jack Hong, Jiayin Cai, Xiaolong Jiang, Yao Hu, Jiayi Shen, Qi Wang, and Cees GM Snoek. Dynaprompt: Dynamic test-time prompt tuning.arXiv preprint arXiv:2501.16404, 2025

    2025 arXiv

  30. [38]

    A sanity check for ai-generated image detection.arXiv preprint arXiv:2406.19435, 2024

    Shilin Yan, Ouxiang Li, Jiayin Cai, Yanbin Hao, Xiaolong Jiang, Yao Hu, and Weidi Xie. A sanity check for ai-generated image detection.arXiv preprint arXiv:2406.19435, 2024

    2024 arXiv

  31. [39]

    Panovos: Bridging non-panoramic and panoramic views with transformer for video segmentation

    Shilin Yan, Xiaohao Xu, Renrui Zhang, Lingyi Hong, Wenchao Chen, Wenqiang Zhang, and Wei Zhang. Panovos: Bridging non-panoramic and panoramic views with transformer for video segmentation. InEuropean Conference on Computer Vision, pages 346–365. Springer, 2024

    2024

  32. [40]

    Referred by multi-modality: A unified temporal transformer for video object segmentation

    Shilin Yan, Renrui Zhang, Ziyu Guo, Wenchao Chen, Wei Zhang, Hongyang Li, Yu Qiao, Hao Dong, Zhongjiang He, and Peng Gao. Referred by multi-modality: A unified temporal transformer for video object segmentation. InProceedings of the AAAI Conference on Artificial Intelligence, ...

    2024

  33. [41]

    Crosslmm: Decoupling long video sequences from lmms via dual cross-attention mechanisms.arXiv preprint arXiv:2505.17020, 2025

    Shilin Yan, Jiaming Han, Joey Tsai, Hongwei Xue, Rongyao Fang, Lingyi Hong, Ziyu Guo, and Ray Zhang. Crosslmm: Decoupling long video sequences from lmms via dual cross-attention mechanisms.arXiv preprint arXiv:2505.17020, 2025

    2025

  34. [42]

    Beyond autoregression: Discrete diffusion for complex reasoning and planning.arXiv preprint arXiv:2410.14157, 2024

    Jiacheng Ye, Jiahui Gao, Shansan Gong, Lin Zheng, Xin Jiang, Zhenguo Li, and Lingpeng Kong. Beyond autoregression: Discrete diffusion for complex reasoning and planning.arXiv preprint arXiv:2410.14157, 2024

    2024 arXiv

  35. [43]

    Dream 7b, 2025

    Jiacheng Ye, Zhihui Xie, Lin Zheng, Jiahui Gao, Zirui Wu, Xin Jiang, Zhenguo Li, and Lingpeng Kong. Dream 7b, 2025. URLhttps://hkunlp.github.io/blog/2025/dream

    2025

  36. [44]

    Hellaswag: Can a machine really finish your sentence?arXiv preprint arXiv:1905.07830, 2019

    Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence?arXiv preprint arXiv:1905.07830, 2019. 13

    1905 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.