Pith. sign in

REVIEW 4 major objections 5 minor 60 references

A parameter-free entropy metric, computed from a single denoising run, detects hallucinated responses in diffusion language models as reliably as a trained detector.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 09:35 UTC pith:MH3RHCTS

load-bearing objection A genuinely new and plausible training-free hallucination detector for D-LLMs; the core effect looks real, but design choices and evaluation share the same data, so the headline AUROC numbers need verification. the 4 major comments →

arxiv 2607.22661 v1 pith:MH3RHCTS submitted 2026-06-29 cs.AI

TRE: Training-Free Hallucination Detection for Diffusion Language Models

Pengcheng Weng , Yanyu Qian , Yue Tan , Yixin Liu This is my paper
classification cs.AI
keywords hallucination detectiondiffusion language modelstraining-free metricpredictive entropyrevealing tokensuncertainty estimationTREquestion answering
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Diffusion language models generate text by progressively denoising a masked canvas, so they carry uncertainty signals that autoregressive detectors never see. This paper claims that the entropy of the tokens being revealed at each denoising step—weighted more heavily at later steps—is enough to tell hallucinated answers from correct ones. The resulting metric, TRE, needs no detector training, no labeled data, and no repeated sampling: it is read off the very generation being checked. On three QA benchmarks and two diffusion models, the paper reports that TRE matches or exceeds the leading trained detector while remaining far cheaper. If the claim holds, hallucination screening for diffusion LLMs becomes a free by-product of decoding.

Core claim

The central claim is that hallucination risk in D-LLMs lands on a specific set of tokens—those transitioning from masked to determined at each denoising step—and that their predictive entropy, summed and linearly weighted by step index, is a sufficient proxy for whether the answer is wrong. The paper divides tokens into unrevealed, revealing, and revealed states, shows with effect-size analysis that revealing-token entropy separates hallucinated from faithful samples best, and shows that the separation concentrates in late steps. TRE is defined as the sum over denoising steps of (t/T) times the total entropy of revealing tokens at that step. The paper reports average AUROC 82.7 vs 72.0 on LL

What carries the argument

The central object is the TRE score built from a revelation-state categorization of tokens. At each denoising step, tokens are either unrevealed (still masked), revealing (their placeholder becomes a concrete token at this step), or revealed (fixed earlier). TRE sums the predictive entropy of exactly the revealing tokens across all steps, multiplying each step's entropy mass by a linear weight t/T. This isolates the moving boundary where uncertainty becomes committed answer content, and the temporal weight makes late commitments dominate without adding a single learned parameter.

Load-bearing premise

The metric presumes that wrong answers are accompanied by elevated predictive entropy when their tokens are revealed, so a fluent, confident fabrication produces no signal for it to catch.

What would settle it

Create a held-out set of QA responses judged wrong by human annotators whose late-step revealing-token entropy is no higher than the entropy of correct responses. If TRE's AUROC on this set falls to near chance, the monotone entropy–risk assumption fails and the metric collapses on precisely the most dangerous errors.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Hallucination detection becomes free: computing TRE adds no extra model call, no repeated sampling, and no labeled data to the generation pipeline.
  • The metric transfers to any new diffusion model or domain immediately, since it has no trained parameters and reads only token-level entropy.
  • Because TRE can be evaluated during decoding, it could gate abstention or trigger verification only for high-risk generations, reducing the cost of safeguards.
  • The finding reframes D-LLM hallucination as a late-commitment phenomenon, pointing subsequent detectors toward reveal-boundary signals rather than whole-sequence statistics.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If later steps truly dominate, a data-driven weighting (e.g., a fitted schedule or a learned scorer over steps) could squeeze more accuracy from the same evidence—an extension the paper leaves untested.
  • The same reveal-boundary entropy flux could transfer to other discrete diffusion settings, such as code generation or structured prediction, where tokens are also revealed in batches.
  • TRE's blind spot—confident factual errors—could be patched by routing only the uncertain tail of generations to an external knowledge check, yielding a two-stage pipeline the paper does not propose.
  • A direct calibration test of the paper's monotonicity assumption (Eq. 28) would clarify when TRE should and should not be trusted; the paper leaves this unmeasured.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes TRE, a training-free hallucination detection metric for diffusion large language models (D-LLMs). The score is defined in Eq. (4) as a linear temporal-weighted sum over denoising steps of the entropy mass of 'revealing' tokens (tokens transitioning from masked to determined at each step). The two design decisions—using revealing-token entropy rather than unrevealed/revealed/all tokens, and weighting late steps more heavily via w_t=t/T—are motivated by empirical analyses in Secs. 4.1–4.2. The paper evaluates TRE on LLaDA-8B-Instruct and Dream-7B-Instruct across TriviaQA, HotpotQA, and CommonsenseQA, reporting AUROC comparable to or better than TraceDet, a state-of-the-art training-based D-LLM detector. The paper also includes a statistical-physics-inspired interpretation of the metric in Sec. 4.3 and Appendix B, plus ablations, sensitivity analyses, efficiency comparisons, and case studies.

Significance. If validated, TRE would be a valuable contribution: it is a closed-form, single-run, training-free metric that requires no additional detector training and no repeated sampling, making it attractive for gray-box deployment on D-LLMs. The paper is honest about the entropy-only limitation and provides a thoughtful, if stylized, theoretical narrative. Strengths include the explicit mathematical definition, the extensive empirical motivation, the ablation study showing which design choices matter, and the release of anonymized code. However, the reported performance is undermined by two evaluation issues: the design appears to be selected on the same datasets used for final evaluation, and no uncertainty estimates are provided, which is critical given the near-tie with TraceDet on Dream. The entropy-risk monotonicity assumption is also load-bearing and untested. These issues need to be addressed before the central claim is fully convincing.

major comments (4)
  1. [Sec. 4.1–4.2, Table 2 vs Table 1] The design of TRE is effectively selected on the evaluation data. The choice of revealing-token entropy (Sec. 4.1, Fig. 2) and the linear temporal weighting (Sec. 4.2, Fig. 3) are motivated using the same datasets (TriviaQA, HotpotQA, CSQA) and backbones (LLaDA-8B, Dream-7B) that later appear in Table 1. Table 2 shows that these choices have a large effect; e.g., on TriviaQA, replacing revealing tokens with revealed tokens changes AUROC from 82.2 to 53.9. The reported AUROC is therefore an optimistic estimate of a prespecified metric. Please evaluate TRE on a held-out design-validation split or on new benchmarks, or report the full set of design variants without selection.
  2. [Table 1, Sec. 5.2] AUROC values are reported as point estimates without confidence intervals or significance tests. On Dream, TRE (81.1) and TraceDet (80.8) are nearly identical; without error bars or a paired test, the claim that TRE is 'comparable or even better' than TraceDet is not statistically supported. Report bootstrap or other confidence intervals and, since all detectors are evaluated on the same generated responses, a McNemar or permutation test.
  3. [Eq. (28), Sec. 6, Appendix E] The metric relies on the assumption that commitment risk is monotonically nondecreasing in predictive entropy (Eq. 28). The paper acknowledges this in Sec. 6, but it is load-bearing for the central claim. This assumption should be tested directly on the evaluated data, e.g., by comparing entropy distributions of hallucinated vs non-hallucinated samples or by reporting calibration of entropy against the automatic labels. Without such a diagnostic, the reported AUROC may reflect dataset-specific alignment of entropy and factuality.
  4. [Sec. 1, Sec. 4.2] The characterization 'parameter-free' is potentially misleading. TRE has no fitted numeric parameters, but the evidence source (revealing tokens) and the weighting schedule (w_t=t/T) are empirical choices with strong effects (Table 2). The paper should clarify that 'parameter-free' means 'no learned parameters' and should treat these design choices as components that require independent validation.
minor comments (5)
  1. [Table 2] Abbreviated column headers 'TriQA' and 'HotQA' should be spelled out as 'TriviaQA' and 'HotpotQA' for clarity.
  2. [Appendix C.4] There is a spacing typo in 'GPT-4o minia' (should be 'GPT-4o mini as'). Please proofread.
  3. [Eqs. (2)–(3)] The empirical analysis in Eq. (2) uses average entropy per token group, while the final metric in Eq. (3) uses entropy mass. The rationale is given, but this switch makes the empirical motivation not perfectly aligned with the final metric; please make the distinction and its justification more explicit.
  4. [Appendix B.2, Eq. (21)] The theoretical interpretation assumes confidence-based reveal (Eq. 21). This should be stated clearly as an assumption and, if possible, verified for the decoding configuration used in the experiments (deterministic decoding, temperature 0).
  5. [Appendix C.4] The annotation-filtering procedure discards examples with inconsistent first and second judgments. Please report the number of discarded examples and the final dataset sizes for reproducibility.

Circularity Check

1 steps flagged

TRE's two design choices are selected on the same benchmarks/backbones used for the headline AUROC, so the reported comparison to TraceDet is partly an in-sample restatement of the selection signal rather than a fully prespecified training-free metric.

specific steps
  1. fitted input called prediction [Sec. 4.1, Sec. 4.2, Eq. (4); headline comparison in Sec. 5.2, Table 1]
    "According to the results shown in Fig. 2 ... ¯H_g consistently achieves the best discriminateness ... we employ H^g_t as the primary evidence for TRE. ... The visualization results are shown in Fig. 3 ... later diffusion steps are more informative ... we adopt the simplest parameter-free rule, w_t = t/T. ... TRE shows comparable or even better performance compared to TraceDet."

    The metric in Eq. (4) is a direct aggregation of revealing-token entropy mass weighted by w_t=t/T. Both design components—restricting the evidence to revealing tokens and emphasizing late steps—were selected by inspecting Cohen's d (Fig. 2) and trajectory gaps (Fig. 3) computed with hallucination labels on the same backbones/datasets (LLaDA, Dream; TriviaQA, HotpotQA, CSQA) used for the headline AUROC in Table 1. No design-validation split is reported. Thus the reported TRE score is a monotone summary of the very entropy-label association used to choose the metric's form, so the Table 1 comparison to TraceDet is not an independent out-of-sample test of a prespecified training-free metric; it partly restates the Fig. 2/3 selection signal in AUROC form.

full rationale

TRE is a closed-form weighted entropy sum with no numeric fitting, so most of the derivation chain (Eq. 1 -> Eq. 3 -> Eq. 4) is definitional rather than circular. The Appendix B statistical-physics story is explicitly labeled an analogy and not a rigorous derivation; Eq. 28 (monotone entropy-risk) is an untested assumption the paper honestly flags in Sec. 6, not a circular step. The self-citations (e.g., [10]) are not load-bearing: the authors' own Fig. 2/3, not the citation, justify the design. The material circularity is methodological: the choice of revealing-token evidence and late-step weighting was made from the same LLaDA/Dream, TriviaQA/HotpotQA/CSQA data that Table 1 evaluates. Thus 'TRE (ours)' is not a prespecified fixed metric but a design selected on the test sets; the headline AUROC partly reports the success of that selection. This does not make Eq. 4 wrong or its entropy signal uninformative, but it means the claimed superiority over TraceDet is not an out-of-sample validation. Score 5 reflects partial circularity: the central performance claim reduces in part to the data used to choose the metric form, while the metric itself still has independent, parameter-free content.

Axiom & Free-Parameter Ledger

2 free parameters · 3 axioms · 0 invented entities

The metric itself introduces no fitted scalar and no new theoretical entity: 'reveal-boundary entropy flux' is a relabeling of a directly computable quantity. The two free parameters are the linear temporal weighting schedule w_t=t/T and the late-stage window S, both chosen qualitatively after inspecting the evaluation datasets. No new particles, forces, or statistical-mechanical degrees of freedom are postulated.

free parameters (2)
  • Temporal weight schedule w_t = t/T = t/T (hand-chosen, no numeric fit)
    Chosen because later steps were observed to be more discriminative on the same three evaluation datasets (Sec. 4.2, Fig. 3). The paper calls the method parameter-free, but the linear schedule is a design decision informed by the evaluation data; exponential and hard-threshold alternatives are ablated (Table 2) but rejected for simplicity.
  • Late-stage window S = last 30% of denoising steps = 30%
    Used in the motivating Cohen's d analysis (Sec. 4.1, Eq. 2, and Appendix A.1) to justify revealing-token evidence. The 30% cutoff is arbitrary and is chosen on the same datasets used for final AUROC evaluation.
axioms (3)
  • domain assumption Per-step categorical distributions π_t^(i) over the vocabulary are accessible for every position and every denoising step.
    TRE is defined directly on these entropies (Eq. 1 and Eq. 3). The paper notes this is a gray-box requirement; purely black-box APIs without token logits would not support it.
  • domain assumption Commitment risk is monotonically nondecreasing in predictive entropy (Eq. 28, Appendix B.2: r_i(t) ≈ ψ(h_i(t)), ψ' ≥ 0).
    The entire signal reduces to entropy; if factual error and entropy are not monotonically linked, TRE carries no information. The paper acknowledges confident hallucinations as a failure mode in Sec. 6 but never tests the monotonicity directly.
  • domain assumption GPT-4o-mini judgments with the conservative re-evaluation filter produce reliable binary ground-truth labels.
    All AUROC scores in Table 1 depend on these labels. The filtering rule that discards examples with inconsistent judgments (Appendix C.2) can bias the evaluation set and inflate measured separation.

pith-pipeline@v1.3.0-alltime-deepseek · 22120 in / 11244 out tokens · 104642 ms · 2026-08-02T09:35:49.836123+00:00 · methodology

0 comments
read the original abstract

Diffusion large language models (D-LLMs) have recently gained increasing attention, yet their reliability is significantly hindered by the hallucination problem. Existing hallucination detection approaches for D-LLMs mainly follow a training-based paradigm, relying on data-driven training to optimize the detector. Such reliance not only limits their generalizability across domains models but also incurs additional training cost and deployment overhead. To address these limitations, we propose TRE, a training-free hallucination detection metric for D-LLMs. TRE is a parameter-free and single-run metric that estimates hallucination risk directly from the entropy signals of a single generation, without requiring any detector training or repeated sampling. TRE extracts entropy signals within the D-LLM decoding process along both the spatial and temporal dimensions. From a token-level spatial perspective, we focus on revealing tokens as the most informative carriers of uncertainty, capturing where uncertainty is actively committed. From a diffusion step-level temporal perspective, we empirically identify the dominance of late-step entropy and hence aggregate these signals with a simple linear weighting scheme to obtain TRE. Extensive experiments on multiple D-LLMs and QA datasets demonstrate that TRE achieves competitive performance, while enjoying strong generalizability, efficiency, and robustness.

Figures

Figures reproduced from arXiv: 2607.22661 by Pengcheng Weng, Yanyu Qian, Yixin Liu, Yue Tan.

Figure 1
Figure 1. Figure 1: Overview of (a) the generation process of D-LLMs and the two key components of [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Evidence informativeness comparison on LLaDA-8B-Instruct with 64 diffusion steps. H¯ c := 1 |S| X t∈S 1 |T c t | X i∈T c t Ht(i), (2) where S denotes the set of late diffusion steps, T c t denotes the token set of category c at step t, and Ht(i) denotes the entropy of token i at diffusion step t, with category c ∈ {u, g, r} and u, g, and r corresponding to unrevealed, revealing, and revealed tokens, respec… view at source ↗
Figure 3
Figure 3. Figure 3: Revealing-token entropy trajectories across different D-LLMs, datasets, and diffusion steps. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Analogy of CDSs and D-LLMs. While empirical results show that TRE is effective, it is natural to ask whether there exists an intuitive perspective to interpret its underlying mechanism. Motivated by this, we draw an analogy between the revealing process of D-LLMs and that of a constrained dynamical system (CDS), and then use this perspective to interpret the design of TRE. In statistical physics, CDSs (e.g… view at source ↗
Figure 5
Figure 5. Figure 5: Sensitivity w.r.t. (a) Diffusion Steps, (b) Generation Length, and (c) Remasking Strategies. [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Time comparison. Efficiency Analysis. To assess the runtime efficiency of TRE, we compare the inference time on 100 QA samples from TriviaQA dataset using LLaDA. As depicted in [PITH_FULL_IMAGE:figures/full_fig_p009_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Case study of step-wise reveal. Sensitivity Analysis. We further study the sensi￾tivity of TRE to generation time steps, generation lengths, and remasking strategies, as shown in [PITH_FULL_IMAGE:figures/full_fig_p009_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Evidence informativeness comparison on LLaDA with 128 diffusion steps. We report [PITH_FULL_IMAGE:figures/full_fig_p014_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Additional revealing-token entropy trajectories across model families, datasets, and [PITH_FULL_IMAGE:figures/full_fig_p015_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Additional hallucinated case study examples. Hallucinated answer tokens are often newly [PITH_FULL_IMAGE:figures/full_fig_p024_10.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

60 extracted references · 8 linked inside Pith

  1. [1]

    Step- unrolled denoising autoencoders for text generation

    Nikolay Savinov, Junyoung Chung, Mikolaj Binkowski, Erich Elsen, and Aaron van den Oord. Step- unrolled denoising autoencoders for text generation. InInternational Conference on Learning Representa- tions, 2022

    2022

  2. [2]

    Diffusion-lm improves controllable text generation.Advances in neural information processing systems, 35:4328–4343, 2022

    Xiang Li, John Thickstun, Ishaan Gulrajani, Percy S Liang, and Tatsunori B Hashimoto. Diffusion-lm improves controllable text generation.Advances in neural information processing systems, 35:4328–4343, 2022

    2022

  3. [3]

    Discrete diffusion modeling by estimating the ratios of the data distribution

    Aaron Lou, Chenlin Meng, and Stefano Ermon. Discrete diffusion modeling by estimating the ratios of the data distribution. InInternational Conference on Machine Learning, 2024

    2024

  4. [4]

    Accelerating diffusion llms via adaptive parallel decoding.arXiv preprint arXiv:2506.00413, 2025

    Daniel Israel, Guy Van den Broeck, and Aditya Grover. Accelerating diffusion llms via adaptive parallel decoding.arXiv preprint arXiv:2506.00413, 2025

    arXiv 2025

  5. [5]

    Block diffusion: Interpolating between autoregressive and diffusion language models

    Marianne Arriola, Aaron Gokaslan, Justin T Chiu, Zhihan Yang, Zhixuan Qi, Jiaqi Han, Subham Sekhar Sahoo, and V olodymyr Kuleshov. Block diffusion: Interpolating between autoregressive and diffusion language models. InInternational Conference on Learning Representations, 2025

    2025

  6. [6]

    Survey of hallucination in natural language generation.ACM computing surveys, 55(12):1–38, 2023

    Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. Survey of hallucination in natural language generation.ACM computing surveys, 55(12):1–38, 2023

    2023

  7. [7]

    A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions.ACM Transactions on Information Systems, 43(2):1– 55, 2025

    Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, et al. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions.ACM Transactions on Information Systems, 43(2):1– 55, 2025

    2025

  8. [8]

    A survey of generalization of graph anomaly detection: From transfer learning to foundation models

    Junjun Pan, Yu Zheng, Yue Tan, and Yixin Liu. A survey of generalization of graph anomaly detection: From transfer learning to foundation models. InThe 16th IEEE International Conference on Knowledge Graphs, 2025

    2025

  9. [9]

    Shifting attention to relevance: Towards the predictive uncertainty quantification of free-form large language models

    Jinhao Duan, Hao Cheng, Shiqi Wang, Alex Zavalny, Chenan Wang, Renjing Xu, Bhavya Kailkhura, and Kaidi Xu. Shifting attention to relevance: Towards the predictive uncertainty quantification of free-form large language models. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5050–5063, 2024

    2024

  10. [10]

    Dynhd: Hallucination detection for diffusion large language models via denoising dynamics deviation learning.arXiv preprint arXiv:2603.16459, 2026

    Yanyu Qian, Yue Tan, Yixin Liu, Wang Yu, and Shirui Pan. Dynhd: Hallucination detection for diffusion large language models via denoising dynamics deviation learning.arXiv preprint arXiv:2603.16459, 2026

    arXiv 2026

  11. [11]

    Tracedet: Hallucination detection from the decoding trace of diffusion large language models

    Shenxu Chang, Junchi Yu, Weixing Wang, Yongqiang Chen, Jialin Yu, Philip Torr, and Jindong Gu. Tracedet: Hallucination detection from the decoding trace of diffusion large language models. InInterna- tional Conference on Learning Representations, 2026

    2026

  12. [12]

    Tdgnet: Hallucination detection in diffusion language models via temporal dynamic graphs.arXiv preprint arXiv:2602.08048, 2026

    Arshia Hemmat, Philip Torr, Yongqiang Chen, and Junchi Yu. Tdgnet: Hallucination detection in diffusion language models via temporal dynamic graphs.arXiv preprint arXiv:2602.08048, 2026

    arXiv 2026

  13. [13]

    Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models

    Potsawee Manakul, Adian Liusie, and Mark Gales. Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models. InProceedings of the 2023 conference on empirical methods in natural language processing, pages 9004–9017, 2023

    2023

  14. [14]

    Detecting hallucinations in large language models using semantic entropy.Nature, 630(8017):625–630, 2024

    Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and Yarin Gal. Detecting hallucinations in large language models using semantic entropy.Nature, 630(8017):625–630, 2024

    2024

  15. [15]

    Simple and effective masked diffusion language models

    Subham S Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin T Chiu, Alexander Rush, and V olodymyr Kuleshov. Simple and effective masked diffusion language models. Advances in Neural Information Processing Systems, 37:130136–130184, 2024. 10

    2024

  16. [16]

    Large language diffusion models.arXiv preprint arXiv:2502.09992, 2025

    Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. Large language diffusion models.arXiv preprint arXiv:2502.09992, 2025

    Pith/arXiv arXiv 2025

  17. [17]

    Siren’s song in the ai ocean: A survey on hallucination in large language models.Computational Linguistics, 51(4):1373–1418, 2025

    Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, et al. Siren’s song in the ai ocean: A survey on hallucination in large language models.Computational Linguistics, 51(4):1373–1418, 2025

    2025

  18. [18]

    Enhancing uncertainty-based hallucination detection with stronger focus

    Tianhang Zhang, Lin Qiu, Qipeng Guo, Cheng Deng, Yue Zhang, Zheng Zhang, Chenghu Zhou, Xinbing Wang, and Luoyi Fu. Enhancing uncertainty-based hallucination detection with stronger focus. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 915–932, 2023

    2023

  19. [19]

    Uncertainty- aware graph neural networks: A multihop evidence fusion approach.IEEE Transactions on Neural Networks and Learning Systems, 2025

    Qingfeng Chen, Shiyuan Li, Yixin Liu, Shirui Pan, Geoffrey I Webb, and Shichao Zhang. Uncertainty- aware graph neural networks: A multihop evidence fusion approach.IEEE Transactions on Neural Networks and Learning Systems, 2025

    2025

  20. [20]

    Language models (mostly) know what they know.arXiv preprint arXiv:2207.05221, 2022

    Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, et al. Language models (mostly) know what they know.arXiv preprint arXiv:2207.05221, 2022

    Pith/arXiv arXiv 2022

  21. [21]

    Semantic uncertainty: Linguistic invariances for uncer- tainty estimation in natural language generation

    Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. Semantic uncertainty: Linguistic invariances for uncer- tainty estimation in natural language generation. InInternational Conference on Learning Representations, 2023

    2023

  22. [22]

    Generating with confidence: Uncertainty quantification for black-box large language models.Transactions on Machine Learning Research, 2024

    Zhen Lin, Shubhendu Trivedi, and Jimeng Sun. Generating with confidence: Uncertainty quantification for black-box large language models.Transactions on Machine Learning Research, 2024

    2024

  23. [23]

    Inside: Llms’ internal states retain the power of hallucination detection

    Chao Chen, Kai Liu, Ze Chen, Yi Gu, Yue Wu, Mingyuan Tao, Zhihang Fu, and Jieping Ye. Inside: Llms’ internal states retain the power of hallucination detection. InInternational Conference on Learning Representations, 2024

    2024

  24. [24]

    Semantic entropy probes: Robust and cheap hallucination detection in llms.arXiv preprint arXiv:2406.15927, 2024

    Jannik Kossen, Jiatong Han, Muhammed Razzak, Lisa Schut, Shreshth Malik, and Yarin Gal. Semantic entropy probes: Robust and cheap hallucination detection in llms.arXiv preprint arXiv:2406.15927, 2024

    Pith/arXiv arXiv 2024

  25. [25]

    Patchfu- sionmlp: A scalable multi-resolution mlp framework for time series prediction.Pattern Recognition, page 113263, 2026

    Xinyu Bi, Chen Chen, Cheng Chen, Xiaoyi Lv, Junyu Zhu, Hongbing Ma, and Enguang Zuo. Patchfu- sionmlp: A scalable multi-resolution mlp framework for time series prediction.Pattern Recognition, page 113263, 2026

    2026

  26. [26]

    The internal state of an llm knows when it’s lying

    Amos Azaria and Tom Mitchell. The internal state of an llm knows when it’s lying. InFindings of the Association for Computational Linguistics: EMNLP 2023, pages 967–976, 2023

    2023

  27. [27]

    Truthx: Alleviating hallucinations by editing large language models in truthful space

    Shaolei Zhang, Tian Yu, and Yang Feng. Truthx: Alleviating hallucinations by editing large language models in truthful space. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8908–8949, 2024

    2024

  28. [28]

    Llm-check: Investigating detection of hallucinations in large language models.Advances in Neural Information Processing Systems, 37:34188–34216, 2024

    Gaurang Sriramanan, Siddhant Bharti, Vinu Sankar Sadasivan, Shoumik Saha, Priyatham Kattakinda, and Soheil Feizi. Llm-check: Investigating detection of hallucinations in large language models.Advances in Neural Information Processing Systems, 37:34188–34216, 2024

    2024

  29. [29]

    Prompt-guided internal states for hallucination detection of large language models

    Fujie Zhang, Peiqi Yu, Biao Yi, Baolei Zhang, Tong Li, and Zheli Liu. Prompt-guided internal states for hallucination detection of large language models. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 21806–21818, 2025

    2025

  30. [30]

    Discovering latent knowledge in language models without supervision

    Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt. Discovering latent knowledge in language models without supervision. InInternational Conference on Learning Representations, 2023

    2023

  31. [31]

    Representation engineering: A top-down approach to ai transparency.arXiv preprint arXiv:2310.01405, 2023

    Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, et al. Representation engineering: A top-down approach to ai transparency.arXiv preprint arXiv:2310.01405, 2023

    Pith/arXiv arXiv 2023

  32. [32]

    Halueval: A large-scale hallucination evaluation benchmark for large language models

    Junyi Li, Xiaoxue Cheng, Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. Halueval: A large-scale hallucination evaluation benchmark for large language models. InProceedings of the 2023 conference on empirical methods in natural language processing, pages 6449–6464, 2023

    2023

  33. [33]

    Factscore: Fine-grained atomic evaluation of factual precision in long form text generation

    Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. Factscore: Fine-grained atomic evaluation of factual precision in long form text generation. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12076–12100, 2023. 11

    2023

  34. [34]

    Faithdial: A faithful benchmark for information-seeking dialogue.Transactions of the Association for Computational Linguistics, 10:1473–1490, 2022

    Nouha Dziri, Ehsan Kamalloo, Sivan Milton, Osmar R Za¨ıane, Mo Yu, Edoardo M Ponti, and Siva Reddy. Faithdial: A faithful benchmark for information-seeking dialogue.Transactions of the Association for Computational Linguistics, 10:1473–1490, 2022

    2022

  35. [35]

    Lost in diffusion: Uncovering hallucination patterns and failure modes in diffusion large language models

    Zhengnan Guo and Fei Tan. Lost in diffusion: Uncovering hallucination patterns and failure modes in diffusion large language models. InFindings of the Association for Computational Linguistics: ACL 2026, 2026

    2026

  36. [36]

    Investigating selective prediction approaches across several tasks in iid, ood, and adversarial settings

    Neeraj Varshney, Swaroop Mishra, and Chitta Baral. Investigating selective prediction approaches across several tasks in iid, ood, and adversarial settings. InFindings of the association for computational linguistics: Acl 2022, pages 1995–2002, 2022

    2022

  37. [37]

    Structured denoising diffusion models in discrete state-spaces.Advances in neural information processing systems, 34:17981–17993, 2021

    Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne Van Den Berg. Structured denoising diffusion models in discrete state-spaces.Advances in neural information processing systems, 34:17981–17993, 2021

    2021

  38. [38]

    Argmax flows and multinomial diffusion: Learning categorical distributions.Advances in neural information processing systems, 34:12454–12465, 2021

    Emiel Hoogeboom, Didrik Nielsen, Priyank Jaini, Patrick Forr ´e, and Max Welling. Argmax flows and multinomial diffusion: Learning categorical distributions.Advances in neural information processing systems, 34:12454–12465, 2021

    2021

  39. [39]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020

    1901

  40. [40]

    Language models are unsupervised multitask learners.OpenAI blog, 1(8):9, 2019

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners.OpenAI blog, 1(8):9, 2019

    2019

  41. [41]

    Denoising diffusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    2020

  42. [42]

    Diffusion models beat gans on image synthesis.Advances in neural information processing systems, 34:8780–8794, 2021

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis.Advances in neural information processing systems, 34:8780–8794, 2021

    2021

  43. [43]

    Understanding diffusion models: A unified perspective.arXiv preprint arXiv:2208.11970, 2022

    Calvin Luo. Understanding diffusion models: A unified perspective.arXiv preprint arXiv:2208.11970, 2022

    Pith/arXiv arXiv 2022

  44. [44]

    Dream 7b: Diffusion large language models.arXiv preprint arXiv:2508.15487, 2025

    Jiacheng Ye, Zhihui Xie, Lin Zheng, Jiahui Gao, Zirui Wu, Xin Jiang, Zhenguo Li, and Lingpeng Kong. Dream 7b: Diffusion large language models.arXiv preprint arXiv:2508.15487, 2025

    Pith/arXiv arXiv 2025

  45. [45]

    Hotpotqa: A dataset for diverse, explainable multi-hop question answering

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D Manning. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. InProceedings of the 2018 conference on empirical methods in natural language processing, pages 2369–2380, 2018

    2018

  46. [46]

    Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension

    Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. InProceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1601–1611, 2017

    2017

  47. [47]

    On faithfulness and factuality in abstractive summarization

    Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. On faithfulness and factuality in abstractive summarization. InProceedings of the 58th annual meeting of the association for computational linguistics, pages 1906–1919, 2020

    1906

  48. [48]

    Prentice hall Upper Saddle River, NJ, 2002

    Hassan K Khalil and Jessy W Grizzle.Nonlinear systems, volume 3. Prentice hall Upper Saddle River, NJ, 2002

    2002

  49. [49]

    Optimization and stabilization of trajectories for constrained dynamical systems

    Michael Posa, Scott Kuindersma, and Russ Tedrake. Optimization and stabilization of trajectories for constrained dynamical systems. In2016 IEEE International Conference on Robotics and Automation (ICRA), pages 1366–1373. IEEE, 2016

    2016

  50. [50]

    Spin glass theory and beyond, 1988

    Marc M´ezard, Giorgio Parisi, Miguel Angel Virasoro, and David J Thouless. Spin glass theory and beyond, 1988

    1988

  51. [51]

    Spin glasses: Experimental facts, theoretical concepts, and open questions

    Kurt Binder and A Peter Young. Spin glasses: Experimental facts, theoretical concepts, and open questions. Reviews of Modern physics, 58(4):801, 1986

    1986

  52. [52]

    Steer llm latents for hallucination detection

    Seongheon Park, Xuefeng Du, Min-Hsuan Yeh, Haobo Wang, and Yixuan Li. Steer llm latents for hallucination detection. InInternational Conference on Machine Learning, 2025. 12

    2025

  53. [53]

    Out-of-distribution detection and selective generation for conditional language models

    Jie Ren, Jiaming Luo, Yao Zhao, Kundan Krishna, Mohammad Saleh, Balaji Lakshminarayanan, and Peter J Liu. Out-of-distribution detection and selective generation for conditional language models. In International Conference on Learning Representations, 2023

    2023

  54. [54]

    Uncertainty estimation in autoregressive structured prediction.arXiv preprint arXiv:2002.07650, 2020

    Andrey Malinin and Mark Gales. Uncertainty estimation in autoregressive structured prediction.arXiv preprint arXiv:2002.07650, 2020

    Pith/arXiv arXiv 2002

  55. [55]

    Commonsenseqa: A question answering challenge targeting commonsense knowledge

    Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. Commonsenseqa: A question answering challenge targeting commonsense knowledge. InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4149–4158, 2019

    2019

  56. [56]

    Rethinking unsupervised time series anomaly detection: Dynamic attention based on route inverse-masking.Applied Soft Computing, page 113971, 2025

    Enguang Zuo, Jie Zhong, Chen Chen, Cheng Chen, Kurban Ubul, and Xiaoyi Lv. Rethinking unsupervised time series anomaly detection: Dynamic attention based on route inverse-masking.Applied Soft Computing, page 113971, 2025

    2025

  57. [57]

    Correcting false alarms from unseen: Adapting graph anomaly detectors at test time

    Junjun Pan, Yixin Liu, Chuan Zhou, Fei Xiong, Alan Wee-Chung Liew, and Shirui Pan. Correcting false alarms from unseen: Adapting graph anomaly detectors at test time. InProceedings of the AAAI Conference on Artificial Intelligence, 2026

    2026

  58. [58]

    Influence-oriented personalized federated learning.arXiv preprint arXiv:2410.03315, 2024

    Yue Tan, Guodong Long, Jing Jiang, and Chengqi Zhang. Influence-oriented personalized federated learning.arXiv preprint arXiv:2410.03315, 2024

    Pith/arXiv arXiv 2024

  59. [59]

    Treexformer: Extracting tabular feature-context information using tree-structured semantics.Information Processing & Management, 62(6):104291, 2025

    Yinhong Li, Hanwen Qu, Chen Chen, Xiaoyi Lv, Enguang Zuo, Kui Wang, and Xulun Cai. Treexformer: Extracting tabular feature-context information using tree-structured semantics.Information Processing & Management, 62(6):104291, 2025

    2025

  60. [60]

    sail.” instead of “Merchant mariner

    Rui Miao, Yixin Liu, Yili Wang, Xu Shen, Yue Tan, Yiwei Dai, Shirui Pan, and Xin Wang. Blindguard: Safeguarding llm-based multi-agent systems under unknown attacks. InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics, 2026. 13 A Details of Motivating Experiments A.1 Evidence Construction Experiments This appendix prov...

    2026