Delta -- Contrastive Decoding Mitigates Text Hallucinations in Large Language Models

Cheng Peng Huang; Hao-Yuan Chen

REVIEW 5 major objections 5 minor 1 cited by

Delta -- Contrastive Decoding Mitigates Text Hallucinations in Large Language Models

T0 review · 5 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper argues that hallucinated tokens can be removed by contrasting the model's logits on an intact prompt with its logits on a randomly masked version, reporting exact-match gains of 3–7 percentage points on four context-rich QA…

desk verdict Delta is a reasonable inference-only contrastive decoding variant, but the paper skips the closest baseline and reports single-run numbers, so the gains are plausible, not proven. read the letter →

arxiv 2502.05825 v1 pith:BKMCEB5N submitted 2025-02-09 cs.CL cs.AI

Cheng Peng Huang , Hao-Yuan Chen This is my paper · ORCID

classification cs.CLcs.AI

keywords hallucinationmitigationcontrastivedecodingrandommaskinginference-timequestionansweringlargelanguagemodelsadaptiveplausibilityconstraintsSQuAD

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

$\Delta$ is an inference-time decoding rule that tries to suppress hallucinated tokens without retraining: at each step it masks a random fraction of the prompt, computes the model's logits on both original and masked inputs, and samples from a distribution that amplifies the original logits and subtracts the masked ones. The paper's hypothesis is that a model given a corrupted prompt falls back on generic statistical associations, so the masked-input logits approximate the hallucination prior, and removing them leaves tokens that are actually supported by the context. On context-rich question answering, the paper reports exact-match gains of roughly 3 percentage points on SQuAD v1.1, 6 on SQuAD v2, 7 on TriviaQA, and 2 on Natural Questions under sampling, with an even larger jump on SQuAD v2's no-answer exact match. The paper also reports that $\Delta$ does not help on context-free benchmarks such as CommonsenseQA and MMLU, which it presents as a limitation tied to the masking mechanism.

What carries the argument

The central object is the contrastive decoding distribution in Eq. 5, which combines the model's original logits with logits from a masked version of the same prompt: $P_{\mathrm{delta}}(y_t \mid z) = \mathrm{softmax}[(1+\alpha)\mathrm{logit}_\theta(y \mid z) - \alpha\,\mathrm{logit}_\theta(y \mid \mathrm{mask}(z))]$. The masking procedure replaces $m = \lfloor r_{\mathrm{mask}} \cdot n \rfloor$ randomly chosen prompt tokens with a single MASK token, and the adaptive plausibility constraint restricts sampling to tokens that are at least a $\beta$ fraction as probable as the top token under the original model. This machinery turns the paper's 'moldy banana' intuition—masking removes a fact like 'moldy,' so the model reverts to the prior 'yellow'—into a usable per-token penalty applied at every decoding step.

What would settle it

Measure whether $\Delta$'s per-token penalization aligns with baseline error rates: if tokens most strongly suppressed by subtracting masked logits are not the tokens the model gets wrong without $\Delta$, then the masked logits are not functioning as a hallucination prior and the reported gains come from some other re-ranking effect.

Watch

Extended reading notes

Core claim

On its own terms, $\Delta$ claims that hallucinated content can be identified by the gap between a model's predictions on an intact input and on a randomly masked version of that input. The decoding rule is $y_t \sim \mathrm{softmax}[(1+\alpha)\mathrm{logit}_\theta(y \mid z) - \alpha\,\mathrm{logit}_\theta(y \mid \mathrm{mask}(z))]$ restricted to tokens whose unmasked probability clears an adaptive threshold, with fixed settings $r_{\mathrm{mask}}=0.7$, $\alpha=0.3$, and $\beta=0.1$ in the experiments. This rule is intended to boost context-dependent tokens while penalizing tokens the model would have produced from its priors alone. The paper reports that the rule improves exact match on SQuAD v1.1, SQuAD v2, TriviaQA, and Natural Questions under sampling, and most strongly improves the ability to answer 'no answer' on SQuAD v2, while leaving context-free benchmarks essentially unchanged.

Load-bearing premise

The method assumes that after part of the prompt is randomly replaced by a MASK token, the model's token probabilities become a trustworthy estimate of what it would hallucinate; if masking suppresses correct answers too, subtracting those logits can re-rank rather than remove hallucinations.

Editorial extensions

If this is right

If Delta works as claimed, any instruction-tuned LLM can get a hallucination reduction on context-rich tasks by adding one extra masked forward pass per decoding step, with no fine-tuning or extra data.
The largest reported effect, more than ten points on SQuAD v2 no-answer exact match, implies that the worst hallucinations in reading comprehension come from cases where the context simply does not contain an answer, and Delta's contrast most helps there.
Because TriviaQA and Natural Questions gains appear mainly with sampling, Delta should be used with sampling decoding when the task involves long, noisy context.
Context-free QA is a known boundary: on CommonsenseQA and MMLU, Delta gives small accuracy declines, so its benefit is tied to external context.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper, the per-token gap between masked and unmasked logits could be used as a hallucination score during answer verification, with strongly penalized tokens flagged as fabrication candidates.
The paper fixes the mask ratio at 0.7 and uses the end-of-sequence token as the MASK; choosing masks that avoid question tokens or target content words could make the contrast sharper, since random masks can hit the very tokens that carry the answer.
A testable corollary is that Delta should help most when an answer is derivable from a few salient context tokens and least when the answer depends on global reasoning; TriviaQA's lack of gain without sampling is consistent with this, though the paper does not study mask locations.
If the masked-input distribution is truly a hallucination prior, then Delta's benefit should correlate with baseline error rates, which would support using Delta as both a decoder and a diagnostic.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: Delta applies the visual-contrastive-decoding trick to text by randomly masking input tokens, then subtracting the masked-prompt logits from the original logits during generation. That is a legitimate idea and a natural extension of CAD, but the paper never compares against CAD, reports no variance for a stochastic masking procedure, and its headline SQuAD v2 gain is mostly abstention on no-answer questions. The core idea is worth taking seriously, but the evidence as written is conditional.

What is genuinely new: the specific combination of random token masking plus contrastive decoding for text QA is not in prior work. The paper also does something right by testing on context-rich and context-free datasets, and the authors are honest that Delta hurts or does nothing on CommonsenseQA and MMLU. The ablation covering masking ratios and alpha values, with all settings above baseline on SQuAD v1.1, is a useful robustness check.

The soft spots are real but not fatal. First, Eq. 3 is structurally the same logit contrast as CAD with mask(z) in place of the no-context prompt, so without a CAD baseline you cannot tell whether Delta adds anything beyond existing context-aware decoding. Second, the paper reports single-run EM/F1 numbers despite sampling masks stochastically at rmask=0.7; a couple of random seeds could shift the numbers by more than the reported gains. Third, the SQuAD v2 improvement is concentrated in NoAns EM (up 14.53 w/o sampling) while HasAns EM drops 1.60, which means the method is largely shifting the model toward abstention rather than demonstrably dehallucinating token-level content. Fourth, the central premise—that masked prompt logits are a faithful hallucination prior—is asserted via the moldy banana example but never measured. That is a real gap, because if masked logits are dominated by generic language priors, the method could be suppressing correct answers rather than selectively removing hallucinated ones.

None of this kills the paper. A competent referee could ask for a CAD baseline, error bars, and a token-level diagnostic, and the authors could plausibly deliver them. The paper is short and readable; the idea is simple and easy to reproduce; and the direction of effect is consistent with the stated mechanism. I would send it to peer review, but I would not treat the current numbers as strong evidence.

For you: this is a reasonable reading-group piece for people interested in inference-time hallucination mitigation, and worth citing once the authors add the missing baseline. I would advise the editor to invite a revision rather than reject.

Referee Report

5 major / 5 minor

Summary. The paper proposes Delta, an inference-time contrastive decoding method that randomly masks a portion of the input prompt and subtracts the masked-prompt logits from the original logits to suppress hallucinated tokens. The method is defined in Eq. 5, with an adaptive plausibility constraint. The authors evaluate Delta on Llama-3.1-8B-Instruct (4-bit) across SQuAD v1.1/v2, TriviaQA, Natural Questions, CommonsenseQA, and MMLU, reporting exact-match gains of roughly 3 points on SQuAD v1.1, 6 points on SQuAD v2, 7 points on TriviaQA, and 2 points on Natural Questions under sampling, with a particularly large gain on SQuAD v2 no-answer exact match. The paper also includes an ablation over the masking ratio and alpha on SQuAD v1.1.

Significance. If the central claim holds, Delta would be a simple, training-free, inference-only method for reducing hallucinations in context-rich QA, with potentially broad applicability because it does not require external models or fine-tuning. The paper is honest about the method's limitation on context-free benchmarks, and the ablation shows robustness to hyperparameters on one dataset. However, the core premise—that masked-prompt logits are a faithful proxy for the model's hallucination prior—is asserted rather than demonstrated, and the headline improvements on SQuAD v2 are driven heavily by increased abstention on unanswerable questions. The experiments appear to be single-run without reported variance, which is a serious concern given the stochastic masking procedure. The contribution is potentially useful but requires substantially stronger evidence to be established.

major comments (5)

[§3.2–§3.4, Eq. (3)] The central claim that subtracting logit_theta(y | mask(z)) removes hallucinated content relies on the premise that mask(z) produces a distribution in which hallucinated tokens are specifically elevated. This premise is illustrated with the moldy banana example in §3.2 but is never directly measured. Without a token-level diagnostic showing that masked logits are elevated on hallucinated answers and not on correct context-dependent answers, Eq. (3) remains an unvalidated heuristic; the observed gains could also come from generic suppression of answer tokens or from the MASK/eos token dominating the subtracted distribution. Please provide such a diagnostic, or a controlled experiment varying the masking distribution and showing a monotonic relationship between masking-induced logit increases and actual hallucination rates.
[Table 1, §5.1] On SQuAD v2 without sampling, the no-answer exact match improves by +14.53 points (23.63 to 38.17) while the has-answer exact match drops by 1.60 points (59.08 to 57.47). This pattern suggests that a large part of Delta's improvement is an increased tendency to abstain on unanswerable questions, rather than a token-level dehallucination that preserves correct answer extraction. The paper does not report a combined SQuAD v2 metric such as EM averaged over both answerable and unanswerable examples, nor does it analyze the HasAns/NoAns trade-off. Please report these joint metrics with error bars and discuss the mechanism explicitly.
[§4.2, §5 (Tables 1–2)] All experiments appear to be run once with a single random masking seed (or no seed control). Because the masking indices are randomly drawn (Eq. 2 and §3.3), the reported differences of 1–3 percentage points on several benchmarks may be within the noise of the stochastic procedure. The paper should report means and standard deviations over multiple masking seeds, or at a minimum state that a fixed seed was used and justify that choice.
[§3.3, §4.2] With rmask = 0.7, the masking is applied uniformly to the whole prompt, including the question tokens, so mask(z) is often a heavily corrupted input rather than a mildly fuzzified context. This makes it unclear whether the subtracted logits encode a 'hallucination prior' or simply a distribution conditioned on a partially unreadable prompt. The method should be tested with masking applied only to the context passage (leaving the question intact), and the effect of mask placement on the answer distribution should be analyzed. This directly bears on the validity of Eq. (3).
[§2, §4.2, Table 1] The related work cites context-aware decoding (CAD; Shi et al., 2024), which is conceptually the closest baseline: CAD contrasts logits with and without context. Since Delta is essentially a context-fuzzing variant of the same contrastive idea, the paper should include CAD as a direct baseline under identical conditions. Without this comparison, it is unclear whether Delta offers any improvement over simply contrasting against the no-context or weakly-context distribution, which would be a much simpler method.

minor comments (5)

[§3.5, Eq. (4) and (5)] There is a notation inconsistency between the definition of V_head(x_{<t}) in Eq. (4) and its use in Eq. (5) as V_head(z_{<t}). The sentence in §3.6 saying 'If z does not belong to V_head(x_{<i})' mixes brackets and subscripts; please unify.
[§4.2] The sentence 'All experiments utilize the end-of-sequence (eos) token as the MASK token' is surprising, because the EOS token is a normal vocabulary token with its own learned semantics. Please clarify what this means operationally and whether the model was explicitly trained with such a masking token.
[§6] The ablation reports 'standard deviations of 0.66 for exact match and 0.21 for F1' over the heatmap cells. Since the heatmap varies hyperparameters and not random seeds, calling these standard deviations is misleading; they are ranges across settings, not estimates of run-to-run variability. Please rephrase.
[§2] There are grammar and citation errors in the sentence beginning 'In addition, the work context-aware decoding (CAD) has Shi et al. (2024) demonstrated'; the citation should be parenthetical and the verb should be finite.
[References] The reference to Wettig et al. is incomplete: the title is cut off as 'Should you mask 15' and the page numbers are missing. Several other references also lack full bibliographic details; please check the reference list for completeness.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Delta's headline claims are external benchmark measurements, and its borrowed masked-logit premise is an unverified assumption rather than a constructed prediction.

full rationale

The claimed derivation chain is Eq. 3, which defines Delta's decoding distribution by contrasting unmasked and randomly masked logits, and the reported gains in exact match/F1 on SQuAD v1.1/v2, TriviaQA, and Natural Questions. These gains are measured against an external baseline, not solved for or fitted from the equations. Hyperparameters (rmask=0.7, alpha=0.3, beta=0.1) are fixed in Section 4.2, and the Section 6 ablation on SQuAD v1.1 with sampling reports a grid (rmask 0.3/0.5/0.7, alpha 0.1-0.5) in which all configurations surpassed baseline, so the headline improvement is not a sensitivity-selected artifact in a way that would make the prediction forced. The central premise that masked prompts yield hallucination-prone logits is imported from Leng et al. (2024) and asserted rather than directly measured in this paper; that is an evidentiary weakness, but it is not circularity because the paper does not derive its benchmark numbers from that premise. Similarly, the sentence describing masked logits as 'hallucinated logits' and subtraction as yielding 'clean' logits is rhetorically self-definitional, but the paper's load-bearing claims are external EM/F1 measurements, so no prediction reduces to a definition by construction. There are no load-bearing self-citations or author-imported uniqueness theorems. The ablation dataset does overlap the SQuAD v1.1 sampling headline, which is a mild tuning concern, but the reported grid robustness prevents it from being a fitted-input-called-prediction circularity.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The method contributes no new entities, but rests on two unmeasured assumptions about how masking shifts token distributions, plus three manually set hyperparameters. The core score-shaping equation is borrowed from contrastive decoding literature.

free parameters (4)

rmask (masking ratio) = 0.7
Controls how many prompt tokens are replaced by MASK; fixed across all experiments, ablation on SQuAD v1.1 varies 0.3 to 0.7 (Sections 4.2, 6).
alpha (logit ratio) = 0.3
Scales the subtracted masked logits in Eq. 3; fixed across experiments, ablation varies 0.1 to 0.5 (Sections 4.2, 6).
beta (APC threshold) = 0.1
Threshold in Adaptive Plausibility Constraints borrowed from prior work; fixed without sensitivity analysis (Section 4.2).
mask token choice = EOS token
The end-of-sequence token is used as MASK for all experiments; no ablation on this choice (Section 4.2).

assumptions (3)

domain assumption Random masking of input text amplifies hallucination in LLM outputs
Core premise of the method, introduced in Section 3.2 with a single example; no direct measurement that masked versions hallucinate more.
domain assumption Subtracting masked logits from original logits yields 'clean' non-hallucinated logits
Eq. 3 states the linear contrast but does not derive why cancellation preserves true context-dependent tokens; this is the mechanism assumption.
domain assumption Adaptive Plausibility Constraints from Li et al. transfer unchanged to this setting
Eq. 4 is borrowed from contrastive decoding and applied with beta fixed at 0.1 without validation on these QA tasks.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Delta -- Contrastive Decoding Mitigates Text Hallucinations in Large Language Models." pith.science (2026). https://pith.science/paper/BKMCEB5N

@misc{pith2026250205825,
  author       = {Pith},
  title        = {Pith review of: Delta -- Contrastive Decoding Mitigates Text Hallucinations in Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BKMCEB5N}},
  note         = {Machine review of arXiv:2502.05825}
}

read the original abstract

Large language models (LLMs) demonstrate strong capabilities in natural language processing but remain prone to hallucinations, generating factually incorrect or fabricated content. This issue undermines their reliability, particularly in high-stakes domains such as healthcare and legal advisory. To address this challenge, we propose Delta, an inference-time method that reduces hallucinations without requiring model retraining or additional data. Delta works by randomly masking parts of the input prompt and contrasting the output distributions for the original and masked inputs, effectively suppressing hallucinations through inference-only computations. We evaluate Delta on context-rich question-answering benchmarks, achieving absolute improvements of approximately 3 and 6 percentage points on SQuAD v1.1 and v2, respectively, and 7 and 2 percentage points on TriviaQA and Natural Questions under-sampling decoding. Delta also improves the no-answer exact match score on SQuAD v2 by over ten percentage points, demonstrating its effectiveness in mitigating hallucinations arising from contextual ambiguity. These results highlight Delta as a computationally efficient and scalable approach for improving the reliability of LLMs in real-world applications.

Figures

Figures reproduced from arXiv: 2502.05825 by the authors.

**Figure 2.** Comparison of Exact Match and F1 Score Heatmaps for Delta Model on SQuAD v1.1 [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Reducing Object Hallucination in Large Audio-Language Models via Audio-Aware Decoding
eess.AS 2025-06 conditional novelty 4.0 of 10

Audio-Aware Decoding, a contrastive decoding method that uses silent audio as the no-context baseline, reduces object hallucination and improves accuracy across three large audio-language models.

Reference graph

Works this paper leans on

17 extracted references · 13 linked inside Pith · cited by 1 Pith paper

[5]

Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu

URL https://arxiv.org/abs/2404.01331. Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions,

arXiv
[6]

Ziwei Ji, Tiezheng Yu, Yan Xu, Nayeon Lee, Etsuko Ishii, and Pascale Fung

URL https: //arxiv.org/abs/2311.05232. Ziwei Ji, Tiezheng Yu, Yan Xu, Nayeon Lee, Etsuko Ishii, and Pascale Fung. Towards mitigating LLM hallucination via self reflection. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Computational Linguistics: EMNLP 2023 , pp. 1827–1843, Sin- gapore, December

arXiv 2023
[7]

doi: 10.18653/v1/2023

Association for Computational Linguistics. doi: 10.18653/v1/2023. findings-emnlp.123. URL https://aclanthology.org/2023.findings-emnlp

doi:10.18653/v1/2023 2023
[10]

Contrastive decoding: Open-ended text generation as optimization, 2023a

Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori Hashimoto, Luke Zettlemoyer, and Mike Lewis. Contrastive decoding: Open-ended text generation as optimization, 2023a. URL https://arxiv.org/abs/2210.15097. 9 Pre-print Zihao Li, Zhuoran Yang, and Mengdi Wang. Reinforcement learning with human feedback: Learning dynamic choices...

arXiv
[11]

URL https://arxiv.org/abs/2305.14552. OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Floren- cia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Moham- mad Bavarian, Jeff Belgum, Irwan Bello, Jake Berdine, Gab...

arXiv
[12]

Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L

URL https://arxiv.org/abs/2303.08774. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kel- ton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, 10 Pre-print and Ryan Lowe. Training langua...

arXiv
[15]

doi: 10.18653/v1/2024.naacl-short.69

Associ- ation for Computational Linguistics. doi: 10.18653/v1/2024.naacl-short.69. URL https: //aclanthology.org/2024.naacl-short.69. Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. CommonsenseQA: A ques- tion answering challenge targeting commonsense knowledge. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of...

doi:10.18653/v1/2024.naacl-short.69 2024
[16]

doi: 10.18653/v1/N19-1421

Association for Com- putational Linguistics. doi: 10.18653/v1/N19-1421. URL https://aclanthology.org/ N19-1421. Alexander Wettig, Tianyu Gao, Zexuan Zhong, and Danqi Chen. Should you mask 15URLhttps: //arxiv.org/abs/2202.08005. Ziwei Xu, Sanjay Jain, and Mohan Kankanhalli. Hallucination is inevitable: An innate limitation of large language models,

arXiv

Show all 17 references

[17]

URL https://arxiv.org/abs/2401.11817. 11

arXiv
[2016]

doi: 10.18653/v1/D16-1264

Association for Computational Linguistics. doi: 10.18653/v1/D16-1264. URL https://aclanthology.org/D16-1264. Weijia Shi, Xiaochuang Han, Mike Lewis, Yulia Tsvetkov, Luke Zettlemoyer, and Wen-tau Yih. Trusting your evidence: Hallucinate less with context-aware decoding. In Kevi...

doi
[2017]

URL https://arxiv.org/ abs/1705.03551. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. ...

arXiv
[2019]

URL https://aclanthology.org/Q19-1026

doi: 10.1162/tacl a 00276. URL https://aclanthology.org/Q19-1026. Sicong Leng, Hang Zhang, Guanzheng Chen, Xin Li, Shijian Lu, Chunyan Miao, and Lidong Bing. Mitigating object hallucinations in large vision-language models through visual contrastive de- coding. In Proceedings ...

doi
[2020]

Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James Glass, and Pengcheng He

URL https://arxiv.org/abs/2005.14165. Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James Glass, and Pengcheng He. Dola: Decoding by contrasting layers improves factuality in large language models,

2005 arXiv
[2021]

Musashi Hinck, Matthew L

URL https: //arxiv.org/abs/2009.03300. Musashi Hinck, Matthew L. Olson, David Cobbley, Shao-Yen Tseng, and Vasudev Lal. Llava- gemma: Accelerating multimodal foundation models with a compact language model,

2009 arXiv
[2022]

Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang

URL https://arxiv.org/abs/2203.02155. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. SQuAD: 100,000+ questions for machine comprehension of text. In Jian Su, Kevin Duh, and Xavier Carreras (eds.), Pro- ceedings of the 2016 Conference on Empirical Methods in...

2016 arXiv
[2023]

Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Ja- cob Steinhardt

URL https://arxiv.org/abs/ 2306.03078. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Ja- cob Steinhardt. Measuring massive multitask language understanding,

arXiv
[2024]

Tim Dettmers, Ruslan Svirschevski, Vage Egiazarian, Denis Kuznedelev, Elias Frantar, Saleh Ashk- boos, Alexander Borzunov, Torsten Hoefler, and Dan Alistarh

URL https://arxiv.org/abs/2309.03883. Tim Dettmers, Ruslan Svirschevski, Vage Egiazarian, Denis Kuznedelev, Elias Frantar, Saleh Ashk- boos, Alexander Borzunov, Torsten Hoefler, and Dan Alistarh. Spqr: A sparse-quantized repre- sentation for near-lossless llm weight compression,

arXiv

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.

[1] [5]

Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu

URL https://arxiv.org/abs/2404.01331. Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions,

arXiv

[2] [6]

Ziwei Ji, Tiezheng Yu, Yan Xu, Nayeon Lee, Etsuko Ishii, and Pascale Fung

URL https: //arxiv.org/abs/2311.05232. Ziwei Ji, Tiezheng Yu, Yan Xu, Nayeon Lee, Etsuko Ishii, and Pascale Fung. Towards mitigating LLM hallucination via self reflection. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Computational Linguistics: EMNLP 2023 , pp. 1827–1843, Sin- gapore, December

arXiv 2023

[3] [7]

doi: 10.18653/v1/2023

Association for Computational Linguistics. doi: 10.18653/v1/2023. findings-emnlp.123. URL https://aclanthology.org/2023.findings-emnlp

doi:10.18653/v1/2023 2023

[4] [10]

Contrastive decoding: Open-ended text generation as optimization, 2023a

Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori Hashimoto, Luke Zettlemoyer, and Mike Lewis. Contrastive decoding: Open-ended text generation as optimization, 2023a. URL https://arxiv.org/abs/2210.15097. 9 Pre-print Zihao Li, Zhuoran Yang, and Mengdi Wang. Reinforcement learning with human feedback: Learning dynamic choices...

arXiv

[5] [11]

URL https://arxiv.org/abs/2305.14552. OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Floren- cia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Moham- mad Bavarian, Jeff Belgum, Irwan Bello, Jake Berdine, Gab...

arXiv

[6] [12]

Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L

URL https://arxiv.org/abs/2303.08774. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kel- ton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, 10 Pre-print and Ryan Lowe. Training langua...

arXiv

[7] [15]

doi: 10.18653/v1/2024.naacl-short.69

Associ- ation for Computational Linguistics. doi: 10.18653/v1/2024.naacl-short.69. URL https: //aclanthology.org/2024.naacl-short.69. Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. CommonsenseQA: A ques- tion answering challenge targeting commonsense knowledge. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of...

doi:10.18653/v1/2024.naacl-short.69 2024

[8] [16]

doi: 10.18653/v1/N19-1421

Association for Com- putational Linguistics. doi: 10.18653/v1/N19-1421. URL https://aclanthology.org/ N19-1421. Alexander Wettig, Tianyu Gao, Zexuan Zhong, and Danqi Chen. Should you mask 15URLhttps: //arxiv.org/abs/2202.08005. Ziwei Xu, Sanjay Jain, and Mohan Kankanhalli. Hallucination is inevitable: An innate limitation of large language models,

arXiv