{"id":"6d20e8e2-5731-4ca4-806c-62854d1e132c","arxiv_id":"2505.08200","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Pre-trained transformer-based UQ heads using LLM attention maps and token probabilities achieve state-of-the-art claim-level hallucination detection in-domain, out-of-domain, and cross-lingually, with about 5% inference overhead.","lead":"This paper introduces small pre-trained modules, called UQ heads, that attach to large language models and flag hallucinated claims in their output. They report state-of-the-art claim-level hallucination detection across domains and languages, with only about 5% added compute, making the approach practical for real systems.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"GPT-4o is both the training and test label oracle; if its supportedness judgments are biased, reported PR-AUC measures agreement with GPT-4o, not true hallucination detection.","rationale":"The reader's weakest assumption correctly identifies that GPT-4o serves as both the training and evaluation label oracle, making the reported PR-AUC potentially a measure of agreement with GPT-4o's supportedness judgments rather than true hallucination detection. This is the most load-bearing concern because the paper's central claim is empirical state-of-the-art performance, and if the label protocol is systematically biased, the entire ranking of methods is called into question. The concern is strengthened by the paper's own limitation statements: Section 4 notes that 'annotation quality could be further improved by leveraging more powerful LLMs or employing an ensemble of models,' and the Ethical Considerations explicitly acknowledge potential bias from GPT-4o annotation. A second, related issue is that all headline comparisons are single-run PR-AUC values with no variance or confidence intervals; however, this is secondary to the label-bias problem, because even perfect variance reporting would not correct a biased oracle. The proposed concrete test—independent human or alternative-LLM annotation on a sample of claims—directly addresses the central assumption and would settle whether the SOTA claim generalizes to labels from a different source. Since the reader's verdict is already CONDITIONAL and this concern reinforces rather than changes the required conditions, I recommend no change to the verdict. The paper remains a competent and useful empirical study with released code and models, but the headline claim should be treated as conditional on the validity of the GPT-4o label protocol until independent-label evaluation is provided.","tokens_in":15425,"tokens_out":2708,"duration_ms":26512,"concrete_test":"Sample 200–300 claims from the existing test responses (or generate fresh responses from Mistral 7B and Gemma 2 9B) and have independent human annotators, or an ensemble of at least two strong LLMs different from GPT-4o, label each claim as supported/unsupported/unknown using identical claim extraction. Recompute PR-AUC for UHead and the strongest supervised baseline (Factoscope) against these independent labels. If UHead's margin over Factoscope shrinks by more than a few points, or if both methods drop sharply relative to the GPT-4o-labeled Table 1 scores, the SOTA claim is not robust to label-source bias. To isolate training contamination, additionally train a fresh UHead on GPT-4o labels and evaluate it on the human labels; a disproportionate PR-AUC drop would confirm that the head learned to mimic GPT-4o's biases.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that UHead achieves state-of-the-art claim-level hallucination detection (Tables 1 and 4). The most load-bearing assumption is that the supported/unsupported labels used for both training and evaluation (Section 4, Appendix B) are accurate ground truth. These labels are produced entirely by GPT-4o through claim extraction and two-stage chain-of-thought labeling. The authors themselves acknowledge in the Ethical Considerations that 'This may introduce cultural, linguistic, or other biases into the data.' If GPT-4o's judgments are systematically biased or noisy in a way that correlates with the attention and probability features UHead consumes, the reported PR-AUC scores quantify agreement with GPT-4o rather than veridical hallucination detection. The method could learn to predict GPT-4o's labeling heuristics rather than factual correctness. Because the same oracle labels both train and test, this concern is not refuted by the method's out-of-domain generalization; consistent transfer of a learned proxy for GPT-4o's bias is exactly what the OOD results would look like. This is compounded by the absence of variance or significance reporting: all headline numbers are single-run PR-AUC values, so even within the GPT-4o label regime the claimed margins over Factoscope (e.g., 5 pps in-domain, 1 pp on landmarks) could be within run-to-run noise.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces UHead, a supervised auxiliary module for claim-level hallucination detection in LLM outputs. The module consists of a small Transformer encoder that processes token-level features, namely raw attention weights to the preceding k tokens and log-probabilities of the top-m token predictions, all extracted from a frozen LLM; a learned claim-token embedding marks the span of the claim being scored. Training data are produced by prompting the target LLM (Mistral 7B Instruct v0.2 or Gemma 2 9B Instruct) with biography questions, extracting atomic claims with GPT-4o, and labeling each claim as supported, unsupported, or unknown via a two-stage chain-of-thought prompt. Experiments compare UHead with unsupervised and supervised baselines on eight English domains (one in-domain, seven out-of-domain) and on Russian, Chinese, and German biography prompts, reporting PR-AUC as the main metric. The paper also reports about 5% computational overhead and a 40 MB memory footprint, and it releases code and pre-trained heads for several LLM families.","tokens_in":15695,"tokens_out":4540,"duration_ms":44751,"significance":"The empirical design is systematic in several respects: supervised baselines are retrained and tuned on the same biography data, hyperparameters are selected on a validation set, test domains and languages are disjoint from training, and the comparison includes both unsupervised scores and three supervised methods. The released code and pre-trained heads are a concrete practical contribution, and the analysis of which attention layers and token windows carry hallucination-related signal is a useful starting point for future work. If the results survive an independent label oracle, UHead would be a strong and lightweight off-the-shelf hallucination detector. The main risk is that GPT-4o serves as both the training and the test label oracle, so the absolute PR-AUC numbers may quantify agreement with GPT-4o's supportedness judgments rather than veridical hallucination detection; in addition, the absence of variance or significance reporting leaves the size of the reported gains uncertain. These concerns are central to the paper's claims and are addressable within the manuscript's scope.","major_comments":[{"comment":"Because GPT-4o produces both the training and the test labels (Section 4, Table 6), the reported PR-AUC values in Tables 1, 2, and 4 measure agreement with GPT-4o's supportedness judgments rather than veridical hallucination detection. The paper's own Ethical Considerations state that this 'may introduce cultural, linguistic, or other biases into the data,' yet no independent validation of the label oracle is provided. I request a human-annotated sample (or an alternative oracle) for at least the out-of-domain and cross-lingual test sets, together with agreement scores, to establish that UHead detects actual hallucinations and not GPT-4o's labeling heuristics.","section":"Section 4 and Appendix B"},{"comment":"The headline tables report single-run PR-AUC values with no variance, confidence intervals, or significance tests. Some comparative margins are small—for example, UHead trails Factoscope by 1 pp on landmarks in Table 1, and the differences in Table 2 are at most 0.02 PR-AUC—so the claimed state-of-the-art performance may be within run-to-run noise. Please report results over multiple seeds, or at least bootstrap confidence intervals, for the key comparisons that support the main claims.","section":"Tables 1, 2, and 4"},{"comment":"The cross-lingual generalization claim is especially sensitive to the label-oracle issue: GPT-4o's labeling behavior may differ across Russian, Chinese, and German, so the observed 'generalization' could reflect transfer of a proxy for GPT-4o's cross-lingual biases rather than transfer of a general hallucination-detection capability. I request per-language human validation or a comparison with an independent oracle on a sample of each language before the cross-lingual result can be interpreted as evidence for the stated claim.","section":"Table 4"}],"minor_comments":[{"comment":"Equation (1) appears to contain a typo: the concatenation lists h1(t) twice instead of showing a sequence of layer indices; it should likely be h^l(t) for l = 1, ..., L.","section":"Equation (1)"},{"comment":"Equation (6) uses n in the superscript of the feature set definition, but the attention window over generated tokens should range over the generated sequence length T, not the prompt length n; please clarify the indexing.","section":"Equation (6)"},{"comment":"Table 6 includes an 'Accuracy' column with values such as 72.9% and 87.4%, but neither the caption nor the text defines what this accuracy measures or on which split it is computed; please add an explanation.","section":"Table 6"},{"comment":"The annotation budget statement in Section 4 ('around $100' for 3,300 prompts) is difficult to reconcile with the 'approximately $4000' total API expenses in Appendix B; please clarify the accounting.","section":"Section 4 vs. Appendix B"},{"comment":"In Table 7, the row for UHead (Factoscope) is missing a warmup value and the architecture cell appears to contain '0.05' followed by a dash, which is ambiguous; please align and label all hyperparameter columns consistently.","section":"Table 7"},{"comment":"Table 8 reports memory footprints for UHead (only hidden states) and UHead (att. + prob. + hs.) but does not provide the corresponding computational overhead percentages that are given for the other rows; please complete these entries.","section":"Table 8"}],"recommendation":"major_revision","confidential_remarks":"The paper is built on a large amount of the authors' own prior infrastructure (LM-Polygraph, Fadeeva et al., Vashurin et al.), which is understandable given the shared codebase, but the baselines should be checked to ensure they are not disadvantaged by implementation choices. The decisive issue is the GPT-4o label oracle; if the authors cannot supply human-validated labels, they should at minimum reframe the contribution as agreement with an LLM-based oracle and temper the state-of-the-art claim accordingly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read this paper. It's a competent empirical study of a practical hallucination detector: a small Transformer-based UQ head that takes raw attention maps and top-k token log-probs from a frozen LLM and predicts whether a claim is unsupported. They pre-train heads for Mistral, Llama, and Gemma 2 and release them. The main result—that this beats unsupervised UQ and supervised baselines like Factoscope and Lookback lens on in-domain and out-of-domain English data, and also transfers to Russian, Chinese, and German—is plausible and well demonstrated within their setup.\n\nWhat's genuinely new is the combination, not the components. Attention features (Lookback lens) and supervised probes (SAPLMA, Factoscope) are known. The contribution is putting raw attention maps and top-k log-probs into a Transformer encoder, and scaling the training data via a GPT-4o annotation pipeline that splits outputs into atomic claims and labels them supported/unsupported. They also show attention features overfit less than hidden states, which is a useful finding for the field.\n\nThe empirical design is better than average: baselines are retrained and tuned on the same biography data, test domains are genuinely separate, hyperparameters are selected on a validation set, and the overhead is measured (about 5% inference time, 40 MB memory). They release code and models, which is real value. The citation pattern is also fair; they build on the relevant literature and the self-citations point to their own released infrastructure, which is appropriate.\n\nThe soft spots are real but not fatal. First, all labels—training and test—come from GPT-4o via chain-of-thought. If GPT-4o's supportedness judgments are systematically biased for certain entities or languages, the PR-AUC numbers quantify agreement with GPT-4o, not veridical hallucination detection. The authors acknowledge this in the Ethical Considerations but don't mitigate it with any human-annotated subset. Second, the main tables report single PR-AUC values with no variance or significance tests. The margins over Factoscope are 5 pps in-domain and as little as 1 pp on landmarks, so run-to-run noise could change some conclusions.\n\nThese concerns don't sink the method. The UQ head is a useful plugin and I'd happily use the released models. But the SOTA claim should be softened until there's variance-aware evaluation and at least a small independent label check.\n\nThis paper deserves a serious referee. I'd send it to reviewers with a request to focus on oracle bias and uncertainty reporting. It's a solid contribution to the hallucination-detection toolkit.","headline":"A useful plug-and-play hallucination detector with broad evaluation, but the GPT-4o label oracle and missing variance reporting temper the SOTA claim.","tokens_in":16304,"tokens_out":2573,"would_cite":true,"duration_ms":25082,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A pre-trained Transformer head that reads a frozen LLM's attention maps and top-k token probabilities achieves state-of-the-art claim-level hallucination detection, including out-of-domain and cross-lingual settings, with about 5% overhead.","keywords":["hallucination detection","uncertainty quantification","attention maps","claim-level detection","pre-trained head","large language models","cross-lingual generalization"],"falsifier":"Re-annotate a sample of the test claims with human fact-checkers using verifiable sources, then recompute PR-AUC for UHead and the strongest baselines on those human labels; if UHead's advantage shrinks or reverses, the reported state of the art is an artifact of the GPT-4o labeling protocol.","tokens_in":15225,"feed_emoji":"🎯","tokens_out":9091,"duration_ms":80501,"temperature":0.7,"pith_summary":"This paper claims that a small supervised module called an uncertainty quantification head (UHead) detects hallucinated atomic claims — indivisible factual statements in an LLM's generated text — more accurately than existing unsupervised uncertainty scores and supervised probes. UHead reads the frozen LLM's raw attention weights to the handful of preceding tokens and the log-probabilities of the top candidate tokens, then passes them through a one- or two-layer Transformer encoder that marks which tokens belong to the claim being scored. Trained on native LLM outputs labeled by GPT-4o, UHead reports the best precision-recall area under the curve for claim-level supported/unsupported classification, both in-domain and out-of-domain, and on Russian, Chinese, and German prompts when trained only on English biographies. The design is practical because it adds roughly 5 percent inference overhead and about 40 MB of parameters to a multi-billion-parameter LLM.","feed_headline":"Attention add-on outperforms uncertainty methods at spotting hallucinations","feed_subtitle":"UHead flags unsupported claims in LLM output across domains and languages with about 5% extra compute.","key_machinery":"The central object is the UHead architecture: a Transformer encoder mounted on a frozen LLM. Its input is a token-level feature vector that concatenates, for each attention head and layer, the raw attention weights from the current token to the previous $k$ tokens (zero-padded where positions do not exist) with the logarithms of the top-$m$ next-token probabilities. A learned embedding is added to each token to indicate whether it belongs to the claim being scored, so the encoder can attend selectively to claim-relevant context. The encoder outputs are mean-pooled over the claim's tokens and fed to a two-layer classifier trained with binary cross-entropy. The load-bearing mechanism is the raw attention map itself: unlike the aggregated lookback ratios of prior work, or hidden-state features that the authors find overfit after 1-3 epochs, flattened attention weights to a short window of preceding tokens continue to generalize, and the per-head analysis suggests a minority of attention heads carry the hallucination signal.","core_discovery":"On the paper's own terms, the discovery is that the signal for whether an atomic claim is unsupported lives primarily in the LLM's attention maps, not in its hidden states, and that a contextualized Transformer head can extract that signal. The authors formalize claim-level hallucination detection as a function $U(c_i\\mid x,y)$ assigning each claim a probability of being unsupported. The head consumes flattened attention weights from every attention head and layer to the previous $k$ tokens (empirically $k=2$ to $5$ suffices) concatenated with the logarithms of the top-$m$ token probabilities, plus a learned embedding that indicates claim membership. In their experiments on eight English test domains, UHead achieves the highest PR-AUC among all baselines; for Mistral 7B it beats the best unsupervised method (CCP) by 16 percentage points in-domain and by up to 23 points out-of-domain, and it surpasses the closest supervised competitor (Factoscope) by 5 points in-domain. For Gemma 2, a head trained only on English biographies improves over the best unsupervised method by 9-13 percentage points on Russian, Chinese, and German test sets. The authors read these results as showing that attention-based features generalize across domains and languages while hidden-state features overfit.","pith_inferences":["If the evaluation labels are swapped from GPT-4o to human fact-checkers or another oracle, the ranking of methods could shift; the paper's headline numbers should be read as measuring agreement with GPT-4o's supportedness judgments.","The finding that only 2-5 preceding tokens matter suggests hallucination tendency is a local attention phenomenon; a targeted experiment could test whether removing or reweighting the high-correlation heads identified in the paper changes generation-time confidence without hurting fluency.","The same head design could transfer to other claim-bearing tasks such as factual consistency of summaries, retrieval-augmented answers, or machine-translation confidence, since its features are not domain-specific.","Because the head relies on attention and token probabilities rather than hidden states, it points toward a lightweight direction for uncertainty scoring in settings where only attention maps are accessible."],"forward_implications":["Claim-level hallucination detection can be added to a frozen LLM without retraining it or altering its generated text, so deployment can be a drop-in adapter.","Because attention features generalize, a detector trained on one domain (biographies) transfers to movies, books, inventions, events, landmarks, cities, and artworks without retraining.","English-only training transfers to Russian, Chinese, and German prompts, at least for Gemma 2 9B, suggesting the uncertainty signal is at least partly language-independent.","The low overhead (about 5% extra compute and 40 MB) means per-token uncertainty scoring can run in the same generation pass in practical applications.","Adding more diverse training domains yields small further gains, so scaling the annotated data is a direct route to better detectors."],"supporting_citations":[{"why":"Supplies the SAPLMA baseline that UHead must outperform and early evidence that hidden states carry hallucination information.","marker":"Azaria and Mitchell (2023)"},{"why":"Supplies the Factoscope baseline and its rich feature set, the closest supervised competitor in the comparisons.","marker":"(He et al., 2024b)"},{"why":"Supplies the Lookback lens attention-ratio baseline and the observation that attention features are informative for hallucination detection.","marker":"(Chuang et al., 2024)"},{"why":"Provides the claim extraction and annotation prompts used to build training data and the CCP unsupervised baseline.","marker":"(Fadeeva et al., 2024)"},{"why":"Provides the cross-lingual test prompts (Russian, Chinese) and the two-stage GPT-4o annotation protocol underlying the labels.","marker":"(Vashurin et al., 2024)"},{"why":"Supplies the evaluation harness used to compare methods consistently and reproduce uncertainty metrics.","marker":"(Fadeeva et al., 2023)"},{"why":"Chain-of-thought prompting is the reason the two-stage claim labeling outperforms one-stage labeling in the annotation pipeline.","marker":"(Wei et al., 2022)"}],"fun_headline_variants":["Attention maps expose hallucinations in LLM outputs with UHead","Pre-trained UQ heads spot LLM lies across languages","Attention-based UQ heads beat unsupervised methods at claim detection","UHead: attention maps catch LLM hallucinations in any language","Transformer head on attention maps detects unsupported claims"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The training and evaluation labels come from GPT-4o's two-stage supported/unsupported judgments about each claim; if those judgments do not track real hallucination, the reported scores measure agreement with GPT-4o rather than true detection quality.","fun_headline_variants_meta":{"raw":{"variants":["Attention maps expose hallucinations in LLM outputs with UHead","Pre-trained UQ heads spot LLM lies across languages","Attention-based UQ heads beat unsupervised methods at claim detection","UHead: attention maps catch LLM hallucinations in any language","Transformer head on attention maps detects unsupported claims"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000363,"raw_usage":{"total_tokens":1991,"prompt_tokens":1012,"completion_tokens":979,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":628,"completion_tokens_details":{"reasoning_tokens":899}},"tokens_in":628,"tokens_out":979,"duration_ms":6847,"temperature":1.0,"reasoning_tokens":899,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:00:42.067058+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-annotate a sample of the test claims with human fact-checkers using verifiable sources, then recompute PR-AUC for UHead and the strongest baselines on those human labels; if UHead's advantage shrinks or reverses, the reported state of the art is an artifact of the GPT-4o labeling protocol.","supporting_citations":[],"review_version":1}