{"id":"843d46ae-75ec-4283-8238-0f6867baa506","arxiv_id":"2412.17056","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"HalluRAG provides a recency-controlled dataset for closed-domain hallucination detection and shows that intermediate activation values carry hallucination signals as strongly as contextualized embeddings.","lead":"The authors built HalluRAG, a dataset of RAG prompts based on Wikipedia articles published after a February 2024 cutoff, so the tested LLMs could not have memorized the answers. They trained small classifiers on the models' internal states and report up to 75% accuracy at spotting hallucinated sentences, and show answerable and unanswerable questions behave differently.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported accuracies may be inflated by sentence-level data leakage, because the paper never states whether train/test splits are at the prompt or sentence level.","rationale":"The reader's formal weakest-assumption field targets recency contamination, which is a real concern but not the most direct threat to the empirical claim. The reader's own rationale, however, identifies the unspecified train/test split as the main methodological issue, and that is the load-bearing problem here: it can invalidate the reported numbers regardless of whether the recency assumption holds. The dataset construction and labeling pipeline are otherwise careful, with public code and a human benchmark for the GPT-4o labels, so the concerns are not about fabrication or poor faith. But the absence of an explicit split granularity is an easily checkable methodological gap. If the split is prompt-level, the results stand and the verdict should remain conditional only on weaker issues such as cross-dataset generalization. If it is sentence-level, the reported accuracies are unreliable and the central claim about IAV effectiveness is not established. I therefore keep the verdict as CONDITIONAL, with the condition being a prompt-level split check.","tokens_in":20825,"tokens_out":2437,"duration_ms":25479,"concrete_test":"Inspect the released code and dataset split logic. If the split is at sentence level, retrain the same MLPs with a prompt-level split: assign all sentences from one generated response to exactly one of train, validation, or test, then compare the resulting accuracies to Tables 1 and 3. If accuracy drops substantially (e.g., below 55% for answerable cases), sentence leakage is confirmed and the headline detection claim is not established.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Section 4.2 describes oversampling and MLP training but does not specify the unit of the train/test split. HalluRAG stores internal states per generated sentence, and each RAG prompt typically yields a multi-sentence response. If sentences from the same prompt are randomly assigned across train and test, the classifier can memorize prompt-level cues — including the question, context snippet, and answerability — rather than learning a general hallucination signal. Because internal states for sentences in one response share the same prompt and surrounding generated text, this leakage can inflate the headline accuracies in Table 1 (up to 75%) and the near-perfect unanswerable results in Table 3. The held-out parameter experiment in Table 11 does not address this, since it withholds configuration values but still allows the same prompt's sentences to straddle the split. The central claim that IAVs detect hallucinations as effectively as CEVs depends on these accuracies being genuine estimates of generalization, and this is currently unverified.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces HalluRAG, a dataset for sentence-level closed-domain hallucination detection in RAG settings, where 'closed-domain' is operationalized through recency: only Wikipedia sentences with references dated after February 22, 2024 are used, under the assumption that this information was not in the training data of LLaMA-2 and Mistral models. For each RAG prompt, the authors generated answers from several LLM configurations, extracted contextualized embedding vectors (CEVs) and intermediate activation values (IAVs) from middle and last decoder blocks, and labeled each sentence as hallucinated or not using a GPT-4o chain-of-thought prompt with a four-boolean truth table. The labeler is validated against 274 human judgments (F1 96.05%). MLP classifiers trained on the internal states achieve test accuracies up to about 75% overall, with int4 quantizations performing best; separate classifiers for answerable and unanswerable prompts improve accuracy, reaching near 100% for Mistral-7B on unanswerable prompts. Cross-dataset evaluation against RAGTruth shows poor generalization, which the authors report as a limitation. The central claims are that IAVs are as effective as CEVs for hallucination detection and that answerable and unanswerable prompts are encoded differently in internal states.","tokens_in":20890,"tokens_out":3843,"duration_ms":36573,"significance":"If the methodology is sound, HalluRAG would be a useful resource: it provides a recency-based operationalization of closed-domain hallucinations, publicly releases data and code, validates the automatic labeler against human judgments, and reports results averaged over ten training runs. The comparison of IAVs with CEVs is a legitimate and understudied question, and the honest reporting of poor cross-dataset generalization is a strength. However, the significance of the headline accuracies depends on whether the reported numbers reflect genuine generalization or leakage from the data-splitting procedure. The paper's own Discussion acknowledges limited diversity and close wording between GPT-4o-generated questions and Wikipedia answers, which tempers the practical claims. With a clarified and leakage-free evaluation, the dataset and findings would be a modest but solid contribution to white-box hallucination detection.","major_comments":[{"comment":"The paper never states whether train/validation/test splits are performed at the sentence level or at the prompt/response level. Because HalluRAG stores internal states per generated sentence and each RAG prompt yields a multi-sentence response, a sentence-level random split can place sentences from the same response in both training and test sets. In that case the MLP can memorize prompt-level cues—the question, the context snippet, the answerability, and the surrounding generated text—rather than learning a general sentence-level hallucination signal. This would inflate the test accuracies in Table 1 and the near-perfect unanswerable accuracies in Table 3. The held-out parameter experiment in Table 11 does not resolve the issue, since it withholds configuration values but still permits the same prompt's sentences to straddle the split. Please specify the split unit explicitly and, if the current experiments use sentence-level splits, re-run the evaluation with prompt-level splits and report both sets of numbers.","section":"§4.2, Tables 1 and 3"},{"comment":"The recency criterion is load-bearing: the paper defines closed-domain hallucinations as hallucinations about information the LLM 'could not have seen during training,' and this definition is used to contrast HalluRAG with datasets like RAGTruth. The only evidence that the post-February-22-2024 Wikipedia sentences are absent from training data is the reference timestamps. Training corpora are not limited to Wikipedia snapshots available at the reference dates; post-cutoff content can enter through later web crawls, page histories, or indirect paraphrase in other sources. Please provide a contamination check beyond timestamps—for example, perplexity or memorization probes on the candidate sentences, or a manual audit of a sample against known training corpora—and state explicitly which training-data cutoffs are assumed for LLaMA-2 and Mistral-7B. Without such a check, the central contrast between closed-domain and parametric-knowledge hallucinations is not fully established.","section":"§4.1, step 1"},{"comment":"The claim that answerable and unanswerable prompts are encoded differently is supported by showing that separate classifiers yield higher accuracies than a single classifier. However, this experimental design does not isolate an encoding difference from task difficulty or from the fact that the answerable and unanswerable partitions have different label distributions and different surface forms. As the authors themselves note in §5.3, for unanswerable prompts the classifier may mainly detect whether the model produced an 'I don't know' response, and Table 11 shows that training on answerable questions alone leads to near-chance accuracy on the full test set. A stronger test would be to train a single classifier with answerability as an explicit input feature, or to compare probing results on matched internal states from answerable and unanswerable prompts while controlling for response length and lexical content. As it stands, the 'encoded differently' claim is an interpretation rather than a demonstrated property of the internal states.","section":"§5.3 and Table 11"}],"minor_comments":[{"comment":"The phrase 'after a specified cut-off data' should read 'cut-off date'.","section":"§6, Discussion"},{"comment":"The abbreviation 'IA Vs' is inconsistently spaced; please use a consistent form such as 'IAVs' or 'IA Vs' throughout.","section":"Throughout"},{"comment":"The table caption uses 'H-H-R' and 'R-R-H' without defining the acronyms in the caption; please spell them out (e.g., 'trained and validated on HalluRAG, tested on RAGTruth').","section":"Table 2"},{"comment":"RAGTruth is mentioned in the Related Work section but is not described in enough detail for readers to understand its annotation scheme, model coverage, or why it is a suitable cross-dataset test; please add a brief description and citation context.","section":"§3, Related Work"},{"comment":"The text says that the MLP structure follows MIND and SAPLMA, but the input size and hidden dimensions are only given as a sequence (input_size—256—128—64—1); please state the actual input dimension per internal state type, since it is needed to reproduce the architecture.","section":"§4.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is honest about its limitations and releases data and code, which is commendable. The main issue is the unspecified data-split unit, which is a standard reproducibility and validity concern that the authors can address by clarifying or re-running the evaluation. The recency-based contamination assumption also needs a direct check. These are fixable within the manuscript's scope, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Jan,\n\nQuick take on 2412.17056. The genuinely new thing here is the IAV result: a probe trained on intermediate MLP activations detects closed-domain hallucinations about as well as one trained on contextualized embeddings. That is worth knowing. The HalluRAG dataset is also a thoughtful contribution — using Wikipedia reference timestamps to select post-cutoff sentences is a clean way to isolate knowledge the model cannot have memorized, and the GPT-4o labeler is validated against 274 human judgments with high agreement. Code and data are public.\n\nThe soft spot is the one the stress test flags, and it is real. Section 4.2 describes oversampling and balancing across answerability and hallucination for train/validation/test, but never states whether the split is at the prompt or sentence level. Since HalluRAG stores states per generated sentence, a sentence-level split would let sentences from the same response appear in both training and test. The classifier could then pick up prompt-level cues — the question, the retrieved chunks, whether the question is answerable — instead of a general hallucination signal. That would inflate the 65–75% numbers in Table 1 and the near-perfect unanswerable scores in Table 3. The held-out parameter experiment in Table 11 does not fix this, because a withheld chunk size still allows the same prompt's sentences to straddle the split. I don't think this is fraud — the paper is transparent about its limits — but it is an unverified methodological detail on which the headline claim rests.\n\nThe recency assumption is a second, softer spot. The paper relies on reference timestamps after Feb 22, 2024 as proof the model never saw the content. That is reasonable but not airtight; a contamination check or at least a discussion of near-duplicate leakage would help.\n\nI want to be fair: the paper does not oversell. It reports only 60–75% accuracy, shows poor cross-dataset generalization, and explicitly says the unanswerable separation mostly detects prompt answerability. Those are honest limitations. The IAV finding and the dataset are worth knowing regardless of the split issue, but I would not trust the accuracy numbers until the split is clarified.\n\nBottom line: this deserves referee time, but the first reviewer request should be to specify the split unit and, ideally, rerun with prompt-level grouping. If the numbers survive that, it is a solid incremental contribution. As it stands, conditionally acceptable — and I'd cite the dataset with a caveat, not the raw accuracy numbers.","headline":"A useful IAV result and a well-built recency-controlled dataset, but the headline accuracies rest on an unspecified split unit that could allow sentence-level leakage.","tokens_in":21493,"tokens_out":3255,"would_cite":true,"duration_ms":28674,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper introduces HalluRAG and claims small MLPs trained on a single layer's internal states detect closed-domain RAG hallucinations with up to about 75% accuracy, and that answerable and unanswerable prompts are encoded differently.","keywords":["hallucination detection","retrieval-augmented generation","internal states","closed-domain hallucinations","recency","sentence-level classification","contextualized embeddings","intermediate activations"],"falsifier":"A decisive test is a contamination probe: take HalluRAG's answerable prompts, remove the retrieved context, and ask the same LLM directly at temperature 0; if the model answers a substantial share correctly from parametric knowledge, those items were not truly unseen, and the classifiers' accuracies should be re-measured on the clean subset. A second decisive probe is to strip \"I don't know\" and apology phrases from unanswerable test sentences; if the near-100% unanswerable accuracy collapses, the classifier is reading refusal style rather than a grounding signal.","tokens_in":20531,"feed_emoji":"🧠","tokens_out":10525,"duration_ms":87261,"temperature":0.7,"pith_summary":"This paper introduces HalluRAG, a dataset that pairs RAG prompts with sentence-level labels for whether the model's answer is grounded, built from Wikipedia facts dated after February 22, 2024 so the models cannot have learned them during training. The central claim is that a small classifier trained on a single layer's internal states—either the final-token contextualized embedding or the intermediate activation inside a decoder MLP—can detect these closed-domain hallucinations, with test accuracy up to about 75% on Mistral-7B-Instruct-v0.1. The paper also claims that intermediate activation values work as well as contextualized embeddings, and that answerable and unanswerable prompts are encoded differently: separate classifiers for the two categories improve accuracy substantially, with near-100% accuracy on unanswerable prompts for Mistral-7B. A sympathetic reader would care because this points toward reference-free, white-box hallucination detection in RAG systems without needing to audit the model's training data. The authors are explicit that the classifiers generalize poorly across datasets, which they read as a need for more diverse hallucination corpora.","feed_headline":"Internal states flag RAG hallucinations at up to 75% accuracy","feed_subtitle":"Facts the model never saw in training leave a readable trace in its layers.","key_machinery":"The machinery is a controlled dataset plus a white-box classifier. HalluRAG is built from English Wikipedia sentences whose reference timestamps postdate February 22, 2024; each sentence becomes an answerable RAG prompt (relevant chunk retrieved) and an unanswerable twin (unrelated chunk retrieved), spanning three prompt templates, three chunk sizes, and one, three, or five chunks. Sentences from LLaMA-2-7B-Chat and Mistral-7B-Instruct-v0.1 are labeled hallucinated versus non-hallucinated by an auto-annotator using four Boolean checks—conflicting, grounded, has_factual_information, and no_clear_answer—with a truth table keyed to answerability, validated against 274 human labels at an F1 of 96.05%. The signal comes from internal states extracted at the last token: contextualized embedding vectors, the model's hidden states, and intermediate activation values, the activations inside the decoder MLP blocks, taken from the middle and last decoder layers. The classifier is a four-layer MLP with ReLU and sigmoid, trained with early stopping and evaluated as ten independent runs.","core_discovery":"On its own terms, the paper's central discovery is that closed-domain hallucinations—answers that are ungrounded because the relevant facts emerged after the model's training cutoff—leave a readable trace in the model's internal states. A binary MLP fed a single vector from the last token, whether a contextualized embedding vector (CEV) or an intermediate activation value (IAV) from a decoder block's MLP, classifies hallucinated versus grounded sentences at test accuracies of roughly 60% to 75% across LLaMA-2-7B-Chat and Mistral-7B-Instruct-v0.1, with the highest accuracies on int4-quantized checkpoints. Separating answerable from unanswerable prompts raises LLaMA-2-7B accuracies to above 75% for answerable and above 80% (up to 90%+ in some quantizations) for unanswerable, while Mistral-7B reaches near-100% on unanswerable prompts. The paper interprets the answerability split as evidence that the two situations are encoded differently, not that the task has become trivial, since answerable classification remains moderately hard.","pith_inferences":["The paper leaves implicit that the answerability distinction could be used as a routing step: a small classifier could first decide whether the retrieved chunk answers the question, then apply a specialized hallucination detector tuned to that regime.","The near-100% unanswerable results may partly reflect surface markers such as \"I don't know\" or apology phrasing; a natural extension would ablate those templates to measure how much of the signal is truly about grounding.","Because provenance by timestamp is only a proxy, a contamination probe—asking the model the HalluRAG questions with no retrieved context and checking how often it already knows the answer—would directly test the closed-domain assumption and could be added to the dataset release.","If int4 quantization consistently improves detection, compressed models may be the practical deployment target for internal-state hallucination monitors, though the paper does not explain why quantization helps."],"forward_implications":["Hallucination detection in RAG can be done at sentence level without an external reference, using only the internal states recorded while the answer is generated.","Intermediate MLP activations carry roughly the same hallucination signal as the more commonly used contextualized embeddings, so future detectors need not restrict themselves to final hidden states.","Separate classifiers for answerable and unanswerable prompts outperform a single classifier, suggesting that production RAG systems should route sentences by prompt answerability before detection.","Because HalluRAG-trained classifiers transfer poorly to RAGTruth and vice versa, robust hallucination detection will require larger, more diverse annotated corpora rather than just one dataset.","Prompt template choice measurably changes hallucination rate, from roughly 16% to 32–40% for LLaMA-2-7B depending on template, so prompt design is itself a hallucination-reduction lever."],"supporting_citations":[{"why":"Defines the internal-state approach showing that an LLM's hidden states indicate whether it is lying, and supplies the template-based classifier that HalluRAG extends and compares against.","marker":"Azaria and Mitchell (2023)"},{"why":"Provides the MIND classifier architecture and the contextualized-embedding baseline that the paper extends with intermediate activation values.","marker":"Su et al. (2024)"},{"why":"Supplies the RAGTruth corpus used for the paper's cross-dataset generalization tests in both directions.","marker":"Wu et al. (2023)"},{"why":"The GPT-4o model used to generate questions, extract quoted answers, and auto-label hallucinated sentences in the HalluRAG pipeline.","marker":"OpenAI et al. (2024)"},{"why":"Releases the LLaMA-2-7B and LLaMA-2-13B chat models whose internal states populate HalluRAG.","marker":"Touvron et al. (2023)"},{"why":"Releases Mistral-7B-Instruct-v0.1, the model that yields the highest hallucination-detection accuracies in the paper.","marker":"Jiang et al. (2023)"},{"why":"Supplies the chain-of-thought prompting technique used in the GPT-4o auto-labeling procedure.","marker":"Wei et al. (2023)"}],"fun_headline_variants":["LLM internal states reveal hallucinations on post-cutoff facts","Catch fabrications by reading a model's hidden layers","Neural traces expose lies about unseen training data","75%+ accuracy detecting hallucinations via internal signals","HalluRAG: internal states flag closed-domain hallucinations"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that reference dates after February 22, 2024 guarantee the LLMs never saw those Wikipedia facts during training; if any post-cutoff content leaked in through web crawls, page history, or paraphrases, the dataset stops being closed-domain and the contrast with parametric-knowledge hallucinations is undermined.","fun_headline_variants_meta":{"raw":{"variants":["LLM internal states reveal hallucinations on post-cutoff facts","Catch fabrications by reading a model's hidden layers","Neural traces expose lies about unseen training data","75%+ accuracy detecting hallucinations via internal signals","HalluRAG: internal states flag closed-domain hallucinations"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000269,"raw_usage":{"total_tokens":1659,"prompt_tokens":1023,"completion_tokens":636,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":639,"completion_tokens_details":{"reasoning_tokens":559}},"tokens_in":639,"tokens_out":636,"duration_ms":6323,"temperature":1.0,"reasoning_tokens":559,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T05:49:23.284902+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A decisive test is a contamination probe: take HalluRAG's answerable prompts, remove the retrieved context, and ask the same LLM directly at temperature 0; if the model answers a substantial share correctly from parametric knowledge, those items were not truly unseen, and the classifiers' accuracies should be re-measured on the clean subset. A second decisive probe is to strip \"I don't know\" and apology phrases from unanswerable test sentences; if the near-100% unanswerable accuracy collapses, the classifier is reading refusal style rather than a grounding signal.","supporting_citations":[{"cited_title":"The internal state of an llm knows when it's lying, 2023","cited_arxiv_id":null,"evidence_quote":"Defines the internal-state approach showing that an LLM's hidden states indicate whether it is lying, and supplies the template-based classifier that HalluRAG extends and compares against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The GPT-4o model used to generate questions, extract quoted answers, and auto-label hallucinated sentences in the HalluRAG pipeline."}],"review_version":1}