{"id":"d72a801c-bafa-4c87-b6ca-2dcea7aadfde","arxiv_id":"2505.09031","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":2.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"A small benchmark study claims self-verification is the best hallucination-reduction method, but the evaluation leaks the ground-truth answer into the verification prompt.","lead":"This paper compares four known tricks for reducing hallucinations in large language models: chain-of-thought prompting, retrieval-augmented generation, self-consistency, and self-verification. It reports that self-verification works best, but the verification prompt hands the model the correct answer, so the headline result is not credible.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Self-verification's reported advantage is not identifiable because the verification prompt includes the ground-truth answer, allowing label leakage.","rationale":"The reader's weakest assumption identifies the same load-bearing flaw: including the ground-truth answer in the self-verification prompt invalidates the claimed best performance. I agree with the REJECT verdict. In addition to §4.7, Algorithm 1 uses ground-truth y to label each self-consistency sample, so the HaluEval results are also compromised. The paper provides no code or data to re-evaluate, and the 500-sample, threshold-tuned protocol would in any case need error bars. The core scientific conclusion is therefore not supported. Verdict remains REJECT.","tokens_in":7626,"tokens_out":3939,"duration_ms":37458,"concrete_test":"Re-run the §6.1 FEVER and TruthfulQA experiments with the §4.7 self-verification prompt modified to omit the ground-truth answer (keep only the query, the model's own generated answer, and the retrieved documents), scoring with the same metrics. In addition, run a control condition where a randomly selected ground-truth answer from a different sample is inserted in place of the true one; if verification decisions track the inserted label, the model is exploiting the leaked answer. If self-verification no longer beats RAG+CoT by the reported margins, the headline claim is an artifact of label leakage.","verdict_should_be":"REJECT","load_bearing_attack":"The central claim—that self-verification is the best-performing mitigation—rests on the evaluation protocol in §4.7. There the verification prompt is described as containing 'the original query, the model's generated answer, the ground truth answer, and some supporting documents' (and Figure 4 shows the correct answer entering the verification stage). A verifier with the ground-truth answer in its input is not performing self-verification; it can label the generated answer as correct simply by matching the provided gold answer. The reported FEVER ~90% and TruthfulQA MC2 ~80% therefore measure answer-prompted classification, not an intrinsic property of self-verification. The same leak appears in Algorithm 1, where each sampled response is compared against ground-truth y with a cosine-similarity threshold to decide 'factual' vs. 'hallucinated'; the resulting HaluEval rates are not independent measurements of hallucination. Because the headline comparison is built on labels that are available to the method, the conclusion cannot be trusted as a comparison of mitigation strategies.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper empirically compares hallucination-mitigation strategies for LLMs: baseline, chain-of-thought (CoT), retrieval-augmented generation (RAG), CoT+RAG, self-consistency, and self-verification. It evaluates GPT-3.5-Turbo, LLaMA-2-7b, and DeepSeek-R1 on 500 samples each from HaluEval, FEVER, and TruthfulQA, using hallucination rate, label accuracy, and a similarity-based MC2 proxy. The paper reports that self-verification achieves the best overall performance (e.g., roughly 90% accuracy on FEVER and 80% MC2 on TruthfulQA) and concludes in Section 6.2 that self-verification is the most robust approach. The experimental protocols are described in Sections 4.4-4.7, evaluation settings in Section 5, and results in Section 6.","tokens_in":7806,"tokens_out":6475,"duration_ms":61632,"significance":"If the comparison were valid, the paper would provide practically useful evidence about which mitigation stack to adopt, and the multi-model, multi-dataset setup is a reasonable starting point. The manuscript clearly describes its pipeline and identifies several engineering choices (e.g., retrieval counts, temperature, token limits) that matter in practice. However, the central empirical claim is currently not supported: the self-verification protocol gives the model the ground-truth answer, so the reported advantage of self-verification is an artifact of answer leakage rather than a measured property of the method. In addition, the lack of held-out tuning, small sample sizes, and absence of statistical uncertainty make the ranking unreliable. As it stands, the paper does not establish which method is most effective for reducing hallucination.","major_comments":[{"comment":"The verification prompt for self-verification explicitly includes the ground-truth answer from the dataset, along with the query, the model-generated answer, and retrieved documents. A model given the gold answer can label a response as correct simply by detecting that it matches the provided gold answer, so the FEVER accuracy around 90% and TruthfulQA MC2 around 80% in Section 6.1 do not measure an independent self-verification capability. The conclusion in Section 6.2 that 'Self-Verification had the best performance' is therefore an artifact of answer leakage rather than a property of the method. The authors should rerun the verification experiments with the gold answer removed from the verification prompt (e.g., verifying only against retrieved evidence) and report those numbers as the self-verification results.","section":"Section 4.7, Figure 4"},{"comment":"The self-consistency decision in Algorithm 1 uses the ground-truth answer y in the cosine-similarity comparison, and the threshold tau=0.5 is said to be chosen 'through trial and error.' Section 5.1 also describes tuning the CoT prompt, the number of retrieved documents (2, 5, and 10), the temperature, and the max-token limit without specifying any train/validation split. If these choices were made on the same 500-sample evaluation sets used for the final numbers, the reported improvements in Section 6 may be optimistically biased and the method ranking may not generalize. Please state which hyperparameters were selected on held-out data and re-evaluate the final configurations on unseen test samples.","section":"Algorithm 1, Section 4.6; Section 5.1"},{"comment":"Only 500 samples per dataset are evaluated, and no error bars, confidence intervals, or statistical significance tests are reported. Because LLM outputs are stochastic, as the temperature sampling in Section 4.6 acknowledges, the small differences highlighted in Section 6.2 (e.g., 'LLaMA-2 slightly outperforming GPT-3.5-Turbo') may be sampling noise. The authors should report variability across independent runs or bootstrap confidence intervals, and run paired statistical tests for the central comparisons.","section":"Section 4.2; Section 6.1"},{"comment":"The 'MC2' metric used for TruthfulQA is not the standard TruthfulQA MC2; it assigns each response a label by taking the highest cosine similarity to the sets of correct and incorrect reference answers and then computes accuracy. The claimed 'MC2 score of around 80%' in Section 6.1 should either be renamed (e.g., 'similarity-based truthfulness accuracy') or be computed with the official TruthfulQA evaluation script; otherwise the number is not comparable to MC2 results in the literature.","section":"Section 5.3.3"}],"minor_comments":[{"comment":"The title as submitted ('Improving the Reliability of LLMs: Combining Chain-of-Thought Reasoning and Retrieval-Augmented Generation') does not match the arXiv title that includes self-consistency and self-verification; please make the titles consistent.","section":"Title"},{"comment":"The phrase 'for all dataset' should be 'for all datasets.'","section":"Section 4.3"},{"comment":"The majority-vote rule does not specify a tie-breaking behavior when count_factual equals count_hallucinated.","section":"Algorithm 1"},{"comment":"Figure 5 refers to 'our website' without providing a URL; for reproducibility, please include a publicly accessible repository or at least a stable URL.","section":"Figure 5 caption"},{"comment":"Several references have incomplete author lists (e.g., Kojima et al., Wei et al., Liu et al., Singh and Kapoor, and Zhou et al.); please provide full author information or use standard bibliography formatting.","section":"References"},{"comment":"The paper states that 500 samples per dataset were evaluated but does not describe the sampling procedure or whether a fixed random seed was used; please state how the subset was drawn.","section":"Section 4.2"}],"recommendation":"reject","confidential_remarks":"The manuscript does not provide a data or code availability statement; a repository would be needed for any resubmission. The novelty is incremental relative to prior work on CoT+RAG, self-consistency, and self-verification, but my recommendation rests on the methodological validity issues, not on novelty alone. The answer leakage in the self-verification protocol is a load-bearing flaw: without re-running the experiments with a corrected protocol, the central claim cannot be evaluated."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The headline claim is not supported: self-verification is reported as best, but the verification prompt includes the ground-truth answer, so the model is essentially told the correct answer before it judges its own output. I agree with the reader's REJECT, and the stress-test note lands.\n\nWhat the paper does well: it is a cleanly written, systematic comparison of known hallucination-mitigation techniques. The authors are transparent about their pipeline—RAG retrieval per dataset, the CoT prompt selection, the self-consistency voting scheme, and the self-verification setup. For a student project, the execution is careful enough that the non-verification comparisons (baseline vs CoT vs RAG vs RAG+CoT) are plausible, even if underpowered. They also cite the relevant prior work, including CoT-RAG, self-consistency, and self-verification.\n\nThe soft spots are real and load-bearing. Section 4.7 and Algorithm 1 hand the verifier the gold answer y; the model can label its own output as correct simply by matching the provided answer. So the FEVER ~90% and TruthfulQA ~80% numbers reflect answer-prompted classification, not self-verification. Additionally, the cosine threshold tau=0.5 is tuned on the evaluation data (Section 5.1), there are no error bars or statistical tests, only 500 samples per dataset, and the TruthfulQA MC2 heuristic (cosine similarity to answer lists) adds noise. No code or data are provided, so independent verification is impossible.\n\nThe central argument that self-verification is the most robust method does not survive. The paper is a reasonable staging of established techniques, and the non-verification comparisons might be useful to someone building a quick pipeline, but the main result is an artifact. If the authors fix the verification protocol—e.g., verify without seeing the answer—and provide code/data, a revised short paper could be a decent workshop contribution. As is, I would not send it to peer review.","headline":"Self-verification's reported advantage is an artifact of putting the ground-truth answer in the verification prompt.","tokens_in":8376,"tokens_out":2490,"would_cite":false,"duration_ms":23282,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Self-verification is the most reliable fix for LLM hallucination, the paper claims.","keywords":["LLM hallucination","chain-of-thought prompting","retrieval-augmented generation","self-consistency","self-verification","factual accuracy","TruthfulQA","FEVER"],"falsifier":"Run the same self-verification setup without the ground-truth answer in the prompt, keeping the query, generated answer, and retrieved documents. If FEVER accuracy and TruthfulQA MC2 drop to the level of RAG-plus-CoT or lower, the claimed advantage of self-verification depends on answer leakage rather than on independent checking.","tokens_in":1593,"feed_emoji":"✅","tokens_out":2600,"duration_ms":62300,"temperature":0.7,"pith_summary":"This paper asks which of four candidate fixes—step-by-step reasoning, retrieval of external evidence, sampling many answers, or having the model check its own answer—actually reduces LLM hallucination. Testing GPT-3.5-Turbo, LLaMA-2-7b, and DeepSeek-R1 on HaluEval, FEVER, and TruthfulQA, it finds that every intervention beats the unmodified baseline model. Retrieval-augmented reasoning and self-verification perform best, and self-verification is the strongest overall method, reaching roughly 90 percent accuracy on FEVER and 80 percent on TruthfulQA. The paper concludes that post-hoc verification grounded in retrieved evidence is the most effective way to keep model outputs truthful.","feed_headline":"Self-verification beats CoT and RAG at stopping LLM hallucination","feed_subtitle":"In tests on FEVER and TruthfulQA, self-checking beat both reasoning and retrieval alone.","key_machinery":"The mechanism that carries the argument is the verification step in the self-verification pipeline: a second prompt receives the original query, the model's own generated answer, the ground-truth answer from the dataset, and the top five retrieved document chunks, and the model must decide whether the answer is hallucinated or correct. That step outperforms the paper's other interventions because it combines external evidence with an explicit judgement. The other load-bearing component is the self-consistency algorithm, which samples nine responses at varied temperatures, scores each against the reference answer with cosine similarity, applies a 0.5 threshold, and takes a majority vote on whether the output is hallucinated. The RAG pipeline, which embeds document chunks, retrieves the five most similar to the query, and feeds them to the model, supplies the external grounding for both retrieval-based and verification-based methods.","core_discovery":"On the paper's own terms, the central discovery is that self-verification—a second pass in which the model receives its first answer, the ground-truth answer, and retrieved supporting documents and is asked to judge whether the answer is factually correct—is the most effective hallucination-mitigation strategy across the tested settings. The claim is comparative: self-verification outperforms baseline generation, chain-of-thought prompting, retrieval-augmented generation, their combination, and self-consistency voting, with LLaMA-2-7b slightly ahead of GPT-3.5-Turbo. On HaluEval, the best result is a RAG-plus-CoT setup reaching an 11 percent hallucination rate; on FEVER and TruthfulQA, self-verification tops the leaderboard. The paper presents these results as evidence that external grounding plus reflective checking, rather than reasoning alone, is what makes LLM outputs trustworthy.","pith_inferences":["A stricter test would remove the ground-truth answer from the verification prompt; if accuracy collapses, the reported edge of self-verification is leakage rather than checking.","The comparison likely understates what a separate, stronger verifier model could do, since the paper uses the same LLM for generation and verification.","The dependence on a hand-tuned cosine-similarity threshold of 0.5 and on top-5 retrieval suggests results could shift with different embeddings or retrieval granularity.","A domain outside Wikipedia-style fact checking, such as medical or legal reasoning, would test whether verification still holds when retrieved documents are less authoritative."],"forward_implications":["Teams deploying LLMs for fact-oriented tasks should add a self-verification pass with retrieved evidence rather than relying on chain-of-thought prompting alone.","Retrieval plus step-by-step reasoning is a close second and gives the lowest hallucination rate on HaluEval, so external grounding matters even when verification is not used.","All five strategies beat the unmodified model, meaning even lightweight prompting is better than no mitigation at all.","Self-verification is not model-specific: the pattern holds across GPT-3.5-Turbo, LLaMA-2-7b, and DeepSeek-R1, with LLaMA-2-7b slightly ahead.","Because self-consistency requires nine samples per query and self-verification needs retrieval plus a second prompt, the accuracy gains come with added inference cost."],"supporting_citations":[{"why":"introduces chain-of-thought prompting, the reasoning method the paper adapts to all datasets.","marker":"(Wei et al., 2022)"},{"why":"supplies the self-consistency sampling and majority-vote strategy the paper implements with nine samples.","marker":"(Wang et al., 2023)"},{"why":"provides the self-verification idea that the paper's best-performing method is built on.","marker":"(Weng et al., 2023)"},{"why":"contributes HaluEval, the benchmark whose hallucination-rate metric anchors the HaluEval results.","marker":"(Li et al., 2023)"},{"why":"contributes FEVER, the fact-verification dataset used for the supported, refuted, and not-enough-info accuracy results.","marker":"(Thorne et al., 2018)"},{"why":"contributes TruthfulQA and the true-versus-false answer structure behind the MC2 truthfulness score.","marker":"(Lin et al., 2021)"},{"why":"shows that RAG combined with CoT reduces hallucination, the baseline that the paper's RAG-plus-CoT comparison extends.","marker":"(Zhou et al., 2023)"},{"why":"describes the CoT-RAG integration that motivates the paper's combined pipeline.","marker":"(Li et al., 2025)"}],"fun_headline_variants":["Self-verification beats CoT and RAG in hallucination tests","Self-verification outperforms CoT and RAG for LLM accuracy","Self-check with retrieval wins over reasoning alone for LLMs","Self-verification top strategy to cut LLM hallucination","Why self-verification beats CoT and RAG for reliability"],"cache_read_input_tokens":10496,"weakest_assumption_plain":"The load-bearing premise is that giving the model the correct answer inside the verification prompt measures its ability to verify; if the model is merely echoing that answer, the method's best performance is an experimental artifact.","fun_headline_variants_meta":{"raw":{"variants":["Self-verification beats CoT and RAG in hallucination tests","Self-verification outperforms CoT and RAG for LLM accuracy","Self-check with retrieval wins over reasoning alone for LLMs","Self-verification top strategy to cut LLM hallucination","Why self-verification beats CoT and RAG for reliability"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000688,"raw_usage":{"total_tokens":3106,"prompt_tokens":919,"completion_tokens":2187,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":535,"completion_tokens_details":{"reasoning_tokens":2096}},"tokens_in":535,"tokens_out":2187,"duration_ms":15696,"temperature":1.0,"reasoning_tokens":2096,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:41:05.791954+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same self-verification setup without the ground-truth answer in the prompt, keeping the query, generated answer, and retrieved documents. If FEVER accuracy and TruthfulQA MC2 drop to the level of RAG-plus-CoT or lower, the claimed advantage of self-verification depends on answer leakage rather than on independent checking.","supporting_citations":[],"review_version":1}