Pith. sign in

REVIEW 3 major objections 4 minor 4 cited by

Resolving Conflicting Evidence in Automated Fact-Checking: A Study on Retrieval-Augmented LLMs

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Source credibility added at answer generation, not retrieval or ranking, is what lets retrieval-augmented LLMs resolve conflicting evidence in fact-checking.

desk verdict A useful dataset and a plausible research direction, but the headline claim is not supported by the reported numbers and the benchmark's validity is an open question. read the letter →

arxiv 2505.17762 v1 pith:2OSCSDAR submitted 2025-05-23 cs.CL cs.IR

Ziyu Ge , Yuhao Wu , Daniel Wai Kit Chin , Roy Ka-Wei Lee , Rui Cao This is my paper
classification cs.CLcs.IR
keywords conflictingevidencefact-checkingretrieval-augmentedgenerationsourcecredibilitymediabiasmisinformationRAGrobustnessdatasetconstruction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether retrieval-augmented language models can fact-check claims when the retrieved documents disagree, and whether telling the model which sources are credible helps. It introduces CONFACT, a dataset of claims paired with documents that take opposite stances, each labeled with stance and source credibility. The authors find that standard RAG pipelines are fragile under such conflicts: they tend to trust the majority or the most superficially relevant source, and even strong closed-source models show no clear advantage. Injecting source-credibility background at the answer-generation stage, especially with chain-of-thought or explicit 'ignore unreliable sources' instructions, improves accuracy and F1, while credibility-based filtering or ranking helps less or can backfire. The paper concludes that source credibility belongs in the reasoning stage, not in pre-filtering, and that automated credibility estimation remains the limiting factor.

What carries the argument

The load-bearing mechanism is media source background: a textual profile of a source's bias and factual reliability, optionally compressed into a numeric credibility score $s_{\mathrm{cred}}$ by a trained predictor $\pi_\theta$. The paper places this signal at three points in the standard RAG pipeline, document filtering before ranking, ranking via $s_m = s_{\mathrm{rel},m} + \beta s_{\mathrm{cred},m}$, and answer generation with source-aware paragraphs, and shows the generation-stage injection is what works. The dataset CONFACT supplies the controlled test bed: claims whose retrieved documents genuinely oppose each other, with stance labels and credibility ratings attached.

What would settle it

Randomly permute the credibility labels among the retrieved documents and rerun the best source-background-augmented configuration on both CONFACT splits; if accuracy does not drop below the true-credibility condition, then credibility information is not what drives the reported gains. As a second check, have independent fact-checkers adjudicate a sample of CONFACT labels; if a substantial fraction are judged unresolvable from the provided documents, the benchmark itself cannot support claims about fact-checking performance.

Watch

Extended reading notes

Core claim

The paper's central claim is that effectively incorporating source credibility significantly enhances the ability of RAG models to resolve conflicting evidence and improve fact-checking performance. Concretely, the authors show that vanilla retrieval-augmented LLMs are vulnerable when retrieved documents disagree: they over-trust unreliable sources, default to the majority view, and hallucinate answers not supported by the evidence. They then demonstrate that attaching source-background information to evidence at the answer-generation stage, using chain-of-thought prompting or explicit instructions to discard unreliable passages, delivers the largest gains, whereas moving credibility into retrieval filtering or ranking scores can remove useful counter-evidence and reduce accuracy. Expert-verified credibility labels outperform automatically generated ones, so the paper frames better source-credibility estimation as the key open problem.

Load-bearing premise

The whole evaluation assumes that the correct yes/no answer for each CONFACT claim can be determined from the retrieved conflicting documents; the paper's own human study puts human accuracy near chance, so if the labels cannot actually be resolved from this evidence, model accuracy on the benchmark does not measure fact-checking skill.

Editorial extensions

If this is right

  • Fact-checking systems that use RAG should attach credibility information to evidence at the reasoning stage rather than using it to pre-filter documents.
  • Retrieving more documents can hurt rather than help; the paper's ablations point toward fewer, more relevant passages.
  • Automated credibility prediction is the bottleneck: hybrid backgrounds from LLM-based generators did not beat expert-verified labels, and noisy credibility scores misled models.
  • Models' tendency to side with the majority viewpoint persists even with credibility cues, so source-aware prompting corrects but does not eliminate the bias.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The near-chance human performance on CONFACT suggests the benchmark may be measuring how resolvable a dispute is rather than objective factuality; a harder test would keep only claims that independent expert adjudication can settle.
  • A natural extension is calibration: the paper reports accuracy and F1 but not whether source backgrounds improve the model's confidence in its answers.
  • Because credibility is treated as static, a testable extension is to vary source reliability by topic or date and measure whether the method's gains persist.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces CONFACT, a dataset of fact-checking claims paired with retrieved documents that express conflicting stances, drawn from AVERITEC and FactCheckQA and annotated with source credibility information. The authors evaluate retrieval-augmented LLMs (LLaMA-3.1-8B, Qwen-2-8B, Mistral-7B, and GPT-4o-mini) on the ModC and HumC splits of this dataset, and propose several ways to incorporate media source backgrounds into the RAG pipeline: source filtering, credibility-weighted ranking, and source-background augmentation at the generation stage. The central claim is that incorporating source credibility significantly improves fact-checking performance under conflicting evidence. The paper also reports a human evaluation intended as a validity check for the dataset.

Significance. If the central claim holds, CONFACT would be a valuable resource for studying source-critical fact-checking, and the paper would provide useful evidence that RAG systems need explicit credibility reasoning when evidence conflicts. The dataset is publicly released and the construction pipeline (Google retrieval, Wayback archiving, GPT-4 stance annotation, human validation) is described in enough detail to be adapted. The comparison of several incorporation strategies is a sensible design for isolating where credibility information helps. However, the paper's own quantitative and human-validity results currently undercut the central claim: the headline improvements in Section 5.1 are not visible in Table 2, and the human evaluation in Section 5.3 is at chance, which calls into question whether accuracy on CONFACT measures resolution of conflicting evidence at all.

major comments (3)
  1. [§5.1 (RQ2) and Table 2] The text states that "LLaMA-3.1 shows a 10% absolute improvement in F1 score, while Mistral achieves a 5% accuracy improvement" on ModC, but Table 2 does not support these numbers. On ModC, LLaMA-3.1's best baseline F1 is 68.50 (CoT) and its best credibility-aware F1 is 69.30 (Hyb. SBAdirect), a gain of 0.8 points; Mistral's best baseline accuracy is 79.54 (MajV) and its best credibility-aware accuracy is 80.03 (Hyb. SBAdirect), a gain of 0.49 points. Similar discrepancies appear on HumC. Since these figures are the main quantitative evidence for the paper's central claim, they must be corrected or replaced with the specific method pairs that actually produced them.
  2. [§5.3 and Table 4] The human validity check reports 49.45%, 50.34%, and 48.47% accuracy for the three conditions on the 20 sampled claims, statistically indistinguishable from chance. This is the paper's own check of whether CONFACT's ground-truth answers are recoverable from the retrieved evidence, and it fails. If human annotators cannot reliably determine the answer from the provided conflicting documents, then model accuracy on CONFACT may reflect the majority label, source priors, or parametric knowledge rather than evidence resolution. Please provide a more decisive validity test, such as a larger human study with explicit treatment of "unsure" responses, and an evidence-only oracle or no-context baseline that controls for parametric knowledge. Also state how "unsure" responses were scored in the current human evaluation.
  3. [§5.1, Appendix C, and Table 2] The paper reports no confidence intervals, error bars, or significance tests, and the credibility-weighting hyperparameters β=0.8 and γ=0.3 were tuned on the evaluation sets themselves. Since many credibility-aware improvements in Table 2 are only 1-3 absolute points, the claim that incorporating source credibility "significantly enhances" performance is not statistically established. Please add bootstrap or per-claim variance estimates, and tune hyperparameters on a held-out validation split before reporting test-set numbers.
minor comments (4)
  1. [Table 3] In the Top-10 Para. block, the LLaMA-3.1 SBAexp accuracy is written "74,22" and should be "74.22".
  2. [§2.2] The sentence "Despite these advancements, the impact of estimated source credibility in fact-checking is still unknown. has received limited attention." is grammatically broken and should be rewritten.
  3. [§3.4 and Table 2] The paper does not include a simple majority-class baseline; the HumC MajV results (82.93/49.07 for all three models) appear close to the majority-class prior of the 236/287 split. Reporting such a baseline would help interpret the absolute numbers.
  4. [§5.3] The human evaluation does not state whether the four NLP researchers were authors of the paper or blind to the study hypothesis, and it does not give the number of judgments per claim per condition. These details matter for interpreting the chance-level result.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's evaluation is empirical, labels and credibility ratings come from external sources, and no prediction reduces to a fitted input.

full rationale

The central claim—that incorporating source credibility improves RAG fact-checking on conflicting evidence—is an empirical result, not a definitional one. CONFACT's ground-truth labels are inherited from AVERITEC and FactCheckQA (Section 3.1), both external datasets. Credibility ratings come from MBFC or from a score predictor trained on Baly et al. (2018), an independent external dataset; the predictor is then applied to CONFACT sources, so it is not fitted to the target labels. The conflict annotation uses GPT-4 with three prompt variants plus human validation, but this determines which documents are conflicting, not the ground-truth answer; the answer labels are not derived from the stance annotations or credibility scores. The proposed RAG variants (SF, CW, SBA) are evaluated against baselines on the same fixed dataset, and improvements are measured rather than constructed. There is no self-citation chain: the paper cites Schlichtkrull (2024) as prior external work, not as an author's own uniqueness theorem, and no load-bearing argument reduces to a self-citation. The chance-level human performance in Table 4 raises a legitimate benchmark-validity question—whether the retrieved evidence alone determines the label—but that is a correctness/validity concern, not circularity, because the labels were not defined in terms of the evidence or the models' outputs. The use of GPT-4 for claim conversion and stance annotation while also testing GPT-4o-mini in the appendix is a possible contamination/leakage concern, but it does not make any derived result equivalent to its input by construction. Under the rule that only explicit reductions count as circularity, no such step is present.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central empirical claims rest on the quality of external annotations (MBFC, AVERITEC, FactCheckQA, GPT-4) and on the two tuned hyperparameters beta and gamma. No new theoretical entities are postulated. The most fragile underlying assumption is that the ground-truth label is determinable from the retrieved evidence, which the paper's own human evaluation contradicts at chance level.

free parameters (2)
  • beta (credibility weighting) = 0.8
    Used in Eq. 2 to balance relevance and credibility scores; selected 'based on preliminary experiments optimizing both accuracy and macro-F1 performance' with no held-out validation reported.
  • gamma (credibility threshold) = 0.3
    Used in CWhard to binarize credibility scores; tuned on the evaluation data by the same preliminary experiment procedure.
assumptions (4)
  • domain assumption MBFC expert credibility ratings are treated as ground truth for source credibility.
    The paper uses MBFC annotations as both the GT-MB signal and the training source for the Hybrid-MB credibility predictor. No independent validation of MBFC itself is provided.
  • domain assumption The ground-truth labels from AVERITEC and FactCheckQA are correct and appropriately transferred to the binary question format.
    Claims labeled supported/refuted or true/false are mapped directly to yes/no answers, assuming the original datasets' judgments are reliable for the new dataset instances.
  • domain assumption GPT-4's stance classification with three prompt variants and majority voting accurately identifies genuinely conflicting documents.
    Stage 1 of dataset construction relies on GPT-4 to select claims for human validation; if GPT-4 is systematically wrong, the dataset inherits that bias.
  • domain assumption The top-10 Google results archived in one week in September 2024 form a sufficient and representative evidence set for each claim.
    Search results are time-dependent and query-dependent; the paper does not analyze recall or coverage of this retrieval set against the full web.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Resolving Conflicting Evidence in Automated Fact-Checking: A Study on Retrieval-Augmented LLMs." pith.science (2026). https://pith.science/paper/2OSCSDAR

@misc{pith2026250517762,
  author       = {Pith},
  title        = {Pith review of: Resolving Conflicting Evidence in Automated Fact-Checking: A Study on Retrieval-Augmented LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2OSCSDAR}},
  note         = {Machine review of arXiv:2505.17762}
}
read the original abstract

Large Language Models (LLMs) augmented with retrieval mechanisms have demonstrated significant potential in fact-checking tasks by integrating external knowledge. However, their reliability decreases when confronted with conflicting evidence from sources of varying credibility. This paper presents the first systematic evaluation of Retrieval-Augmented Generation (RAG) models for fact-checking in the presence of conflicting evidence. To support this study, we introduce \textbf{CONFACT} (\textbf{Con}flicting Evidence for \textbf{Fact}-Checking) (Dataset available at https://github.com/zoeyyes/CONFACT), a novel dataset comprising questions paired with conflicting information from various sources. Extensive experiments reveal critical vulnerabilities in state-of-the-art RAG methods, particularly in resolving conflicts stemming from differences in media source credibility. To address these challenges, we investigate strategies to integrate media background information into both the retrieval and generation stages. Our results show that effectively incorporating source credibility significantly enhances the ability of RAG models to resolve conflicting evidence and improve fact-checking performance.

Figures

Figures reproduced from arXiv: 2505.17762 by the authors.

Figure 1
Figure 1. The retrieved documents from Google to verify the claim. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. (a) illustrates a general framework of RAG methods in [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. presents a sample from the CONFACT dataset. Each instance in CONFACT comprises a question (converted from its original claim), a ground-truth answer, and a set of con￾flicting evidence. C Implementation Details To handle long retrieved documents effectively, we apply a paragraph-based chunking strategy, where each document is [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Workflow of the Ensemble Method. I. SBAexp Question: [Question to be answered based on the evidence provided] Some evidence below may have been perturbed with wrong information. Find the perturbed passages and ignore them when eliciting the correct answer. Evidence 1: …
Figure 5
Figure 5. Figure 5: Screenshot of Annotation Platform I Media Background Prediction I.1 Demonstrations for Media Background Prediction. As shown in [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: Illustration of the process to generate the media back [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. When Specifications Conflict: A Symmetry-Based Framework for Measuring LLM Preferences

    cs.AI 2026-07 conditional novelty 6.0 of 10

    Under controlled conflicting specifications, LLMs systematically prefer formal and naturalized-formal task descriptions over pure natural language and input–output examples, with domain-specific exceptions.

  2. MEDIAREF: A Public Knowledge Store for Media Background Checks

    cs.CL 2026-07 unverdicted novelty 6.0 of 10

    MEDIAREF is a public, updatable web-document store that lets LLMs generate media background checks more reproducibly and with higher fact recall than zero-shot generation alone.

  3. Courtroom-Style Multi-Agent Debate with Progressive RAG and Role-Switching for Controversial Claim Verification

    cs.CL 2026-03 unverdicted novelty 6.0 of 10

    PROClaim achieves 81.7% accuracy on Check-COVID claim verification by combining courtroom roles, progressive RAG, and multi-judge aggregation, outperforming standard multi-agent debate by 10 percentage points.

  4. HaluNet: Learning Hallucination Risk from Internal Signals in LLM Question Answering

    cs.CL 2025-12 conditional novelty 4.0 of 10

    HaluNet fuses token-level log-likelihood, entropy, and hidden-state embeddings via a multi-branch neural network to rank hallucinated LLM QA answers with reported AUROC up to 0.922 in-domain.

Reference graph

Works this paper leans on

38 extracted references · 30 canonical work pages · cited by 4 Pith papers

  1. [1]

    Query refinement prompts for closed-book long-form QA

    Reinald Kim Amplayo, Kellie Webster, Michael Collins, Dipanjan Das, and Shashi Narayan. Query refinement prompts for closed-book long-form QA . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL , pages 7997--8012, 2023

    work page 2023

  2. [2]

    Glass, and Preslav Nakov

    Ramy Baly, Georgi Karadzhov, Dimitar Alexandrov, James R. Glass, and Preslav Nakov. Predicting factuality of reporting and bias of news media sources. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing , pages 3528--3539, 2018

    work page 2018

  3. [3]

    Glass, and Preslav Nakov

    Ramy Baly, Georgi Karadzhov, Jisun An, Haewoon Kwak, Yoan Dinkov, Ahmed Ali, James R. Glass, and Preslav Nakov. What was written vs. who read it: News media profiling using text analysis and social media context. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020 , pages 3364--3374, 2020

    work page 2020

  4. [4]

    Trusted Source Alignment in Large Language Models

    Vasilisa Bashlovkina, Zhaobin Kuang, Riley Matthews, Edward Clifford, Yennie Jun, William W. Cohen, and Simon Baumgartner. Trusted source alignment in large language models. CoRR , abs/2311.06697, 2023

    work page Pith review arXiv 2023

  5. [5]

    Can llm-generated misinformation be detected? In The Twelfth International Conference on Learning Representations, ICLR , 2024

    Canyu Chen and Kai Shu. Can llm-generated misinformation be detected? In The Twelfth International Conference on Learning Representations, ICLR , 2024

    work page 2024

  6. [6]

    Evaluating entity disambiguation and the role of popularity in retrieval-based NLP

    Anthony Chen, Pallavi Gudipati, Shayne Longpre, Xiao Ling, and Sameer Singh. Evaluating entity disambiguation and the role of popularity in retrieval-based NLP . In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP , pages 4472--4485, 2021

    work page 2021

  7. [7]

    Complex claim verification with evidence retrieved in the wild

    Jifan Chen, Grace Kim, Aniruddh Sriram, Greg Durrett, and Eunsol Choi. Complex claim verification with evidence retrieved in the wild. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), NAACL , pages 3569--3587, 2024

    work page 2024

  8. [8]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 , 2024

    arXiv 2024

Show all 38 references
  1. [9]

    Retrieval-augmented generation for large language models: A survey

    Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Qianyu Guo, Meng Wang, and Haofen Wang. Retrieval-augmented generation for large language models: A survey. CoRR , abs/2312.10997, 2023

    2023 arXiv

  2. [10]

    A survey on automated fact-checking

    Zhijiang Guo, Michael Sejr Schlichtkrull, and Andreas Vlachos. A survey on automated fact-checking. Trans. Assoc. Comput. Linguistics , 10:178--206, 2022

    2022

  3. [11]

    Retrieval augmented language model pre-training

    Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. Retrieval augmented language model pre-training. In International conference on machine learning , pages 3929--3938. PMLR, 2020

    2020

  4. [12]

    Why so gullible? enhancing the robustness of retrieval-augmented models against counterfactual noise

    Giwon Hong, Jeonghwan Kim, Junmo Kang, Sung - Hyon Myaeng, and Joyce Jiyoung Whang. Why so gullible? enhancing the robustness of retrieval-augmented models against counterfactual noise. In Findings of the Association for Computational Linguistics: NAACL , pages 2474--2495, 2024

    2024

  5. [13]

    Austin Hounsel, Jordan Holland, Ben Kaiser, Kevin Borgolte, Nick Feamster, and Jonathan R. Mayer. Identifying disinformation websites using infrastructure features. In 10th USENIX Workshop on Free and Open Communications on the Internet, FOCI , 2020

    2020

  6. [14]

    Efficient memory management for large language model serving with paged attention

    Woosuk Kwon et al. Efficient memory management for large language model serving with paged attention. In Proc. of the ACM SIGOPS 29th Symposium on Operating Systems Principles , 2023

    2023

  7. [15]

    Ambigdocs: Reasoning across documents on different entities under the same name

    Yoonsang Lee, Xi Ye, and Eunsol Choi. Ambigdocs: Reasoning across documents on different entities under the same name. CoRR , abs/2404.12447, 2024

    2024 arXiv

  8. [16]

    u ttler, Mike Lewis, Wen - tau Yih, Tim Rockt \

    Patrick S. H. Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K \" u ttler, Mike Lewis, Wen - tau Yih, Tim Rockt \" a schel, Sebastian Riedel, and Douwe Kiela. Retrieval-augmented generation for knowledge-intensive NLP tasks. In ...

    2020

  9. [17]

    A survey of large language models attribution

    Dongfang Li, Zetian Sun, Xinshuo Hu, Zhenyu Liu, Ziyang Chen, Baotian Hu, Aiguo Wu, and Min Zhang. A survey of large language models attribution. CoRR , abs/2311.03731, 2023

    2023 arXiv

  10. [18]

    Ambigqa: Answering ambiguous open-domain questions

    Sewon Min, Julian Michael, Hannaneh Hajishirzi, and Luke Zettlemoyer. Ambigqa: Answering ambiguous open-domain questions. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP , pages 5783--5797, 2020

    2020

  11. [19]

    La plateforme, 2023

    Mistral.AI. La plateforme, 2023

    2023

  12. [20]

    Leveraging joint interactions for credibility analysis in news communities

    Subhabrata Mukherjee and Gerhard Weikum. Leveraging joint interactions for credibility analysis in news communities. In Proceedings of the 24th ACM International Conference on Information and Knowledge Management, CIKM , pages 353--362, 2015

    2015

  13. [21]

    Preslav Nakov, David P. A. Corney, Maram Hasanain, Firoj Alam, Tamer Elsayed, Alberto Barr \' o n - Cede \ n o, Paolo Papotti, Shaden Shaar, and Giovanni Da San Martino. Automated fact-checking for assisting human fact-checkers. In Proceedings of the Thirtieth International Jo...

    2021

  14. [22]

    Qacheck: A demonstration system for question-guided multi-hop fact-checking

    Liangming Pan, Xinyuan Lu, Min - Yen Kan, and Preslav Nakov. Qacheck: A demonstration system for question-guided multi-hop fact-checking. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP , pages 264--273, 2023

    2023

  15. [23]

    Fact-checking complex claims with program-guided reasoning

    Liangming Pan, Xiaobao Wu, Xinyuan Lu, Anh Tuan Luu, William Yang Wang, Min - Yen Kan, and Preslav Nakov. Fact-checking complex claims with program-guided reasoning. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pape...

    2023

  16. [24]

    Credibility assessment of textual claims on the web

    Kashyap Popat, Subhabrata Mukherjee, Jannik Str \" o tgen, and Gerhard Weikum. Credibility assessment of textual claims on the web. In Proceedings of the 25th ACM International Conference on Information and Knowledge Management, CIKM , pages 2173--2178, 2016

    2016

  17. [25]

    Where the truth lies: Explaining the credibility of emerging claims on the web and social media

    Kashyap Popat, Subhabrata Mukherjee, Jannik Str \" o tgen, and Gerhard Weikum. Where the truth lies: Explaining the credibility of emerging claims on the web and social media. In Proceedings of the 26th International Conference on World Wide Web Companion , pages 1003--1012, 2017

    2017

  18. [26]

    In-context retrieval-augmented language models

    Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton - Brown, and Yoav Shoham. In-context retrieval-augmented language models. Trans. Assoc. Comput. Linguistics , 11:1316--1331, 2023

    2023

  19. [27]

    Robertson and Hugo Zaragoza

    Stephen E. Robertson and Hugo Zaragoza. The probabilistic relevance framework: BM25 and beyond. Found. Trends Inf. Retr. , 3(4):333--389, 2009

    2009

  20. [28]

    Averitec: A dataset for real-world claim verification with evidence from the web

    Michael Schlichtkrull, Zhijiang Guo, and Andreas Vlachos. Averitec: A dataset for real-world claim verification with evidence from the web. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS , 2023

    2023

  21. [29]

    Generating media background checks for automated source critical reasoning

    Michael Schlichtkrull. Generating media background checks for automated source critical reasoning. CoRR , abs/2409.00781, 2024

    2024 arXiv

  22. [30]

    FEVER : a large-scale dataset for fact extraction and VER ification

    James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. FEVER : a large-scale dataset for fact extraction and VER ification. In Proceedings of the 2018 Conference of the North A merican Chapter of the Association for Computational Linguistics , pages 809-...

    2018

  23. [31]

    Wang, Yiming Ma, Renfei Gao, Beichen Guo, Zhuoran Li, Han Zhu, Wenqi Fan, Zexin Lu, and Ka Chung Ng

    Lionel Z. Wang, Yiming Ma, Renfei Gao, Beichen Guo, Zhuoran Li, Han Zhu, Wenqi Fan, Zexin Lu, and Ka Chung Ng. Megafake: A theory-driven dataset of fake news generated by large language models. CoRR , abs/2408.11871, 2024

    2024 arXiv

  24. [32]

    Searching for best practices in retrieval-augmented generation

    Xiaohua Wang, Zhenghua Wang, Xuan Gao, Feiran Zhang, Yixin Wu, Zhibo Xu, Tianyuan Shi, Zhengyuan Wang, Shizheng Li, Qi Qian, Ruicheng Yin, Changze Lv, Xiaoqing Zheng, and Xuanjing Huang. Searching for best practices in retrieval-augmented generation. In Proceedings of the 2024...

    2024

  25. [33]

    liar, liar pants on fire : A new benchmark dataset for fake news detection

    William Yang Wang. liar, liar pants on fire : A new benchmark dataset for fake news detection. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics , pages 422--426, 2017

    2017

  26. [34]

    Chi, Quoc V

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Inf...

    2022

  27. [35]

    Qwen2 technical report

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. Qwen2 technical report. arXiv preprint arXiv:2407.10671 , 2024

    2024 arXiv

  28. [36]

    Reinforcement retrieval leveraging fine-grained feedback for fact checking news claims with black-box LLM

    Xuan Zhang and Wei Gao. Reinforcement retrieval leveraging fine-grained feedback for fact checking news claims with black-box LLM . In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING) , pages 1...

    2024

  29. [37]

    Glass, and Preslav Nakov

    Yifan Zhang, Giovanni Da San Martino, Alberto Barr \' o n - Cede \ n o, Salvatore Romeo, Jisun An, Haewoon Kwak, Todor Staykovski, Israa Jaradat, Georgi Karadzhov, Ramy Baly, Kareem Darwish, James R. Glass, and Preslav Nakov. Tanbih: Get to know what you are reading. In Procee...

    2019

  30. [38]

    write newline

    " write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence '...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.