Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

COVE: COntext and VEracity prediction for out-of-context images

T0 review · 3 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read COVE debunks out-of-context images in a fixed order: first reconstruct the photo's true context, then judge the caption against it — beating veracity-only systems on real-world data and leaving an artifact readers reuse for new captions.

desk verdict COVE is a genuinely new context-first veracity pipeline with an honest evaluation, but its 'real-world' win rests on captions synthesized from the very context it predicts. read the letter →

arxiv 2502.01194 v1 pith:SSBLRSMQ submitted 2025-02-03 cs.CL

Jonathan Tonglet , Gabriel Thiem , Iryna Gurevych This is my paper · ORCID
classification cs.CL
keywords out-of-contextimagesmultimodalmisinformationautomatedfact-checkingcontextpredictionveracityvisuallargelanguagemodelsimagecontextualization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Out-of-context (OOC) images — real photographs paired with captions that misstate who, where, when, or why — are the most common form of multimodal misinformation. The paper argues that automated fact-checking should mirror human practice: first reconstruct the image's true context, then decide whether the caption matches it. It introduces COVE, a pipeline that predicts a structured context of seven items (source, date, location, motivation, people, things, event) from web captions, Wikipedia entities, and machine-generated image descriptions, then feeds that context to a veracity classifier. COVE reports gains over the previous best context-prediction system on every context item, often by more than five percentage points, and over established veracity-only systems on the real-world 5Pils-OOC benchmark by up to 4.5 Macro F1 points. A human study supports the reuse claim: readers given one predicted COVE context classified new captions about the same image at much higher accuracy than readers given only a veracity explanation.

What carries the argument

The load-bearing object is the predicted context itself: a structured representation of the image as seven answerable items. COVE builds it through five evidence-gathering steps — reverse-image search provides web captions and visual entities, with CLIP cosine-similarity thresholds deciding which captions match and even labeling some captions accurate or OOC outright; the OVEN index of six million Wikidata entities plus GENRE entity linking proposes candidate people and objects; LlavaNext generates captions of the whole image, of the people in it, and of cropped detected objects; Llama 3 answers one question per context item with the ranked evidence as input; and a knowledge-gap step generates questions, retrieves Wikipedia passages with ColBERT/WikiChat, and validates answers to fill missing dates and locations. Finally a veracity model — few-shot Llama 3 or fine-tuned DebertaV3 — compares the caption against the assembled context. The mechanism that carries the argument is the sequential dependency: the context is an intermediate representation that both drives the veracity decision and stays legible to human readers.

What would settle it

Take a sample of images from a real-world OOC set, have human fact-checkers write the seven context items from the image alone without seeing any caption, and re-run COVE's context and veracity evaluation against those human labels: if the advantage over veracity-only baselines and the near-perfect ground-truth bound persist, the context-first claim stands, while a large drop would show the reported gains largely reflect caption reconstruction. A second check would apply the human-study protocol to images whose true context is withheld, to see whether the reuse benefit survives on captions the model has not seen.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central claim is that veracity prediction for OOC images should be built on top of context prediction rather than run as a separate task. COVE operationalizes this by first generating a complete, structured context — seven items, three of which (people, things, event) are new — and then comparing the caption against that context. The experiments are designed to show the order pays off: COVE beats the 5Pils baseline on all context items, by 1.0 to 18.9 percentage points on NewsCLIPpings and 0.3 to 12.1 on 5Pils-OOC; it matches the best veracity systems on synthetic data and exceeds the best veracity-only system (SNIFFER's 51.9% F1) by 4.5 points on real-world data; and with ground-truth context, veracity becomes near-perfect, up to 95.3% F1, which the paper reads as evidence that an accurate context almost determines whether a caption is true. The human study adds the reuse claim: annotators given COVE's predicted context improved from 34.4% to 85.6% accuracy on newly written captions about the same image, a larger gain than annotators given SNIFFER's explanations.

Load-bearing premise

The load-bearing premise is that the machine-generated context labels used as ground truth faithfully represent the image — Llama 3 decomposes accurate captions into context items and GPT-4 writes the real-world accurate captions from context items — so if those labels are wrong, both the context scores and the near-perfect veracity upper bound measure how well the model reconstructs the caption rather than the image's true context.

Editorial extensions

If this is right

  • Removing the context step and feeding raw evidence instead drops OOC recall by 29.1 percentage points, so the predicted context is what carries the veracity signal.
  • Context quality bounds veracity quality: with ground-truth context, veracity reaches 94.4% F1 on NewsCLIPpings and 95.3% on 5Pils-OOC, and missing context items are the largest single error source, so improving context prediction is the direct path to better detection.
  • One context prediction serves many captions: since the context describes the image rather than any single caption, it can be reused to verify future captions about the same image, which the human study demonstrates.
  • Synthetic benchmarks overstate progress: every method drops sharply from NewsCLIPpings to the real-world 5Pils-OOC, with RED-DOT falling below random, while COVE's drop is smaller and it becomes the best method on real-world data.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The near-perfect ground-truth upper bound is partly circular: ground-truth context items for NewsCLIPpings (and three items in 5Pils-OOC) are produced by decomposing the accurate caption, so 'veracity from true context' partially measures how well a caption reconstructs itself; a human-written, image-only context label set would make the upper bound a fairer test.
  • The reuse result implies an amortized-cost argument the paper leaves implicit: one expensive, evidence-heavy context prediction can be run once per image and then pays for cheap verification of many captions, which fits how a single viral image gets re-captioned repeatedly.
  • The sequential pattern may generalize beyond images: 'reconstruct the reference situation, then compare the claim against it' is a plausible recipe for other fact-checking settings where a structured gold context can be defined, such as claims about events, documents, or medical content.
  • Evidence reliability is the likeliest next bottleneck: the error analysis shows irrelevant Wikipedia entities harming non-Western images, and the paper tests filtering only with a fixed list of trusted domains, so adaptive credibility filtering of web captions and entities is a testable improvement path.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. COVE is a two-step pipeline for out-of-context (OOC) image verification. First, it predicts a seven-item context (source, date, location, motivation, people, things, event) for a given image using diverse evidence: reverse-image-search web captions, visual entities, Wikipedia entity retrieval, automated captions, and a knowledge-gap completion step. Second, it feeds the predicted context together with the caption into a veracity classifier (a frozen Llama 3 in a few-shot setting or a fine-tuned DebertaV3). The paper reports that COVE improves over the 5Pils context-prediction baseline on all context items, is competitive with existing veracity models on the synthetic NewsCLIPpings dataset, and outperforms them on the real-world 5Pils-OOC test set by up to 4.5 Macro F1 points. A human study with six annotators is also presented, showing that COVE's predicted context is a reusable artifact for verifying new captions about the same image.

Significance. If the reported results were fully convincing, COVE would be a valuable contribution: it is the first system to explicitly and sequentially combine context prediction and veracity prediction, it introduces three new context items (people, things, event), it evaluates on a real-world OOC dataset, and its human study addresses an important practical property—the reusability of model outputs for downstream human fact-checking. The code and data are released, the pipeline is described in detail, and the ablation study (Table 3) clearly isolates the contribution of the context-prediction component. However, the main real-world claim rests on a test set whose accurate captions are generated by GPT-4 from the very context items that COVE predicts, which creates a systematic coupling that may explain part of the performance advantage. Because this issue is load-bearing for the headline claim, the current version of the paper cannot be accepted without substantial revision.

major comments (3)
  1. [Section 4.1 and Appendix E (Figure 8)] The construction of the 5Pils-OOC test set creates a systematic bias in favor of COVE. Accurate captions are not human-written; they are synthesized by GPT-4 from the ground-truth context items date, location, and motivation, which are exactly the context items that COVE is designed to predict. COVE's veracity module then compares the caption against its predicted context, so accurate captions are minimal paraphrases of the very information the system attempts to recover, whereas the OOC captions are genuine fact-checked misinformation. Baselines such as RED-DOT, AITR, and SNIFFER are trained on synthetic NewsCLIPpings and lack this structured context-comparison mechanism. Consequently, the claimed up-to-4.5-point Macro F1 advantage on real-world data (Table 2) does not convincingly establish real-world superiority. The authors should either re-construct 5Pils-OOC with authentic accurate captions (for example, from the fact-checking articles or the original news sources) or clearly restrict the claim to a test set with synthetic accurate captions.
  2. [Section 4.6] The 'ground truth context' upper-bound experiments are circular, as the authors themselves acknowledge: the ground-truth context items are obtained in Appendix D by decomposing the accurate captions with Llama 3. The near-perfect R_ACC values in Table 2 are therefore expected by construction, and the conclusion that 'predicting a comprehensive and accurate context ... ensures high performance' is an overstatement. This experiment should be presented only as a sanity check of the veracity module's ability to use the provided context, not as an estimate of the full pipeline's achievable ceiling.
  3. [Appendix D and Table 1] The ground-truth context labels for NewsCLIPpings and for the newly introduced items on 5Pils-OOC are produced by decomposing accurate captions with Llama 3, with no human validation. On 5Pils-OOC, the accurate captions are themselves generated by GPT-4 from only date, location, and motivation, so the derived labels for people, things, and event are very weak proxies for the actual image content. The context prediction scores in Table 1 are therefore difficult to interpret as measuring true context understanding. The paper should include a human-validated subset of these labels or compare with an alternative labeling method before the context-prediction gains can be fully trusted.
minor comments (5)
  1. [Table 2] The row label '5Pils baseline - LLama 3' is confusing; it should read '5Pils baseline + Llama 3' to indicate that Llama 3 performs veracity prediction with context items from the baseline.
  2. [Section 4.3 and Table 2] No confidence intervals or statistical significance tests are reported for the 5Pils-OOC veracity results. Given the test set size (1,248 instances), the authors should at least report bootstrap confidence intervals for the Macro F1 differences.
  3. [Table 1] The metric abbreviations in the header ('M', '∆', 'CO∆', 'F1') are not defined in the table caption; they are defined only in Section 4.2, which makes the table difficult to read in isolation.
  4. [Figure 8] The accurate-caption prompt says 'Combine the 3 in one sentence'; the '3' should be replaced with the actual context items (date, location, motivation) for clarity.
  5. [Section 4.8] The human study uses only six annotators, all described as students; the paper should explicitly note the small and possibly non-representative sample as a limitation of the study's generalizability.

Circularity Check

3 steps flagged · score 6.0 of 10

COVE's claimed real-world advantage is partly circular: 5Pils-OOC accurate captions are generated from the very context items COVE predicts, and the ground-truth-context 'validation' and context-prediction labels are derived from or define the captions by construction.

  1. self definitional [Section 4.6, veracity prediction paragraph]
    "Furthermore, we validate that the COVE results using the ground truth context lead to near-perfect RACC, both with DebertaV3 and Llama 3. This is expected because the ground truth context items are obtained by decomposing accurate captions."

    This 'validation' is not an independent check: for NewsCLIPpings the ground-truth context is decomposed from the accurate caption, and for 5Pils-OOC the accurate caption is generated from the ground-truth context items. Feeding that same context back into the veracity model and observing near-100% R_ACC confirms only the label-construction loop, not that predicted context drives veracity. The paper's own sentence concedes the result is expected by construction.

  2. fitted input called prediction [Section 4.1 and Appendix E (Figure 8 prompt)]
    "As 5Pils does not contain accurate captions, we generate them automatically using GPT4 (OpenAI, 2023) based on the ground truth context items. ... Write the facts only, avoid journalistic style and adjectives, avoid introducing new information."

    The 5Pils-OOC 'real-world' accurate captions are synthetic paraphrases of the exact context items COVE is designed to predict. COVE's veracity module compares the caption against its predicted context, so for the accurate class a correct context prediction almost directly yields the correct verdict. The claimed up-to-4.5-point Macro F1 advantage over baselines is therefore confounded by the fact that the accurate captions were manufactured from the context items COVE predicts, rather than being naturally occurring captions.

1 more flagged steps
  1. self definitional [Section 4.1 and Limitations (3)]
    "NewsCLIPpings does not contain ground truth context items. Hence, we create them by decomposing the accurate caption in a set of context items with Llama 3, as explained in Appendix D. ... The context items of NewsCLIPpings and parts of the items of 5Pils-OOC are weakly labeled by decomposing accurate captions."

    Context-prediction targets on NewsCLIPpings, and the people/things/event items on 5Pils-OOC, are Llama-3 decompositions of the accurate caption, while the evidence given to COVE includes web captions retrieved for the same image. The evaluation can therefore be solved by extracting or reconstructing caption content rather than by establishing the image's true context. The paper itself calls these labels 'weakly labeled,' so the context-prediction scores are not independent ground truth; they partially measure caption reconstruction.

full rationale

The central method (predict context first, then use it for veracity) is not itself a circular derivation, and the comparison against the 5Pils baseline on shared evidence is meaningful. However, the paper's evaluation contains two construction loops that make some claimed results expected by construction. First, context labels on NewsCLIPpings and for three items on 5Pils-OOC are generated by decomposing the accurate caption, so context-prediction accuracy is partly caption reconstruction rather than independent image contextualization. Second, the 'real-world' 5Pils-OOC accurate captions are generated by GPT-4 from the ground-truth context items, while COVE's veracity step checks captions against predicted context; this couples the accurate class to the exact items COVE predicts and biases the real-world comparison with baselines that lack this structured context. The paper honestly discloses the weak-label issue in Limitations and explicitly says the near-perfect ground-truth-context R_ACC 'is expected,' but disclosure does not remove the circularity in those evaluation steps. The predicted-context results, ablations, and human study provide independent content, so the paper is not entirely forced; the circularity is partial, giving a score of 6 rather than higher.

Assumptions & free parameters 10 free parameters · 5 assumptions · 0 invented entities

The pipeline rests on evidence-retrieval similarity thresholds tuned on a validation subsample, on LLM-generated annotations used as ground truth, and on the assumption that web and Wikipedia evidence retrieved for an image reflects its true context. These are transparently documented but not independently verified.

free parameters (10)
  • tmatch = 0.92
    Cosine similarity threshold between CLIP embeddings of instance image and web images; captions above it treated as accurate evidence (Section 3.1, Appendix G).
  • tnon_match = 0.7
    Similarity below this plus exact web caption match triggers OOC label (Section 3.1, Appendix G).
  • visual_entity_threshold = 0.1
    Minimum score for Google Vision visual entities to be included as evidence (Appendix G).
  • k (OVEN neighbors) = 5
    Number of nearest Wikipedia entities retrieved from OVEN index (Appendix G).
  • twiki_text = 0.23
    CLIP text-embedding similarity threshold for retaining a Wikipedia entity candidate (Appendix G).
  • twiki_image = 0.92 for PERSON, 0.7 otherwise
    Image-similarity threshold for retaining Wikipedia entity candidates (Appendix G).
  • object_confidence_threshold = 0.8
    Minimum confidence for Person objects to be cropped and captioned (Appendix G).
  • l (web captions provided) = 10
    Number of ranked web captions fed to context prediction prompt (Appendix G).
  • knowledge_gap_relevance_threshold = 20
    Minimum WikiChat relevance score for a Wikipedia passage to be used (Appendix G).
  • DebertaV3 fine-tuning hyperparameters = 5 epochs, batch 4, lr 5e-6, weight decay 0.01
    Tuned on 1500-instance validation subsample; affects veracity classifier performance (Section 4.5, Appendix G).
assumptions (5)
  • domain assumption Reverse-image-search web captions and matched web images are reliable evidence of the image's true context.
    Section 3.1 uses exact caption match and CLIP similarity to assign veracity labels and as primary context evidence; Limitations item 4 notes web captions may contain misinformation and are not filtered.
  • domain assumption CLIP cosine similarity between images and entity names or Wikipedia images indicates presence of that entity.
    Section 3.2 retains Wikipedia entities based on CLIP thresholds; Section 4.9 reports these retrievals are a main error source in non-Western contexts.
  • ad hoc to paper Llama 3 can decompose the accurate caption into reliable ground-truth context items.
    Appendix D uses this decomposition to create all NewsCLIPpings context labels and the people/things/event labels for 5Pils-OOC; this makes context evaluation partly a caption-reconstruction task.
  • ad hoc to paper GPT-4-generated accurate captions from fact-checked context items are valid substitutes for real accurate captions.
    Appendix E constructs 5Pils-OOC accurate pairs this way; the generated captions may be more or less natural than human captions and may affect veracity comparisons.
  • domain assumption Llama 3 correctly reasons from predicted context to veracity when provided few-shot demonstrations.
    Sections 3.6 and 4.5 rely on Llama 3 for final verdicts; Section 4.9 reports reasoning errors in a small fraction of cases.

how reviews work

0 comments
Cite this review

Pith. "Pith review of COVE: COntext and VEracity prediction for out-of-context images." pith.science (2026). https://pith.science/paper/SSBLRSMQ

@misc{pith2026250201194,
  author       = {Pith},
  title        = {Pith review of: COVE: COntext and VEracity prediction for out-of-context images},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SSBLRSMQ}},
  note         = {Machine review of arXiv:2502.01194}
}
read the original abstract

Images taken out of their context are the most prevalent form of multimodal misinformation. Debunking them requires (1) providing the true context of the image and (2) checking the veracity of the image's caption. However, existing automated fact-checking methods fail to tackle both objectives explicitly. In this work, we introduce COVE, a new method that predicts first the true COntext of the image and then uses it to predict the VEracity of the caption. COVE beats the SOTA context prediction model on all context items, often by more than five percentage points. It is competitive with the best veracity prediction models on synthetic data and outperforms them on real-world data, showing that it is beneficial to combine the two tasks sequentially. Finally, we conduct a human study that reveals that the predicted context is a reusable and interpretable artifact to verify new out-of-context captions for the same image. Our code and data are made available.

Figures

Figures reproduced from arXiv: 2502.01194 by the authors.

Figure 1
Figure 1. The two steps of COVE: (1) Generating the true context of the image. (2) Predicting the veracity of a caption by comparing it with the generated context. items are neither consistent nor inconsistent with the caption and provide more details than needed to verify the caption. In [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The architecture of COVE consists of six steps. The first three are performed in parallel and consist of retrieving evidence. Step 4 predicts the context items in a QA setting. Step 5 updates missing items based on the existing ones and Wikipedia knowledge. Step 6 predicts the veracity of the caption based on the predicted context. and captions, the veracity of certain instances is already predicted during evidence … view at source ↗
Figure 4
Figure 4. Knowledge gap completion. Questions are generated based on the predicted context and answered with Wikipedia passages. If the answers are relevant, the context is updated. item, its associated question, and relevant named entities are discussed in Appendix B . 3.5 Knowledge gap completion The date and location can often be missing after step 4. However, these two items can be derived from other context items and wor… view at source ↗
Figures from the paper (12 more)
Figure 5
Figure 5. Figure 5: Change in veracity prediction before and after [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Two examples of successful knowledge gap completion on 5Pils-OOC. [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Prompt template for caption decomposition with Llama 3. [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: Prompt to generate an accurate caption with [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: A composite image, where the sub-images have distinct true contexts. 5Pils for context and veracity prediction, we need to make some adjustments. First, the scope of 5Pils is broader than ours. It includes other types of misinformation than OOC images, namely, manipula…
Figure 10
Figure 10. Figure 10: Prompt template for context prediction with Llama 3. [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]
Figure 11
Figure 11. Figure 11: Prompt template for knowledge gap completion - question generation with Llama 3. [PITH_FULL_IMAGE:figures/full_fig_p017_11.png]
Figure 12
Figure 12. Figure 12: Prompt template for knowledge gap completion - question answering with Llama 3. [PITH_FULL_IMAGE:figures/full_fig_p017_12.png]
Figure 13
Figure 13. Figure 13: Prompt template for knowledge gap completion - validation with Llama 3. [PITH_FULL_IMAGE:figures/full_fig_p018_13.png]
Figure 14
Figure 14. Figure 14: Prompt template for veracity prediction with Llama 3. [PITH_FULL_IMAGE:figures/full_fig_p018_14.png]
Figure 15
Figure 15. Figure 15: Human study examples. From left to right, the image with the old caption, the COVE artifact, the [PITH_FULL_IMAGE:figures/full_fig_p020_15.png]
Figure 16
Figure 16. Figure 16: Error examples with Llama 3. Each row shows the image (left), the context prediction (center), the [PITH_FULL_IMAGE:figures/full_fig_p020_16.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. GETReason: Enhancing Image Context Extraction through Hierarchical Multi-Agent Reasoning

    cs.CV 2025-05 conditional novelty 6.0 of 10

    A multi-agent vision-language framework that extracts event, time, and location from public event images, evaluated with a new soft metric on VLM-augmented datasets.

Reference graph

Works this paper leans on

54 extracted references · 14 canonical work pages · cited by 1 Pith paper

  1. [1]

    Sahar Abdelnabi, Rakibul Hasan, and Mario Fritz. 2022. https://doi.org/10.1109/CVPR52688.2022.01452 Open-domain, content-based, multi-modal fact-checking of out-of-context images via online resources . In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022 , pages 14920--14929. IEEE

    work page arXiv 2022

  2. [2]

    Mubashara Akhtar, Michael Schlichtkrull, Zhijiang Guo, Oana Cocarascu, Elena Simperl, and Andreas Vlachos. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.361 Multimodal automated fact-checking: A survey . In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 5430--5448, Singapore. Association for Computational Linguistics

    doi:10.18653/v1/2023.findings-emnlp.361 2023

  3. [3]

    Amith Ananthram, Elias Stengel-Eskin, Carl Vondrick, Mohit Bansal, and Kathleen McKeown. 2024. https://arxiv.org/abs/2406.11665 See it from my perspective: Diagnosing the western cultural bias of large vision-language models in image understanding . ArXiv preprint, abs/2406.11665

    arXiv 2024

  4. [4]

    Shivangi Aneja, Chris Bregler, and Matthias Nie ner. 2023. https://doi.org/10.1609/AAAI.V37I12.26648 COSMOS: catching out-of-context image misuse using self-supervised learning . In Thirty-Seventh AAAI Conference on Artificial Intelligence, AAAI 2023, Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence, IAAI 2023, Thirteenth Symp...

    doi:10.1609/aaai.v37i12.26648 2023

  5. [5]

    Satanjeev Banerjee and Alon Lavie. 2005. https://aclanthology.org/W05-0909 METEOR : An automatic metric for MT evaluation with improved correlation with human judgments . In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization , pages 65--72, Ann Arbor, Michigan. Association for Comput...

    2005

  6. [6]

    Tobias Braun, Mark Rothermel, Marcus Rohrbach, and Anna Rohrbach. 2024. https://arxiv.org/abs/2412.10510 Defame: Dynamic evidence-based fact-checking with multimodal experts . ArXiv preprint, abs/2412.10510

    arXiv 2024

  7. [7]

    Nicola De Cao, Gautier Izacard, Sebastian Riedel, and Fabio Petroni. 2021. https://openreview.net/forum?id=5k8F6UU39V Autoregressive entity retrieval . In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021 . OpenReview.net

    2021

  8. [8]

    Gonzalez, Ion Stoica, and Eric P

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. https://lmsys.org/blog/2023-03-30-vicuna/ Vicuna: An open-source chatbot impressing gpt-4 with 90\

    2023

Show all 54 references
  1. [9]

    Zacharias Chrysidis, Stefanos-Iordanis Papadopoulos, Symeon Papadopoulos, and Panagiotis Petrantonakis. 2024. https://doi.org/10.1145/3643491.3660278 Credible, unreliable or leaked?: Evidence verification for enhanced automated fact-checking . In Proceedings of the 3rd ACM Int...

    2024

  2. [10]

    Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven C. H. Hoi. 2023. http://papers.nips.cc/paper\_files/paper/2023/hash/9a6a435e75419a836fe47ab6793623e6-Abstract-Conference.html Instructblip: Towards gener...

    2023

  3. [11]

    Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre-Emmanuel Mazar \'e , Maria Lomeli, Lucas Hosseini, and Herv \'e J \'e gou. 2024. https://arxiv.org/abs/2401.08281 The faiss library . ArXiv preprint, abs/2401.08281

    2024 arXiv

  4. [12]

    Nicholas Dufour, Arkanath Pathak, Pouya Samangouei, Nikki Hariri, Shashi Deshetti, Andrew Dudfield, Christopher Guess, Pablo Hern \'a ndez Escayola, Bobby Tran, Mevan Babakar, et al. 2024. https://arxiv.org/abs/2405.11697 Ammeba: A large-scale survey and dataset of media-based...

    2024 arXiv

  5. [13]

    Joseph L Fleiss. 1971. https://doi.org/10.1037/h0031619 Measuring nominal scale agreement among many raters. Psychological bulletin, 76(5):378--382

    1971 doi

  6. [14]

    Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2023. https://openreview.net/pdf?id=sE7-XhLxHA Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing . In The Eleventh International Conference on Learning Representations, ICLR 2...

    2023

  7. [16]

    Ziniu Hu, Ahmet Iscen, Chen Sun, Kai - Wei Chang, Yizhou Sun, David Ross, Cordelia Schmid, and Alireza Fathi. 2023 b . http://papers.nips.cc/paper\_files/paper/2023/hash/029df12a9363313c3e41047844ecad94-Abstract-Conference.html AVIS: autonomous visual information seeking with ...

    2023

  8. [17]

    Mohammed Abdul Khaliq, Paul Yu-Chun Chang, Mingyang Ma, Bernhard Pflugfelder, and Filip Mileti \'c . 2024. https://doi.org/10.18653/v1/2024.fever-1.29 RAGAR , your falsehood radar: RAG -augmented reasoning for political fact-checking using multimodal large language models . In...

    2024 doi

  9. [18]

    Sohail Ahmed Khan, Laurence Dierickx, Jan-Gunnar Furuly, Henrik Brattli Vold, Rano Tahseen, Carl-Gustav Linden, and Duc-Tien Dang-Nguyen. 2024. https://doi.org/10.1002/asi.24970 Debunking war information disorder: A case study in assessing the use of multimedia verification to...

    2024 doi

  10. [19]

    Opdahl, Fazle Rabbi, Sergej Stoppel, Christoph Trattner, and Duc-Tien Dang-Nguyen

    Sohail Ahmed Khan, Ghazaal Sheikhi, Andreas L. Opdahl, Fazle Rabbi, Sergej Stoppel, Christoph Trattner, and Duc-Tien Dang-Nguyen. 2023. https://doi.org/10.1109/ACCESS.2023.3236993 Visual user-generated content verification in journalism: An overview . IEEE Access, 11:6748--6769

    2023

  11. [20]

    Omar Khattab and Matei Zaharia. 2020. https://doi.org/10.1145/3397271.3401075 Colbert: Efficient and effective passage search via contextualized late interaction over BERT . In Proceedings of the 43rd International ACM SIGIR conference on research and development in Informatio...

    2020

  12. [21]

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. https://doi.org/10.1145/3600006.3613165 Efficient memory management for large language model serving with pagedattention . In Proceedings of the 29...

    2023

  13. [22]

    Kumud Lakara, Juil Sock, Christian Rupprecht, Philip Torr, John Collomosse, and Christian Schroeder de Witt. 2024. https://arxiv.org/abs/2410.20140 Mad-sherlock: Multi-agent debates for out-of-context misinformation detection . ArXiv preprint, abs/2410.20140

    2024

  14. [23]

    Chin-Yew Lin. 2004. https://aclanthology.org/W04-1013 ROUGE : A package for automatic evaluation of summaries . In Text Summarization Branches Out, pages 74--81, Barcelona, Spain. Association for Computational Linguistics

    2004

  15. [24]

    Fuxiao Liu, Yinghan Wang, Tianlu Wang, and Vicente Ordonez. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.542 Visual news: Benchmark and challenges in news image captioning . In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6...

    2021 doi

  16. [25]

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2024. https://openaccess.thecvf.com/content/CVPR2024/html/Liu_Improved_Baselines_with_Visual_Instruction_Tuning_CVPR_2024_paper.html Improved baselines with visual instruction tuning . In Proceedings of the IEEE/CVF Confer...

    2024

  17. [26]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. http://papers.nips.cc/paper\_files/paper/2023/hash/6dcf277ea32ce3288914faf369fe6de0-Abstract-Conference.html Visual instruction tuning . In Advances in Neural Information Processing Systems 36: Annual Conference on...

    2023

  18. [27]

    Grace Luo, Trevor Darrell, and Anna Rohrbach. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.545 N ews CLIP pings: A utomatic G eneration of O ut-of- C ontext M ultimodal M edia . In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pag...

    2021 doi

  19. [28]

    MetaAI. 2024. https://arxiv.org/abs/2407.21783 The llama 3 herd of models . ArXiv preprint, abs/2407.21783

    2024 arXiv

  20. [29]

    Annique Mossou and Ross Higgins. 2021. https://www.bellingcat.com/resources/2021/11/01/a-beginners-guide-to-social-media-verification/ A beginner's guide to social media verification . Accessed: 2023-09-15

    2021

  21. [30]

    Eric M\" u ller-Budack, Jonas Theiner, Sebastian Diering, Maximilian Idahl, and Ralph Ewerth. 2020. https://doi.org/10.1145/3372278.3390670 Multimodal analytics for real-world news using measures of cross-modal entity consistency . In Proceedings of the 2020 International Conf...

    2020

  22. [31]

    OpenAI. 2023. https://arxiv.org/abs/2303.08774 Gpt-4 technical report . Technical report, OpenAI

    2023 arXiv

  23. [32]

    Liangming Pan, Xinyuan Lu, Min-Yen Kan, and Preslav Nakov. 2023 a . https://doi.org/10.18653/v1/2023.emnlp-demo.23 QAC heck: A demonstration system for question-guided multi-hop fact-checking . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Proc...

    2023 doi

  24. [33]

    Liangming Pan, Xiaobao Wu, Xinyuan Lu, Anh Tuan Luu, William Yang Wang, Min-Yen Kan, and Preslav Nakov. 2023 b . https://doi.org/10.18653/v1/2023.acl-long.386 Fact-checking complex claims with program-guided reasoning . In Proceedings of the 61st Annual Meeting of the Associat...

    2023 doi

  25. [34]

    Stefanos-Iordanis Papadopoulos, Christos Koutlis, Symeon Papadopoulos, and Panagiotis C Petrantonakis. 2023. https://arxiv.org/abs/2311.09939 Red-dot: Multimodal fact-checking via relevant evidence detection . ArXiv preprint, abs/2311.09939

    2023 arXiv

  26. [35]

    Stefanos-Iordanis Papadopoulos, Christos Koutlis, Symeon Papadopoulos, and Panagiotis C Petrantonakis. 2024 a . https://arxiv.org/abs/2407.13488 Similarity over factuality: Are we making progress on multimodal out-of-context misinformation detection? ArXiv preprint, abs/2407.13488

    2024 arXiv

  27. [36]

    Stefanos-Iordanis Papadopoulos, Christos Koutlis, Symeon Papadopoulos, and Panagiotis C Petrantonakis. 2024 b . https://doi.org/10.1007/s13735-023-00312-6 Verite: a robust benchmark for multimodal misinformation detection accounting for unimodal bias . International Journal of...

    2024 doi

  28. [37]

    Kha-Luan Pham, Minh-Khoi Nguyen-Nhat, Anh-Huy Dinh, Quang-Tri Le, Manh-Thien Nguyen, Anh-Duy Tran, Minh-Triet Tran, and Duc-Tien Dang-Nguyen. 2024. https://doi.org/10.1007/978-3-031-53302-0_10 Ookpik- a collection of out-of-context image-caption pairs . In MultiMedia Modeling,...

    2024 doi

  29. [38]

    Peng Qi, Zehong Yan, Wynne Hsu, and Mong Li Lee. 2024. https://openaccess.thecvf.com/content/CVPR2024/html/Qi_SNIFFER_Multimodal_Large_Language_Model_for_Explainable_Out-of-Context_Misinformation_Detection_CVPR_2024_paper.html Sniffer: Multimodal large language model for expla...

    2024

  30. [39]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. http://proceedings.mlr.press/v139/radford21a.html Learning transferable visual models...

    2021

  31. [40]

    Ekraam Sabir, Wael AbdAlmageed, Yue Wu, and Prem Natarajan. 2018. https://doi.org/10.1145/3240508.3240707 Deep multimodal image-repurposing detection . In 2018 ACM Multimedia Conference on Multimedia Conference, MM 2018, Seoul, Republic of Korea, October 22-26, 2018 , pages 1337--1345

    2018

  32. [41]

    Michael Schlichtkrull, Zhijiang Guo, and Andreas Vlachos. 2023. http://papers.nips.cc/paper\_files/paper/2023/hash/cd86a30526cd1aff61d6f89f107634e4-Abstract-Datasets\_and\_Benchmarks.html Averitec: A dataset for real-world claim verification with evidence from the web . In Adv...

    2023

  33. [42]

    Michael Sejr Schlichtkrull. 2024. https://doi.org/10.18653/v1/2024.findings-emnlp.283 Generating media background checks for automated source critical reasoning . In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 4927--4947, Miami, Florida, USA. A...

    2024 doi

  34. [43]

    Sina Semnani, Violet Yao, Heidi Zhang, and Monica Lam. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.157 W iki C hat: Stopping the hallucination of large language model chatbots by few-shot grounding on W ikipedia . In Findings of the Association for Computational Ling...

    2023 doi

  35. [44]

    Craig Silverman. 2013. https://verificationhandbook.com/ Verification handbook . Accessed: 2023-09-15

    2013

  36. [45]

    Sahar Tahmasebi, Eric Müller-Budack, and Ralph Ewerth. 2025. https://arxiv.org/abs/2501.11403 Verifying cross-modal entity consistency in news using vision-language models . ArXiv preprint, abs/2501.11403

    2025 arXiv

  37. [46]

    Jonathan Tonglet, Marie-Francine Moens, and Iryna Gurevych. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.448 image, tell me your story! predicting the original meta-context of visual misinformation . In Proceedings of the 2024 Conference on Empirical Methods in Natural La...

    2024 doi

  38. [47]

    Shaydanay Urbani. 2020. https://firstdraftnews.org/long-form-article/verifying-online-information/ Verifying online information . Accessed: 2023-09-15

    2020

  39. [48]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mari...

    2020 doi

  40. [49]

    Xin Yuan, Jie Guo, Weidong Qiu, Zheng Huang, and Shujun Li. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.259 Support or refute: Analyzing the stance of evidence to detect out-of-context mis- and disinformation . In Proceedings of the 2023 Conference on Empirical Methods i...

    2023 doi

  41. [50]

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. 2023. https://doi.org/10.1109/ICCV51070.2023.01100 Sigmoid loss for language image pre-training . In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 11941--11952

    2023

  42. [51]

    Fanrui Zhang, Jiawei Liu, Qiang Zhang, Esther Sun, Jingyi Xie, and Zheng-Jun Zha. 2023 a . https://doi.org/10.1145/3581783.3612183 Ecenet: Explainable and context-enhanced network for muti-modal fact verification . In Proceedings of the 31st ACM International Conference on Mul...

    2023

  43. [52]

    Weinberger, and Yoav Artzi

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. https://openreview.net/forum?id=SkeHuCVFDr Bertscore: Evaluating text generation with BERT . In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April...

    2020

  44. [53]

    Yizhou Zhang, Loc Trinh, Defu Cao, Zijun Cui, and Yan Liu. 2023 b . https://arxiv.org/abs/2304.07633 Interpretable detection of out-of-context misinformation with neural-symbolic-enhanced large multimodal model . ArXiv preprint, abs/2304.07633

    2023 arXiv

  45. [54]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  46. [55]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.