Pith. sign in

REVIEW 3 major objections 6 minor 3 cited by

AIGI-Holmes: Towards Explainable and Generalizable AI-Generated Image Detection via Multimodal Large Language Models

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a multimodal large language model, trained with a three-stage pipeline and a collaborative decoding strategy, can detect AI-generated images with state-of-the-art accuracy while producing explanations humans can…

desk verdict Solid detection benchmark with real generalization gains, but the explainability claim is undercut by self-referential evaluation. read the letter →

arxiv 2507.02664 v2 pith:XFTJAAUU submitted 2025-07-03 cs.CV

Ziyin Zhou , Yunpeng Luo , Yuanchen Wu , Ke Sun , Jiayi Ji , Ke Yan , Shouhong Ding , Xiaoshuai Sun
show 2 more authors
Yunsheng Wu Rongrong Ji
This is my paper
classification cs.CV
keywords AI-generatedimagedetectionmultimodallargelanguagemodelsexplainabilitydirectpreferenceoptimizationgeneralizationforensicexplanationsdatasetconstruction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to solve two problems in AI-generated image detection: detectors are black boxes, and they generalize poorly to the newest generation models. Its answer is AIGI-Holmes, a multimodal large language model trained to say not just 'fake' but why, with reasons a person can check. The training recipe combines a new explanation-rich dataset, a three-stage pipeline (visual expert pre-training, supervised fine-tuning, direct preference optimization), and a collaborative decoding step that merges the MLLM's semantic read with a low-level forensic expert's signal. If the claims hold, detection stops being a verdict without evidence, and the detector transfers to unseen autoregressive and diffusion generators, where the paper reports accuracy gains of up to 15.2% over prior state of the art.

What carries the argument

The load-bearing mechanism is the Holmes Pipeline, a three-stage training scheme. Stage one ('Visual Expert Pre-training') fine-tunes a CLIP vision encoder with LoRA and fully fine-tunes a two-layer ResNet fed by NPR features, both trained with binary cross-entropy, so the model's visual front end learns domain-specific low-level features. Stage two performs supervised fine-tuning on Holmes-SFTSet, teaching the MLLM to generate explanatory answers. Stage three applies direct preference optimization on Holmes-DPOSet, whose contrastive pairs come from positive/negative prompts and from human and MLLM expert revisions rewritten by a language model. At inference, 'Collaborative Decoding' reweights the MLLM's real/fake logits with the visual experts' logits ($\text{logit}_{\text{new}} = \alpha\,\text{logit}_{\text{raw}} + \beta\,\text{logit}_{\text{clip}} + \gamma\,\text{logit}_{\text{npr}}$ with $\alpha=1,\beta=1,\gamma=0.2$), letting the semantic reasoner and the low-level artifact detector veto each other. The Multi-Expert Jury annotation system — four open MLLMs that annotate and cross-score each other's outputs, with expert defect filtering and a few thousand human corrections — supplies the training signal that makes the pipeline work.

What would settle it

Train AIGI-Holmes on the same 65K images but with explanations filtered by only one annotator model instead of the cross-model consensus; if accuracy on Protocol III stays at 99.2%, the Multi-Expert Jury is not load-bearing, and if it drops sharply, the consensus is what carries the result.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that the combination of the Holmes-Set dataset, the Holmes Pipeline, and collaborative decoding turns an off-the-shelf MLLM (LLaVA augmented with an NPR visual expert) into a detector that is both more accurate and more interpretable than specialized forensics models. In the hardest protocol, which tests unseen state-of-the-art generators, AIGI-Holmes reaches a mean accuracy of 99.2% versus 97.0% for AIDE and 96.2% for RINE, with the largest single-generator gain of 15.2% over NPR. The same model produces explanations that score higher than GPT-4o's on BLEU-1 (0.622 vs 0.433) and in human pairwise ELO ratings (11.420 vs 10.271), and its explanation quality stays stable under JPEG compression, blur, and resizing. The paper argues this shows the synergy of low-level perception and high-level semantic reasoning, aligned to human judgment through preference optimization.

Load-bearing premise

The whole training signal depends on the four open multimodal models that wrote the explanations agreeing with each other and with the handful of human corrections, so any artifact type all four of them systematically miss will be invisible to the final detector.

Editorial extensions

If this is right

  • A detector that produces explanations can be audited: a human can check each 'fake' verdict by looking for the cited anatomical, physical, or textual defect.
  • Generalization to unseen generators becomes practical: the method is evaluated on autoregressive models (V AR, Janus, Infinity, Show-o, LlamaGen) and diffusion models (FLUX, SD3.5, PixArt-XL) it was never trained on.
  • The three-stage pipeline transfers across LLM backbones (Llama-3-8B, Mistral-7B, Vicuna-7B) with minimal accuracy change, suggesting the recipe is backbone-agnostic.
  • Explanation quality is robust to common degradations (JPEG, blur, downsampling), which matters for real-world images that have been recompressed in transmission.
  • Preference optimization is what aligns explanations with human judgment: DPO raises human ELO by 0.75 and improves detection accuracy by 0.4–0.6%.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The data-annotation scheme implies a scalability claim the paper does not test: if four strong MLLMs can label 65K images by consensus with modest human correction, the same jury procedure could be applied to new generator families as they appear, keeping the detector current without retraining from scratch.
  • The collaborative decoding strategy is a generic fusion recipe: any semantic model and any low-level forensic feature extractor could be combined this way, so the result suggests a design pattern for other forensics tasks (deepfake video, image tampering) rather than a solution specific to AI-generated still images.
  • The reported 15.2% gain over NPR on Protocol III may partly reflect that NPR was trained on only 65K images in this comparison; a reader should watch for evaluations where all baselines are trained on their optimal dataset sizes, since the ablation table shows baseline gains flatten with more data.
  • Because the human-preference corrections were made by three authors plus two annotators, the ELO advantage over GPT-4o might be sensitive to who the human judges are; an independent judging pool would be the natural stress test.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes AIGI-Holmes, a multimodal-LLM-based AI-generated image (AIGI) detector that outputs both a real/fake decision and a free-text explanation. The contributions are: (1) Holmes-SFTSet, a 65K-image instruction dataset with structured MLLM-generated explanations, and Holmes-DPOSet, a preference dataset of 65K contrastive prompt pairs plus 4K human-/MLLM-revised samples; (2) a 'Multi-Expert Jury' annotation pipeline in which four open MLLMs annotate, cross-score each other's annotations (MLLM-as-a-judge consensus), and are filtered by expert defect detectors and partial human revision; (3) the Holmes Pipeline, i.e., visual-expert pre-training with CLIP and NPR heads, SFT, and DPO; and (4) collaborative decoding, which fuses the MLLM's real/fake token logits with the two visual-expert classifier logits at inference. The method is evaluated on three protocols: ProGAN-trained generalization to GAN/diffusion generators (P1), AntiFakePrompt's SD3-centric benchmark (P2), and a new P3 benchmark of ten unseen autoregressive and state-of-the-art diffusion generators, with reported mean accuracy of 99.2% versus 97.0% for the best retrained baseline on P3. Explanation quality is assessed with BLEU/ROUGE/METEOR/CIDEr against a 1K reviewed set, MLLM-as-Judge scores from the four annotating models, and a 100-image human ELO comparison.

Significance. The detection-side contribution is substantial and mostly well executed: all baselines are retrained on the same data, the three protocols span GAN, diffusion, and autoregressive generators, the P3 test set uses ten current generators (VAR, Janus, Janus-Pro, LlamaGen, Show-o, Infinity, PixArt-XL, FLUX, SD3.5), and the ablations cover LLM choice, training strategy, visual-expert components, and data scale. The 99.2% P3 mean accuracy and the 93.16% P1 accuracy would be valuable if confirmed. The dataset (65K SFT images plus 4K revised preference pairs) is an order of magnitude larger than FakeBench/LOKI, and the promised release plus the candid limitations in Appendix F speak well of the authors. However, the second headline claim, 'human-verifiable explanations', is not yet validated: the automatic and MLLM-judge metrics reuse the annotation pipeline itself, and the only independent signal is a 100-image human ELO study with unreported evaluator statistics. The explainability contribution is the paper's main novelty over black-box detectors, so the paper's significance is currently capped pending an independent verification study.

major comments (3)
  1. [§3.1, Table 3] The explanation-quality evaluation is self-referential in three places, and together they mean the paper's central claim of 'human-verifiable explanations' (abstract, §4.3) is not established. (a) The SFT annotations are produced by the four MLLM experts and retained only when they receive high consensus scores from those same four models (§3.1, Fig. 8); Table 3's 'MLLM-as-Judge' columns use exactly these four models, so the trained model is scored by the models whose output distribution it was trained to imitate. (b) The 1K Ground-Truth explanation set for BLEU/ROUGE/METEOR/CIDEr is 'reviewed by annotators ... and corrected uniformly by the Deepseek-V3 model' using the same prompt (Fig. 14) that produces the D2 DPO preference pairs; the automatic metrics therefore measure consistency with the pipeline's own output style, not whether the cited evidence exists in the image. (c) The human ELO study uses 100 images (10 per P3 forgery type), and §3.1 does not report the number of evaluators, their independence from the authors, or inter-annotator agreement; the 1.149-point ELO margin over GPT-4o is not statistically grounded. Since D1 is also fully MLLM-generated and only 4K preference samples receive human/MLLM revision, the 'human-aligned' signal in the training loop is small. I emphasize that this circularity affects the explanation claim, not the binary detection results, which rest on externally labeled images. I recommend an evaluation protocol independent of the annotation pipeline: external annotators verifying whether each cited defect is actually present (with reported IAA), and/or an independent explanation ground truth not produced via the Fig. 14 rewrite policy.
  2. [Eq. (6), Tables 2-7, §4.5] No variance or significance information is reported for any detection result; all tables appear to be single runs. For LoRA fine-tuning of a 7B LLM, run-to-run variation can be comparable to the headline margins (P3: 99.2 vs AIDE's 97.0, i.e., 2.2%; P1: 93.16 vs AIDE's 92.77, i.e., 0.4%). The collaborative-decoding weights in Eq. (6) are fixed constants (α=1, β=1, γ=0.2) with no sensitivity analysis; because the fused quantities are heterogeneous, namely LLM next-token logits and BCE-trained MLP logits on different scales, the reported CD contributions (4.0%/1.7% in §4.5) cannot be separated from the particular weight setting. Please report mean±std (or confidence intervals) over multiple seeds for the key comparisons, and a sweep over the three weights in Eq. (6).
  3. [Tables 4, 9, 10, §4.5] The ablation tables are internally inconsistent as presented. In §4.5 the VEP-S gain is stated as 3.5%/7.1% on P1/P3, which equals row 3 minus row 1 of Table 4 (86.8−83.3, 97.2−90.1), not the VEP-S-marked row minus the baseline (84.8−83.3 = 1.5); similarly, the DPO gain (0.6%/0.4%) and CD gain (4.0%/1.7%) correspond to row pairings that are not identifiable from the checkmark layout as rendered. Moreover, the same nominal configuration, i.e., the pipeline without a fused expert, appears as 86.8/97.2 in Table 4 and as 86.8/98.2 in Tables 9 and 10, so the P3 values differ by 1.0% across tables for what should be the same setting. Because the stage-wise contributions are the paper's methodological claims, every row of Tables 4, 9, and 10 must state its exact configuration (VEP-S on/off, DPO on/off, CD on/off, and which visual experts are fused).
minor comments (6)
  1. [§4.1 (Protocol-III)] The sentence 'we trained on the dataset containing various Diffusion methods proposed in Sec. 2' is erroneous (Sec. 2 is Related Work); please specify the exact Protocol-III training set (sources, image counts) and state explicitly that the ten P3 test generators were excluded from training, since the 'unseen generator' claim is the basis of the generalization conclusion.
  2. [Table 3 vs Algorithm 2] The reported ELO ratings (8.4-11.4) do not match Algorithm 2's initialization at 1000 with the update rule K=4; please state the normalization (for example, division by 100) so the reader can interpret the reported values.
  3. [§4.2] The phrase 'improvements of 15.2%, 3.0%, and 2.2% over the previous best methods NPR, RINE, and AIDE' calls NPR a previous-best method, but in Table 2 NPR's P3 mean accuracy (84.0) is below LaRE's (85.0); also, the claimed 1.8% margin on FLUX does not match Table 2 (98.7% vs RINE's 97.1%, i.e., 1.6%).
  4. [Table 7] The column header 'AntifakePrompt LoRA Orig. +LaMa +LaMa' is uninterpretable as rendered, and the reader cannot tell which column is the final AIGI-Holmes model; please label the columns clearly.
  5. [Appendix D.4] The claim that AIGI-Holmes achieves its performance 'even when trained on only a quarter of the dataset' is not supported by Table 11, which reports AIGI-Holmes results only at the 1× scale; a 0.25× row or a corrected sentence is needed.
  6. [Abstract, §1 contributions] The phrase 'the first explanation-rich datasets' overstates the novelty given that FakeBench [51] and LOKI [100] already provide explanation data (as acknowledged in §2.2) and that the contemporaneous LEGION [40] and FakeVLM [92] are discussed in Appendix B; the accurate claim would be 'first with contrastive preference pairs' or 'largest to date'.

Circularity Check

2 steps flagged · score 6.0 of 10

Explanation-evaluation loop: the same MLLMs that generate and filter the SFT annotations serve as judges, and the Deepseek-V3 correction procedure that builds DPO preference pairs also builds the test 'Ground Truth'; detection accuracy itself is externally benchmarked and not circular.

  1. fitted input called prediction [Section 3.1 'Automated Annotation' and Section 4.3, Table 3]
    "To ensure annotation quality, we adopt an MLLM-as-a-judge approach [8], where MLLM-Experts cross-evaluate each annotation. Only annotations with top consensus scores are retained in the dataset. ... For MLLM scoring, we refer to [8], using a prompt that considers relevance, accuracy, comprehensiveness, creativity, and granularity to compare and score the model’s responses."

    The four MLLM-Experts (Qwen2VL-72B, InternVL2-76B, InternVL2.5-78B, Pixtral-124B) that generate and consensus-filter the Holmes-SFTSet explanations are the same four models used as MLLM-as-Judge evaluators in Table 3. A model trained to imitate these explanations is therefore scored by judges drawn from the same distribution that produced its training targets. The 'MLLM-as-Judge' score measures agreement with the annotation pipeline's own output conventions, not independently verified human correctness, so the explanation-quality evaluation is partly a fitted input called prediction.

  2. fitted input called prediction [Section 3.1 'Preference Modification' and 'Comprehensive Evaluation'; Figure 14]
    "we use 1K test samples containing Ground Truth, which have been reviewed by annotators for deficiencies in explanations from a professional perspective and corrected uniformly by the Deepseek-V3 model. ... We then use Deepseek-V3 [55], an advanced open-source large language model, to modify the original responses of the SFT model based on the suggestions of human experts, resulting in new, more human-aligned correct explanations. ... We add the pairs of samples before and after modification to the DPO dataset D2."

    The same Deepseek-V3 modification procedure (Figure 14 prompt) creates both the DPO preference targets in D2 and the 1K 'Ground Truth' test references used for BLEU/ROUGE/CIDEr. The model is trained via DPO to prefer the Deepseek-V3-edited style, then evaluated for similarity to Deepseek-V3-edited references. The improvement in explanation metrics is therefore partly self-consistency with the training-target generator, not evidence that the explanations are human-verifiable. The small 100-image human ELO provides some external signal, but the human-revised text is itself passed through the same Deepseek-V3 correction loop, and no inter-annotator agreement is reported.

full rationale

The detection claims are not circular: Section 4.2 evaluates on P3 with unseen autoregressive and diffusion generators, labels come from external sources, and all baselines are retrained on the same training set for fair comparison. The circularity is confined to the explanation/human-alignment contribution, which is a headline claim of the paper. Two concrete reductions support this. First, the four MLLMs used to produce and consensus-filter the SFT annotations are the same models used as MLLM-as-Judge scorers in Table 3, so the training distribution is rewarded by the evaluator distribution. Second, the 1K test 'Ground Truth' explanations are corrected by Deepseek-V3 using the same modification procedure that builds the DPO preference pairs D2, so automatic explanation metrics measure agreement with the pipeline's own output style rather than independent correctness. The human ELO evaluation uses only 100 images and the paper reports no inter-annotator agreement, so it does not break the self-referential loop. Self-citations such as [17] for pairwise ELO are not load-bearing; the circularity is in the evaluation design, not in citation chains. Overall, the detection benchmark is externally grounded, but the explainability/human-alignment claim is partially circular, warranting a score of 6.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim relies on hand-set inference weights and annotation acceptance thresholds, plus the reliability of MLLM-generated explanations and a small human revision set as ground truth. No new physical or conceptual entities are introduced beyond the dataset and the learned model.

free parameters (3)
  • collaborative_decoding_weights = alpha=1, beta=1, gamma=0.2 (Eq. 6)
    Hand-set weights for fusing MLLM, CLIP, and NPR logits at inference; no sensitivity analysis or validation-set selection is reported, and the ablation credits this component with 1.7 to 4.0 percent accuracy.
  • MLLM_jury_consensus_threshold = not reported (delta_SFT in Algorithm 1)
    Annotations are kept only if the average MLLM-as-judge score exceeds this threshold; the threshold value is not given, yet it decides which 65K samples enter the SFT set.
  • human_revision_budget = 2K human-modified plus 2K MLLM-modified plus 1K test samples
    These hand-chosen data volumes determine how much independent human signal enters the DPO data and explanation evaluation; they are not varied or justified.
assumptions (4)
  • domain assumption Source dataset labels (CNNDetection, GenImage, DRCT) and generated-image labels are correct ground truth.
    The detection labels for 45K plus 20K training images are taken from external datasets without auditing; any label noise propagates into visual expert pretraining and SFT.
  • domain assumption Cross-validated MLLM annotations are accurate enough to serve as SFT ground truth for explanations.
    Section 3.1 keeps annotations based on MLLM-as-judge consensus with only a small human modification stage; if the jury shares blind spots, the explanation training data is systematically wrong.
  • domain assumption The three authors plus two guided annotators represent general human preference.
    Preference Modification says human experts consist of three authors and two annotators; the 'human-aligned' claim assumes this small group's judgments generalize.
  • ad hoc to paper Logit-space linear fusion of CLIP, NPR, and MLLM predictions is a valid combination rule.
    Equation 6 defines the fusion without derivation or comparison to other fusion rules; correctness of this rule is assumed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AIGI-Holmes: Towards Explainable and Generalizable AI-Generated Image Detection via Multimodal Large Language Models." pith.science (2026). https://pith.science/paper/XFTJAAUU

@misc{pith2026250702664,
  author       = {Pith},
  title        = {Pith review of: AIGI-Holmes: Towards Explainable and Generalizable AI-Generated Image Detection via Multimodal Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XFTJAAUU}},
  note         = {Machine review of arXiv:2507.02664}
}
read the original abstract

The rapid development of AI-generated content (AIGC) technology has led to the misuse of highly realistic AI-generated images (AIGI) in spreading misinformation, posing a threat to public information security. Although existing AIGI detection techniques are generally effective, they face two issues: 1) a lack of human-verifiable explanations, and 2) a lack of generalization in the latest generation technology. To address these issues, we introduce a large-scale and comprehensive dataset, Holmes-Set, which includes the Holmes-SFTSet, an instruction-tuning dataset with explanations on whether images are AI-generated, and the Holmes-DPOSet, a human-aligned preference dataset. Our work introduces an efficient data annotation method called the Multi-Expert Jury, enhancing data generation through structured MLLM explanations and quality control via cross-model evaluation, expert defect filtering, and human preference modification. In addition, we propose Holmes Pipeline, a meticulously designed three-stage training framework comprising visual expert pre-training, supervised fine-tuning, and direct preference optimization. Holmes Pipeline adapts multimodal large language models (MLLMs) for AIGI detection while generating human-verifiable and human-aligned explanations, ultimately yielding our model AIGI-Holmes. During the inference stage, we introduce a collaborative decoding strategy that integrates the model perception of the visual expert with the semantic reasoning of MLLMs, further enhancing the generalization capabilities. Extensive experiments on three benchmarks validate the effectiveness of our AIGI-Holmes.

Figures

Figures reproduced from arXiv: 2507.02664 by the authors.

Figure 1
Figure 1. (a): Comparison of AIGI-Holmes with existing methods, (b): A qualitative example to illustrate the effect of AIGI-Holmes, (c): [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Details of the Holmes-Set Construction. The figure illustrates our data pipeline, consisting of four key components: Data Source (including Data Collection and Image Generation), Automated Annotation, Preference Modification (based on human expert feedback), and Comprehensive Evaluation (to assess model generalizability and interpretability). age [108], and DRCT [7] for image description and forgery explanation. To … view at source ↗
Figure 3
Figure 3. Overview of AIGI-Holmes. We enhance LLaVA [ [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (27 more)
Figure 4
Figure 4. Figure 4: Qualitative results of AIGI-Holmes on AI-Generated images. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Robustness of the explanation on JPEG Compression [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: General Positive Prompt for annotating real images and General Negative Prompt for annotating AI-generated images. [PITH_FULL_IMAGE:figures/full_fig_p019_6.png]
Figure 7
Figure 7. Figure 7: General Positive Prompt for annotating AI-generated images and General Negative Prompt for annotating real images. [PITH_FULL_IMAGE:figures/full_fig_p020_7.png]
Figure 8
Figure 8. Figure 8: Prompt for cross-model validation and evaluation using state-of-the-art multimodal large language models. [PITH_FULL_IMAGE:figures/full_fig_p021_8.png]
Figure 9
Figure 9. Figure 9: Specialist Prompt for AI-generated images containing face defects. [PITH_FULL_IMAGE:figures/full_fig_p022_9.png]
Figure 10
Figure 10. Figure 10: Specialist Prompt for AI-generated images containing body defects. [PITH_FULL_IMAGE:figures/full_fig_p022_10.png]
Figure 11
Figure 11. Figure 11: Specialist Prompt for AI-generated images containing defects in text&logos. [PITH_FULL_IMAGE:figures/full_fig_p022_11.png]
Figure 12
Figure 12. Figure 12: Specialist Prompt for AI-generated images containing defects in projective geometry. [PITH_FULL_IMAGE:figures/full_fig_p023_12.png]
Figure 13
Figure 13. Figure 13: Specialist prompt for AI-generated images containing defects in commonsense and physical laws. [PITH_FULL_IMAGE:figures/full_fig_p023_13.png]
Figure 14
Figure 14. Figure 14: The original response of the model and the revision suggestions are input into the query prompt of DeepseekV3 [ [PITH_FULL_IMAGE:figures/full_fig_p023_14.png]
Figure 15
Figure 15. Figure 15: Qualitative Results of the Test Set for P3 [PITH_FULL_IMAGE:figures/full_fig_p024_15.png]
Figure 16
Figure 16. Figure 16: The score distribution of Multi-Expert Jury ratings, the correlation heatmap of the ratings, and the proportion of annotations by [PITH_FULL_IMAGE:figures/full_fig_p024_16.png]
Figure 17
Figure 17. Figure 17: The annotation interface for obtaining preference samples with modifications suggested by human experts. [PITH_FULL_IMAGE:figures/full_fig_p025_17.png]
Figure 18
Figure 18. Figure 18: The interface for evaluating sample selection of human preferences in Arenas. [PITH_FULL_IMAGE:figures/full_fig_p026_18.png]
Figure 19
Figure 19. Figure 19: A heatmap of the winning counts in pairwise scoring among state-of-the-art multimodal language models (MLLMs) [PITH_FULL_IMAGE:figures/full_fig_p027_19.png]
Figure 20
Figure 20. Figure 20: The schematic flowchart of the DeepseekV3 [ [PITH_FULL_IMAGE:figures/full_fig_p028_20.png]
Figure 21
Figure 21. Figure 21: Annotated sample and its score in Holmes-SFTSet for Qwen2VL-72B. [PITH_FULL_IMAGE:figures/full_fig_p029_21.png]
Figure 22
Figure 22. Figure 22: Annotated sample and its score in Holmes-SFTSet for InternVL2-76B. [PITH_FULL_IMAGE:figures/full_fig_p030_22.png]
Figure 23
Figure 23. Figure 23: Annotated sample and its score in Holmes-SFTSet for InternVL2.5-78B. [PITH_FULL_IMAGE:figures/full_fig_p031_23.png]
Figure 24
Figure 24. Figure 24: Annotated sample and its score in Holmes-SFTSet for Pixtral-124B. [PITH_FULL_IMAGE:figures/full_fig_p032_24.png]
Figure 25
Figure 25. Figure 25: The presentation of preference sample pairs in [PITH_FULL_IMAGE:figures/full_fig_p033_25.png]
Figure 26
Figure 26. Figure 26: The presentation of preference sample pairs in [PITH_FULL_IMAGE:figures/full_fig_p034_26.png]
Figure 27
Figure 27. Figure 27: Comparison of qualitative results between Qwen2VL-72B and our method. [PITH_FULL_IMAGE:figures/full_fig_p035_27.png]
Figure 28
Figure 28. Figure 28: Comparison of qualitative results between Intern2VL-76B and our method. [PITH_FULL_IMAGE:figures/full_fig_p036_28.png]
Figure 29
Figure 29. Figure 29: Comparison of qualitative results between InternVL2.5-78B and our method. [PITH_FULL_IMAGE:figures/full_fig_p037_29.png]
Figure 30
Figure 30. Figure 30: Comparison of qualitative results between Pixtral-124B and our method. [PITH_FULL_IMAGE:figures/full_fig_p038_30.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. XPlainVerse: A Million-Scale Benchmark for Explainable Deepfake Detection

    cs.CV 2026-07 conditional novelty 6.5 of 10

    A million-scale deepfake benchmark with Edit-Check filtering, dual expert/lay explanations, and EntityScore/EvidenceScore shows fine-tuned detectors collapse under generator shift while surface fluency remains.

  2. AgentFoX: LLM Agent-Guided Fusion with eXplainability for AI-Generated Image Detection

    cs.CV 2026-03 conditional novelty 6.0 of 10

    An LLM agent guided by Expert and Clustering Profiles fuses heterogeneous AIGI detectors, resolves conflicts, and outputs explainable forensic reports that beat single experts and standard ensembles on high-conflict a...

  3. HunyuanImage 3.0 Technical Report

    cs.CV 2025-09 conditional novelty 6.0 of 10

    HunyuanImage 3.0 is an open 80B-parameter multimodal autoregressive image generator that reportedly matches leading closed models on in-house benchmarks.

Reference graph

Works this paper leans on

109 extracted references · 32 canonical work pages · cited by 3 Pith papers

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774,

    arXiv

  2. [2]

    Llama 3 model card

    AI@Meta. Llama 3 model card. 2024. 3

    2024

  3. [3]

    Meteor: An automatic metric for mt evaluation with improved correlation with human judgments

    Satanjeev Banerjee and Alon Lavie. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pages 65–72, 2005. 5

    2005

  4. [4]

    Black-Forest-Labs. Flux. https://github.com/ black-forest-labs/flux, 2024. 1, 7, 2

    2024

  5. [5]

    Antifakeprompt: Prompt-tuned vision-language models are fake image detectors

    You-Ming Chang, Chen Yeh, Wei-Chen Chiu, and Ning Yu. Antifakeprompt: Prompt-tuned vision-language models are fake image detectors. arXiv preprint arXiv:2310.17419,

    arXiv

  6. [6]

    Drct: Diffusion reconstruction contrastive training towards universal detection of diffusion generated images

    Baoying Chen, Jishen Zeng, Jianquan Yang, and Rui Yang. Drct: Diffusion reconstruction contrastive training towards universal detection of diffusion generated images. In Forty- first International Conference on Machine Learning. 3

  7. [7]

    DRCT: Diffusion reconstruction contrastive training towards universal detection of diffusion generated images

    Baoying Chen, Jishen Zeng, Jianquan Yang, and Rui Yang. DRCT: Diffusion reconstruction contrastive training towards universal detection of diffusion generated images. In Pro- ceedings of the 41st International Conference on Machine Learning, pages 7621–7639. PMLR, 2024. 2, 3, 4

    2024

  8. [8]

    Mllm-as-a-judge: Assessing multimodal llm-as-a-judge with vision-language benchmark

    Dongping Chen, Ruoxi Chen, Shilin Zhang, Yaochen Wang, Yinuo Liu, Huichi Zhou, Qihui Zhang, Yao Wan, Pan Zhou, and Lichao Sun. Mllm-as-a-judge: Assessing multimodal llm-as-a-judge with vision-language benchmark. In Forty- first International Conference on Machine Learning, 2024. 4, 5

    2024

Show all 109 references
  1. [9]

    Pixart- σ : Weak-to-strong training of diffusion transformer for 4k text-to-image generation, 2024

    Junsong Chen, Chongjian Ge, Enze Xie, Yue Wu, Lewei Yao, Xiaozhe Ren, Zhongdao Wang, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart- σ : Weak-to-strong training of diffusion transformer for 4k text-to-image generation, 2024. 7, 2

    2024

  2. [10]

    Textdiffuser: Diffusion models as text painters

    Jingye Chen, Yupan Huang, Tengchao Lv, Lei Cui, Qifeng Chen, and Furu Wei. Textdiffuser: Diffusion models as text painters. Advances in Neural Information Processing Systems, 36, 2024. 4

    2024

  3. [11]

    Diffusionfake: En- hancing generalization in deepfake detection via guided sta- ble diffusion

    Shen Chen, Taiping Yao, Hong Liu, Xiaoshuai Sun, Shouhong Ding, Rongrong Ji, et al. Diffusionfake: En- hancing generalization in deepfake detection via guided sta- ble diffusion. Advances in Neural Information Processing Systems, 37:101474–101497, 2024. 1

    2024

  4. [12]

    Janus- pro: Unified multimodal understanding and generation with data and model scaling

    Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. Janus- pro: Unified multimodal understanding and generation with data and model scaling. arXiv preprint arXiv:2501.17811,

    arXiv

  5. [13]

    arXiv preprint arXiv:2410.06126, 2024

    Yize Chen, Zhiyuan Yan, Siwei Lyu, and Baoyuan Wu.X2- dfd: A framework for e Xplainable and eXtendable deep- fake detection. arXiv preprint arXiv:2410.06126, 2024. 1, 5

    2024 arXiv

  6. [14]

    Diffusionface: Towards a comprehensive dataset for diffusion-based face forgery analysis

    Zhongxi Chen, Ke Sun, Ziyin Zhou, Xianming Lin, Xi- aoshuai Sun, Liujuan Cao, and Rongrong Ji. Diffusionface: Towards a comprehensive dataset for diffusion-based face forgery analysis. arXiv preprint arXiv:2403.18471, 2024. 1

    2024 arXiv

  7. [15]

    Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhang- wei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271, 2024. 4

    2024 arXiv

  8. [16]

    How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites

    Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhang- wei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, et al. How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites. arXiv preprint arXiv:2404.16821, 2024. 4

    2024 arXiv

  9. [17]

    Chatbot arena: An open platform for evaluating llms by human preference

    Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anasta- sios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Banghua Zhu, Hao Zhang, Michael Jordan, Joseph E Gonzalez, et al. Chatbot arena: An open platform for evaluating llms by human preference. In Forty-first International Conferen...

    2024

  10. [18]

    Raising the bar of ai-generated image detection with clip

    Davide Cozzolino, Giovanni Poggi, Riccardo Corvi, Matthias Nießner, and Luisa Verdoliva. Raising the bar of ai-generated image detection with clip. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4356–4366, 2024. 7

    2024

  11. [19]

    Instructblip: Towards general- purpose vision-language models with instruction tuning

    Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. Instructblip: Towards general- purpose vision-language models with instruction tuning. arXiv preprint arXiv:2305.06500, 2023. 2

    2023 arXiv

  12. [20]

    Instructblip: Towards general- purpose vision-language models with instruction tuning,

    Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. Instructblip: Towards general- purpose vision-language models with instruction tuning,

  13. [21]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural informa- tion processing systems, 34:8780–8794, 2021. 1

    2021

  14. [22]

    Scaling rectified flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first International Conference on Machin...

  15. [23]

    Humanrefiner: Benchmarking abnormal human generation and refining with coarse-to-fine pose-reversible guidance

    Guian Fang, Wenbiao Yan, Yuanfan Guo, Jianhua Han, Zu- tao Jiang, Hang Xu, Shengcai Liao, and Xiaodan Liang. Humanrefiner: Benchmarking abnormal human generation and refining with coarse-to-fine pose-reversible guidance. In European Conference on Computer Vision, pages 201–217...

    2024

  16. [24]

    Leveraging frequency analysis for deep fake image recognition

    Joel Frank, Thorsten Eisenhofer, Lea Sch¨onherr, Asja Fis- cher, Dorothea Kolossa, and Thorsten Holz. Leveraging frequency analysis for deep fake image recognition. In Inter- national conference on machine learning, pages 3247–3258. PMLR, 2020. 3, 2

    2020

  17. [25]

    Rich models for steganal- ysis of digital images

    Jessica Fridrich and Jan Kodovsky. Rich models for steganal- ysis of digital images. IEEE Transactions on information Forensics and Security, 7(3):868–882, 2012. 3

    2012

  18. [26]

    Commonsense-t2i challenge: Can text-to- image generation models understand commonsense? arXiv preprint arXiv:2406.07546, 2024

    Xingyu Fu, Muyu He, Yujie Lu, William Yang Wang, and Dan Roth. Commonsense-t2i challenge: Can text-to- image generation models understand commonsense? arXiv preprint arXiv:2406.07546, 2024. 4, 1, 2

    2024 arXiv

  19. [27]

    Generative adversarial nets

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in Neural Information Processing Systems, 27, 2014. 1

    2014

  20. [28]

    Rethinking vision-language model in face forensics: Multi-modal interpretable forged face detector

    Xiao Guo, Xiufeng Song, Yue Zhang, Xiaohong Liu, and Xiaoming Liu. Rethinking vision-language model in face forensics: Multi-modal interpretable forged face detector. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 105–116, 2025. 1

    2025

  21. [29]

    Infinity: Scaling bitwise autoregressive modeling for high-resolution image synthesis, 2024

    Jian Han, Jinlai Liu, Yi Jiang, Bin Yan, Yuqi Zhang, Zehuan Yuan, Bingyue Peng, and Xiaobing Liu. Infinity: Scaling bitwise autoregressive modeling for high-resolution image synthesis, 2024. 7, 2

    2024

  22. [30]

    Wildfake: A large-scale chal- lenging dataset for ai-generated images detection

    Yan Hong and Jianfu Zhang. Wildfake: A large-scale chal- lenging dataset for ai-generated images detection. arXiv preprint arXiv:2402.11843, 2024. 3

    2024 arXiv

  23. [31]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021. 6

    2021 arXiv

  24. [32]

    Sida: Social media image deepfake detection, localization and explanation with large multimodal model

    Zhenglin Huang, Jinwei Hu, Xiangtai Li, Yiwei He, Xingyu Zhao, Bei Peng, Baoyuan Wu, Xiaowei Huang, and Guan- gliang Cheng. Sida: Social media image deepfake detection, localization and explanation with large multimodal model. arXiv preprint arXiv:2412.04292, 2024. 1

    2024 arXiv

  25. [33]

    Ffaa: Multimodal large language model based explainable open-world face forgery analysis assistant

    Zhengchao Huang, Bin Xia, Zicheng Lin, Zhun Mou, and Wenming Yang. Ffaa: Multimodal large language model based explainable open-world face forgery analysis assistant. arXiv preprint arXiv:2408.10072, 2024. 1, 3, 4, 5

    2024 arXiv

  26. [34]

    Gpt-4o system card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perel- man, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Weli- hinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024. 3

    2024 arXiv

  27. [35]

    Can chatgpt detect deepfakes? a study of using multimodal large language models for media forensics

    Shan Jia, Reilin Lyu, Kangran Zhao, Yize Chen, Zhiyuan Yan, Yan Ju, Chuanbo Hu, Xin Li, Baoyuan Wu, and Si- wei Lyu. Can chatgpt detect deepfakes? a study of using multimodal large language models for media forensics. In Proceedings of the IEEE/CVF Conference on Computer Vi- s...

    2024

  28. [36]

    Mistral 7b

    Albert Qiaochu Jiang, Alexandre Sablayrolles, Arthur Men- sch, Chris Bamford, Devendra Singh Chaplot, Diego de Las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, L’elio Renard Lavaud, Marie- Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril...

    arXiv

  29. [37]

    Fusing global and local features for gen- eralized ai-synthesized image detection

    Yan Ju, Shan Jia, Lipeng Ke, Hongfei Xue, Koki Nagano, and Siwei Lyu. Fusing global and local features for gen- eralized ai-synthesized image detection. In 2022 IEEE In- ternational Conference on Image Processing (ICIP), pages 3465–3469. IEEE, 2022. 2

    2022

  30. [38]

    How to distinguish ai-generated images from authentic photographs

    Negar Kamali, Karyn Nakamura, Angelos Chatzimparmpas, Jessica Hullman, and Matthew Groh. How to distinguish ai-generated images from authentic photographs. arXiv preprint arXiv:2406.08651, 2024. 4, 5

    2024 arXiv

  31. [39]

    Characterizing photorealism and artifacts in diffusion model- generated images

    Negar Kamali, Karyn Nakamura, Aakriti Kumar, Ange- los Chatzimparmpas, Jessica Hullman, and Matthew Groh. Characterizing photorealism and artifacts in diffusion model- generated images. arXiv preprint arXiv:2502.11989, 2025. 4, 5

    2025 arXiv

  32. [40]

    Legion: Learning to ground and explain for synthetic image detection

    Hengrui Kang, Siwei Wen, Zichen Wen, Junyan Ye, Wei- jia Li, Peilin Feng, Baichuan Zhou, Bin Wang, Dahua Lin, Linfeng Zhang, et al. Legion: Learning to ground and explain for synthetic image detection. arXiv preprint arXiv:2503.15264, 2025. 1

    2025 arXiv

  33. [41]

    Progressive growing of gans for improved quality, stability, and variation

    Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for improved quality, stability, and variation. arXiv preprint arXiv:1710.10196, 2017. 3

    2017 arXiv

  34. [42]

    A style-based generator architecture for generative adversarial networks

    Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4401–4410, 2019. 1

    2019

  35. [43]

    Bi-lora: A vision-language approach for synthetic image detection

    Mamadou Keita, Wassim Hamidouche, Hessen Bougueffa Eutamene, Abdenour Hadid, and Abdelmalik Taleb-Ahmed. Bi-lora: A vision-language approach for synthetic image detection. arXiv preprint arXiv:2404.01959, 2024. 1, 3

    2024 arXiv

  36. [44]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 4015–4026, 2023. 3

    2023

  37. [45]

    Leveraging rep- resentations from intermediate encoder-blocks for synthetic image detection

    Christos Koutlis and Symeon Papadopoulos. Leveraging rep- resentations from intermediate encoder-blocks for synthetic image detection. In European Conference on Computer Vision, pages 394–411. Springer, 2024. 1, 7, 2, 4

    2024

  38. [46]

    Truthlens: Explainable deepfake detection for face manipulated and fully synthetic data

    Rohit Kundu, Athula Balachandran, and Amit K Roy- Chowdhury. Truthlens: Explainable deepfake detection for face manipulated and fully synthetic data. arXiv preprint arXiv:2503.15867, 2025. 1

    2025 arXiv

  39. [47]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Pro- ceedings of the ACM SIGOPS 29th Symposium on Operating Syste...

    2023

  40. [48]

    Quality-agnostic deepfake detection with intra-model collaborative learning

    Binh M Le and Simon S Woo. Quality-agnostic deepfake detection with intra-model collaborative learning. In Pro- ceedings of the IEEE/CVF International Conference on Com- puter Vision, pages 22378–22389, 2023. 2

    2023

  41. [49]

    Forgerygpt: Multimodal large language model for explainable image forgery detection and localization

    Jiawei Li, Fanrui Zhang, Jiaying Zhu, Esther Sun, Qiang Zhang, and Zheng-Jun Zha. Forgerygpt: Multimodal large language model for explainable image forgery detection and localization. arXiv preprint arXiv:2410.10238, 2024. 1

    2024 arXiv

  42. [50]

    Improving synthetic image detection to- wards generalization: An image transformation perspective

    Ouxiang Li, Jiayin Cai, Yanbin Hao, Xiaolong Jiang, Yao Hu, and Fuli Feng. Improving synthetic image detection to- wards generalization: An image transformation perspective. arXiv preprint arXiv:2408.06741, 2024. 2

    2024 arXiv

  43. [51]

    Fakebench: Uncover the achilles’ heels of fake images with large multimodal models

    Yixuan Li, Xuelin Liu, Xiaoyang Wang, Shiqi Wang, and Weisi Lin. Fakebench: Uncover the achilles’ heels of fake images with large multimodal models. arXiv preprint arXiv:2404.13306, 2024. 2, 3, 4

    2024 arXiv

  44. [52]

    A large-scale interpretable multi- modality benchmark for facial image forgery localization

    Jingchun Lian, Lingyu Liu, Yaxiong Wang, Yujiao Wu, Li Zhu, and Zhedong Zheng. A large-scale interpretable multi- modality benchmark for facial image forgery localization. arXiv preprint arXiv:2412.19685, 2024. 1

    2024 arXiv

  45. [53]

    Rouge: A package for automatic evaluation of summaries

    Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74–81, 2004. 5

    2004

  46. [54]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceeding...

    2014

  47. [55]

    Deepseek-v3 technical report

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024. 5, 1, 2, 10, 15

    2024 arXiv

  48. [56]

    Detecting generated images by real images

    Bo Liu, Fan Yang, Xiuli Bi, Bin Xiao, Weisheng Li, and Xinbo Gao. Detecting generated images by real images. In European Conference on Computer Vision, pages 95–110. Springer, 2022. 2

    2022

  49. [57]

    Forgery-aware adaptive transformer for generalizable synthetic image detection

    Huan Liu, Zichang Tan, Chuangchuang Tan, Yunchao Wei, Yao Zhao, and Jingdong Wang. Forgery-aware adaptive transformer for generalizable synthetic image detection. arXiv preprint arXiv:2312.16649, 2023. 3

    2023 arXiv

  50. [58]

    Llava-next: Im- proved reasoning, ocr, and world knowledge, 2024

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Im- proved reasoning, ocr, and world knowledge, 2024. 7

    2024

  51. [59]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024. 3, 5, 1

    2024

  52. [60]

    Forgery-aware adaptive transformer for generalizable synthetic image detection

    Huan Liu, Zichang Tan, Chuangchuang Tan, Yunchao Wei, Jingdong Wang, and Yao Zhao. Forgery-aware adaptive transformer for generalizable synthetic image detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 2

    2024

  53. [61]

    Lareˆ 2: Latent reconstruction error based method for diffusion-generated image detection

    Yunpeng Luo, Junlong Du, Ke Yan, and Shouhong Ding. Lareˆ 2: Latent reconstruction error based method for diffusion-generated image detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17006–17015, 2024. 1, 7, 2

    2024

  54. [62]

    Phybench: A physical common- sense benchmark for evaluating text-to-image models

    Fanqing Meng, Wenqi Shao, Lixin Luo, Yahong Wang, Yi- ran Chen, Quanfeng Lu, Yue Yang, Tianshuo Yang, Kaipeng Zhang, Yu Qiao, et al. Phybench: A physical common- sense benchmark for evaluating text-to-image models. arXiv preprint arXiv:2406.11802, 2024. 4, 1, 2

    2024 arXiv

  55. [63]

    Pixtral large

    Mistral-AI. Pixtral large. https://mistral.ai/ news/pixtral-large/, 2024. 4

    2024

  56. [64]

    Towards uni- versal fake image detectors that generalize across generative models

    Utkarsh Ojha, Yuheng Li, and Yong Jae Lee. Towards uni- versal fake image detectors that generalize across generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 24480– 24489, 2023. 3, 2

    2023

  57. [65]

    Bleu: a method for automatic evaluation of machine translation

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318,

  58. [66]

    Mllm- enhanced face forgery detection: A vision-language fusion solution

    Siran Peng, Zipei Wang, Li Gao, Xiangyu Zhu, Tianshuo Zhang, Ajian Liu, Haoyuan Zhang, and Zhen Lei. Mllm- enhanced face forgery detection: A vision-language fusion solution. arXiv preprint arXiv:2505.02013, 2025. 1

    2025 arXiv

  59. [67]

    Explainable tam- pered text detection via multimodal large models

    Chenfan Qu, Jian Liu, Haoxing Chen, Baihan Yu, Jingjing Liu, Weiqiang Wang, and Lianwen Jin. Explainable tam- pered text detection via multimodal large models. arXiv preprint arXiv:2412.14816, 2024. 1, 3, 4

    2024 arXiv

  60. [68]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. In International conference on machine learning...

    2021

  61. [69]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christo- pher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 2024. 2, 3

    2024

  62. [70]

    Towards the detection of diffusion model deepfakes

    Jonas Ricker, Simon Damm, Thorsten Holz, and Asja Fis- cher. Towards the detection of diffusion model deepfakes. arXiv preprint arXiv:2210.14571, 2022. 2

    2022 arXiv

  63. [71]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684–10695, 2022. 1

    2022

  64. [72]

    Forsyth, and Anand Bhattad

    Ayush Sarkar, Hanlin Mai, Amitabh Mahapatra, Svetlana Lazebnik, D.A. Forsyth, and Anand Bhattad. Shadows don’t lie and lines can’t bend! generative models don’t know pro- jective geometry...for now. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog...

    2024

  65. [73]

    De-fake: Detection and attribution of fake images generated by text-to- image diffusion models

    Zeyang Sha, Zheng Li, Ning Yu, and Yang Zhang. De-fake: Detection and attribution of fake images generated by text-to- image diffusion models. arXiv preprint arXiv:2210.06998,

    arXiv

  66. [74]

    On learn- ing multi-modal forgery representation for diffusion gener- ated video detection

    Xiufeng Song, Xiao Guo, Jiache Zhang, Qirui Li, Lei Bai, Xiaoming Liu, Guangtao Zhai, and Xiaohong Liu. On learn- ing multi-modal forgery representation for diffusion gener- ated video detection. In Proceeding of Thirty-eighth Confer- ence on Neural Information Processing Syst...

    2024

  67. [75]

    Domain general face forgery detection by learning to weight

    Ke Sun, Hong Liu, Qixiang Ye, Yue Gao, Jianzhuang Liu, Ling Shao, and Rongrong Ji. Domain general face forgery detection by learning to weight. In Proceedings of the AAAI conference on artificial intelligence, pages 2638–2646, 2021. 1

    2021

  68. [76]

    An information theoretic approach for attention-driven face forgery detection

    Ke Sun, Hong Liu, Taiping Yao, Xiaoshuai Sun, Shen Chen, Shouhong Ding, and Rongrong Ji. An information theoretic approach for attention-driven face forgery detection. In European conference on computer vision, pages 111–127. Springer, 2022

    2022

  69. [77]

    Dual contrastive learning for general face forgery detection

    Ke Sun, Taiping Yao, Shen Chen, Shouhong Ding, Jilin Li, and Rongrong Ji. Dual contrastive learning for general face forgery detection. In Proceedings of the AAAI conference on artificial intelligence, pages 2316–2324, 2022

    2022

  70. [78]

    Continual face forgery detection via historical distribution preserving

    Ke Sun, Shen Chen, Taiping Yao, Xiaoshuai Sun, Shouhong Ding, and Rongrong Ji. Continual face forgery detection via historical distribution preserving. International Journal of Computer Vision, 133(3):1067–1084, 2025. 1

    2025

  71. [79]

    Towards gen- eral visual-linguistic face forgery detection

    Ke Sun, Shen Chen, Taiping Yao, Ziyin Zhou, Jiayi Ji, Xi- aoshuai Sun, Chia-Wen Lin, and Rongrong Ji. Towards gen- eral visual-linguistic face forgery detection. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 19576–19586, 2025. 1

    2025

  72. [80]

    Autoregressive model beats diffusion: Llama for scalable image generation

    Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autoregressive model beats diffusion: Llama for scalable image generation. arXiv preprint arXiv:2406.06525, 2024. 7, 2

    2024 arXiv

  73. [81]

    Forgerysleuth: Em- powering multimodal large language models for image ma- nipulation detection

    Zhihao Sun, Haoran Jiang, Haoran Chen, Yixin Cao, Xipeng Qiu, Zuxuan Wu, and Yu-Gang Jiang. Forgerysleuth: Em- powering multimodal large language models for image ma- nipulation detection. arXiv preprint arXiv:2411.19466, 2024. 1, 3, 4, 5

    2024

  74. [82]

    Learning on gradients: Generalized ar- tifacts representation for gan-generated images detection

    Chuangchuang Tan, Yao Zhao, Shikui Wei, Guanghua Gu, and Yunchao Wei. Learning on gradients: Generalized ar- tifacts representation for gan-generated images detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12105–12114, 2023. 2

    2023

  75. [83]

    Rethinking the up-sampling operations in cnn-based generative network for generaliz- able deepfake detection

    Chuangchuang Tan, Yao Zhao, Shikui Wei, Guanghua Gu, Ping Liu, and Yunchao Wei. Rethinking the up-sampling operations in cnn-based generative network for generaliz- able deepfake detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p...

    2024

  76. [84]

    Visual autoregressive modeling: Scalable image generation via next-scale prediction

    Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction. 2024. 7, 2

    2024

  77. [85]

    Visual autoregressive modeling: Scalable im- age generation via next-scale prediction

    Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Li- wei Wang. Visual autoregressive modeling: Scalable im- age generation via next-scale prediction. arXiv preprint arXiv:2404.02905, 2024. 1

    2024 arXiv

  78. [86]

    Cider: Consensus-based image description eval- uation

    Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. Cider: Consensus-based image description eval- uation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4566–4575, 2015. 5

    2015

  79. [87]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Jun- yang Lin. Qwen2-vl: Enhancing vision-language model’s ...

    2024 arXiv

  80. [88]

    Cnn-generated images are surprisingly easy to spot...for now

    Sheng-Yu Wang, Oliver Wang, Richard Zhang, Andrew Owens, and Alexei A Efros. Cnn-generated images are surprisingly easy to spot...for now. In CVPR, 2020. 2, 3

    2020

  81. [89]

    Cnn-generated images are surprisingly easy to spot

    Sheng-Yu Wang, Oliver Wang, Richard Zhang, Andrew Owens, and Alexei A Efros. Cnn-generated images are surprisingly easy to spot... for now. In Proceedings of the IEEE/CVF conference on computer vision and pattern recog- nition, pages 8695–8704, 2020. 1, 3, 7, 2

    2020

  82. [90]

    Cogvlm: Visual expert for pretrained language models

    Weihan Wang, Qingsong Lv, Wenmeng Yu, Wenyi Hong, Ji Qi, Yan Wang, Junhui Ji, Zhuoyi Yang, Lei Zhao, Xixuan Song, et al. Cogvlm: Visual expert for pretrained language models. arXiv preprint arXiv:2311.03079, 2023. 2

    2023 arXiv

  83. [91]

    Dire for diffusion-generated image detection

    Zhendong Wang, Jianmin Bao, Wengang Zhou, Weilun Wang, Hezhen Hu, Hong Chen, and Houqiang Li. Dire for diffusion-generated image detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 22445–22455, 2023. 1, 3, 2

    2023

  84. [92]

    Spot the fake: Large multimodal model-based synthetic image detection with artifact explanation

    Siwei Wen, Junyan Ye, Peilin Feng, Hengrui Kang, Zichen Wen, Yize Chen, Jiang Wu, Wenjun Wu, Conghui He, and Weijia Li. Spot the fake: Large multimodal model-based synthetic image detection with artifact explanation. arXiv preprint arXiv:2503.14905, 2025. 1

    2025

  85. [93]

    Janus: Decoupling visual encoding for unified multimodal understanding and generation

    Chengyue Wu, Xiaokang Chen, Zhiyu Wu, Yiyang Ma, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, Chong Ruan, et al. Janus: Decoupling visual encoding for unified multimodal understanding and generation. arXiv preprint arXiv:2410.13848, 2024. 7, 2

    arXiv

  86. [94]

    Q-bench: A benchmark for general-purpose foundation models on low-level vision

    Haoning Wu, Zicheng Zhang, Erli Zhang, Chaofeng Chen, Liang Liao, Annan Wang, Chunyi Li, Wenxiu Sun, Qiong Yan, Guangtao Zhai, et al. Q-bench: A benchmark for general-purpose foundation models on low-level vision. arXiv preprint arXiv:2309.14181, 2023. 2, 5

    2023 arXiv

  87. [95]

    H. Wu, J. Zhou, and S. Zhang. Generalizable synthetic image detection via language-guided contrastive learning. arXiv preprint:2305.13800, 2023. 2

    2023 arXiv

  88. [96]

    Show-o: One single transformer to unify multimodal understanding and generation

    Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single transformer to unify multimodal understanding and generation. arXiv preprint arXiv:2408.12528, 2024. 7, 2

    2024 arXiv

  89. [97]

    Fakeshield: Explainable image forgery detection and localization via multi-modal large language models

    Zhipei Xu, Xuanyu Zhang, Runyi Li, Zecheng Tang, Qing Huang, and Jian Zhang. Fakeshield: Explainable image forgery detection and localization via multi-modal large language models. arXiv preprint arXiv:2410.02761, 2024. 1, 3, 4, 5

    2024 arXiv

  90. [98]

    A sanity check for ai-generated image detection

    Shilin Yan, Ouxiang Li, Jiayin Cai, Yanbin Hao, Xi- aolong Jiang, Yao Hu, and Weidi Xie. A sanity check for ai-generated image detection. arXiv preprint arXiv:2406.19435, 2024. 1, 3, 7, 2, 4

    2024 arXiv

  91. [99]

    Heie: Mllm-based hierarchical explainable aigc image implausi- bility evaluator

    Fan Yang, Ru Zhen, Jianing Wang, Yanhao Zhang, Haoxiang Chen, Haonan Lu, Sicheng Zhao, and Guiguang Ding. Heie: Mllm-based hierarchical explainable aigc image implausi- bility evaluator. arXiv preprint arXiv:2411.17261 , 2024. 1

    2024 arXiv

  92. [100]

    Loki: A comprehensive synthetic data detection benchmark using large multimodal models

    Junyan Ye, Baichuan Zhou, Zilong Huang, Junan Zhang, Tianyi Bai, Hengrui Kang, Jun He, Honglin Lin, Zihao Wang, Tong Wu, et al. Loki: A comprehensive synthetic data detection benchmark using large multimodal models. arXiv preprint arXiv:2410.09732, 2024. 2, 3, 4

    arXiv

  93. [101]

    Unlocking the capabilities of large vision-language models for generalizable and explainable deepfake detection

    Peipeng Yu, Jianwei Fei, Hui Gao, Xuan Feng, Zhihua Xia, and Chip Hong Chang. Unlocking the capabilities of large vision-language models for generalizable and explainable deepfake detection. arXiv preprint arXiv:2503.14853, 2025. 1

    2025 arXiv

  94. [102]

    Common sense reasoning for deepfake de- tection

    Yue Zhang, Ben Colman, Xiao Guo, Ali Shahriyari, and Gaurav Bharaj. Common sense reasoning for deepfake de- tection. In European Conference on Computer Vision, pages 399–415. Springer, 2024. 1

    2024

  95. [103]

    Why are visually-grounded language models bad at image classi- fication? arXiv preprint arXiv:2405.18415, 2024

    Yuhui Zhang, Alyssa Unell, Xiaohan Wang, Dhruba Ghosh, Yuchang Su, Ludwig Schmidt, and Serena Yeung-Levy. Why are visually-grounded language models bad at image classi- fication? arXiv preprint arXiv:2405.18415, 2024. 2, 5

    2024 arXiv

  96. [104]

    Common sense reasoning for deepfake de- tection

    Yue Zhang, Ben Colman, Xiao Guo, Ali Shahriyari, and Gaurav Bharaj. Common sense reasoning for deepfake de- tection. In European Conference on Computer Vision, pages 399–415. Springer, 2025. 1, 3

    2025

  97. [105]

    Judging llm-as-a-judge with mt-bench and chatbot arena

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuo- han Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Infor- mation Processing Systems, 36:46595–46623, 2023. 3

    2023

  98. [107]

    Patchcraft: Exploring texture patch for efficient ai-generated image detection

    Nan Zhong, Yiran Xu, Sheng Li, Zhenxing Qian, and Xinpeng Zhang. Patchcraft: Exploring texture patch for efficient ai-generated image detection. arXiv preprint arXiv:2311.12397, pages 1–18, 2024. 3

    2024 arXiv

  99. [108]

    Genimage: A million-scale benchmark for detecting ai-generated image, 2023

    Mingjian Zhu, Hanting Chen, Qiangyu Yan, Xudong Huang, Guanyu Lin, Wei Li, Zhijun Tu, Hailin Hu, Jie Hu, and Yunhe Wang. Genimage: A million-scale benchmark for detecting ai-generated image, 2023. 2, 3, 4 AIGI-Holmes: Towards Explainable and Generalizable AI-Generated Image De...

    2023

  100. [109]

    ▷ LLM-assisted refinement 31: D2 ← D2 ∪ {(I, A′)} 32: end for 33: for each (I, A) ∈ Dmllm do 34: A′ 2 ← MLLMRevise(A) ▷ MLLM-Expert modifications 35: A′′ ← DeepseekV3(A, A′

  101. [110]

    This is a real image

    ▷ LLM-assisted refinement 36: D2 ← D2 ∪ {(I, A′′)} 37: end for 38: // Stage 4: Comprehensive Evaluation 39: Dtest ← HumanRevise(1K) 40: Compute {BLEU, CIDEr, ROUGE-L, METER} on Dtest 41: MLLM-Score ← 1 |J | P J∈J BatchScore(Dtest, J) 42: Human-Score ← ExpertPreferenceAssessmen...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.