Pith. sign in

REVIEW 3 major objections 7 minor 2 cited by

Multimodal Conditional Information Bottleneck for Generalizable AI-Generated Image Detection

T0 review · 3 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read In CLIP's feature space, arbitrary text sits closer to fake images than to real ones, and InfoFD, a text-guided conditional information bottleneck built on that bias, reaches 97.82% average accuracy on GenImage and 98.92% on CO-SPY.

desk verdict The CLIP text-image bias is a solid empirical find and the CO-SPY numbers are striking, but the IB theory is not sound as written and the bias's backbone stability is unproven. read the letter →

arxiv 2505.15217 v1 pith:DCHVDZZ6 submitted 2025-05-21 cs.CV

Haotian Qin , Dongliang Chang , Yueying Gao , Bingyao Yu , Lei Chen , Zhanyu Ma This is my paper · ORCID
classification cs.CV
keywords AI-generatedimagedetectionCLIPinformationbottleneckmultimodalconditioninggeneralizationtext-guideddiffusionmodelsforgery
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper seeks to establish that CLIP's text encoder carries a stable, generator-independent 'bias': in the frozen CLIP feature space, arbitrary text features are more similar to fake images than to real ones, with the gap largest at the 11th transformer layer, and that this bias can be turned into a strong generalizable detector. The proposed InfoFD framework trains only a lightweight head on top of frozen CLIP features, compressing them with a multimodal conditional information bottleneck that conditions on both text and class labels, and uses a dynamic text orthogonalization scheme to balance prompt semantics against the global bias. If the central claim is right, effective detection of images from unseen generative models does not require fine-tuning CLIP or knowing the generator family; it follows from exploiting a geometric property already present in CLIP's text-image embedding space. The paper reports 97.82% average accuracy on GenImage and 98.92% average accuracy on CO-SPY when trained only on GenImage/Stable Diffusion 1.4, which is why the result matters for practical AI-image forensics.

What carries the argument

The load-bearing mechanism is the multimodal conditional information bottleneck (MM-CIB) objective, $\min I(Z; X | T, Y) - \beta I(Z; Y)$, instantiated as a loss $L = L_{\mathrm{MMD}} + \beta L_{\mathrm{cls}}$. A stochastic encoder $e(z|x,t)$ maps augmented CLIP image features into a latent $z$, and a variational conditional $r(z|t,y)$ supplies the target distribution, with the gap between them measured by maximum mean discrepancy rather than KL divergence for numerical stability. The text side is built from CLIP text features that are linearly projected, averaged per class, and orthogonalized by Gram-Schmidt so that the two conditionals $r(z|t,\mathrm{real})$ and $r(z|t,\mathrm{fake})$ share no overlapping direction; Dynamic Text Orthogonalization then replaces the per-image prompt with a weighted fusion of the current batch and a historical memory, preserving prompt semantics while retaining the global 'bias'. Around this core, Composite Gaussian Perturbation adds mixed Gaussian and uniform noise to the image features to simulate cross-domain variation, and all training runs on the output of CLIP's first 12 transformer layers, the layer-11 [CLS] token being the feature that carries the largest real-vs-fake bias.

What would settle it

Take a CLIP variant with a different backbone size or different training data, encode the same real and fake image sets, and compute the cosine similarity between the pooled random-text feature and each image at each layer; if the fake-over-real ordering at layer 11 vanishes or reverses for one generator family, the bias is not universal and InfoFD's core premise fails. A second test is to train a generator specifically to minimize the layer-11 bias, then check whether InfoFD's accuracy collapses while a frequency-domain detector still works.

Watch

Extended reading notes

Core claim

InfoFD's central discovery is the 'bias': in the CLIP feature space, cosine similarity between any text feature—even a string of random characters—and an image is higher when the image is machine-generated than when it is real, and this ordering repeats across GAN and diffusion generators and is strongest in layer 11's output. The paper argues this is not a prompt-semantics effect, since random text preserves it and semantic prompts only sharpen it. On top of that observation, InfoFD introduces a multimodal conditional information bottleneck that compresses CLIP's layer-11 image features by minimizing $I(Z; X | T, Y) - \beta I(Z; Y)$, with the variational conditional distribution $r(z|t,y)$ built from Gram-Schmidt-orthogonalized real/fake text prototypes, plus a Dynamic Text Orthogonalization module that fuses mini-batch and historical text features to keep both semantic content and the global bias. With this setup, the frozen CLIP backbone plus a small trained head reaches 97.82% average accuracy on GenImage and 98.92% on CO-SPY, including near-90% or better on generators like ADM, FLUX.1-dev, and SDXL variants that were never seen in training.

Load-bearing premise

The load-bearing premise is that CLIP's layer-11 text-image cosine-similarity 'bias' is a stable, generator-independent property of the frozen model; if that bias is an accident of one CLIP checkpoint or of the particular generators used to measure it, the reported generalization to unseen generators does not follow.

Editorial extensions

If this is right

  • Frozen CLIP features suffice: a lightweight trained head can detect AI images without fine-tuning the vision-language backbone, making the method cheap to train and easy to update as new generators appear.
  • Prompt text does not need to describe the image; random or category-level text already carries the fake-vs-real signal, so detection does not depend on prompt engineering.
  • The layer-11 bias transfers across generator families: training on one diffusion model generalizes to GANs and to newer diffusion models, with the largest gains on generators whose layer-11 bias is most pronounced.
  • Compressing CLIP features with conditional text information removes redundant information and preserves class-relevant signal, as shown by mutual-information estimates comparing MM-CIB to a plain information bottleneck.
  • Under common image degradations such as JPEG compression and Gaussian blur, the detector keeps high average precision even when accuracy at threshold 0.5 drops, and the degradation profile depends on which CLIP layer is chosen.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the layer-11 bias is as universal as the paper suggests, CLIP's text space likely contains a single 'generated-image' direction, and one could search for the text vector that maximizes the real/fake cosine gap across many generators; that vector would be a probe, not just a training signal.
  • The method's dependence on layer 11 invites a testable extension: the exact layer that maximizes the bias may vary across CLIP backbones or training data, so a layer-selection rule based on measured bias could make the approach portable to other vision-language models.
  • Because the paper leaves the bias's origin unexplained, a natural next experiment is to check whether the bias shrinks or disappears when CLIP is trained on data that is explicitly filtered to exclude machine-generated images; that would connect the phenomenon to CLIP's training distribution.
  • A practical consequence the authors do not draw: the same text-bias signal could be embedded in a public API or browser extension at near-zero inference cost, since features are precomputed and the trained head is a small network.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. This paper proposes InfoFD, a CLIP-based detector of AI-generated images that trains only a lightweight head on frozen CLIP features. The authors first report a 'bias': in CLIP's feature space, text features (including random strings) have higher cosine similarity to fake images than to real images, most strongly at layer 11 of the ViT-L/14 encoder. InfoFD then applies a Text-Guided Conditional Information Bottleneck (TGCIB) with Composite Gaussian Perturbation and a loss that matches the encoded representation's mean to a class-conditional text target, and a Dynamic Text Orthogonalization (DTO) module that accumulates and orthogonalizes pooled text features across batches. Evaluated under GenImage's SD V1.4 training protocol and on the CO-SPY benchmark, the method reports 97.82% and 98.92% average accuracy, outperforming several published baselines. The paper claims that this is the first multimodal conditional information bottleneck for the task and that text and class labels are not required at inference.

Significance. Conditional on the empirical results, the paper makes a practically valuable contribution: it demonstrates strong cross-generator generalization, including recent models in CO-SPY (SDXL, FLUX, LCM, etc.), using a frozen CLIP encoder and a small trainable head, and it releases code. The ablation of the text-guidance design and the additional AIGCBenchmark evaluation are credit-worthy. However, the theoretical information-bottleneck derivation is not supported as written, and the generality of the 'bias' is only demonstrated for one CLIP checkpoint. If the authors either supply a valid derivation or explicitly reframe the LMMD loss as an empirical regularizer, and if they add cross-backbone evidence for the bias, the paper would be a solid empirical contribution.

major comments (3)
  1. [§4.1 and Appendix B.1 (Eq. (2), Eq. (5), Eqs. (13)–(14))] The derivation does not establish a valid variational bound. Equation (10) obtains an upper bound involving E[log(e/r)] − βI(Z;Y), but Equation (13) then replaces the KL term with the squared L2 distance between the means of e and r. With the identity kernel in R^d, MMD is not an upper bound on KL, and it discards the covariance term. For Gaussian e(z|x,t)=N(μe,Σe) and r(z|t,y)=N(μr,I), the KL is 0.5(tr(Σe)+||μe−μr||^2−d−log det Σe), so the LMMD loss can be driven to zero while the true KL remains large. Since the paper's central methodological narrative (Section 1 and Figure 1a) is that InfoFD implements a conditional information bottleneck, this claim needs either a genuine bound or an explicit reframing of LMMD as a heuristic regularizer. The assumed Markov properties Z↔X↔Y and Z↔T↔Y are also asserted without justification, which matters if the IB interpretation is retained.
  2. [§3, Figure 2, Appendix A.2, Tables 1–2] The 'bias' that determines the layer choice, the DTO text direction, and the LMMD target r(z|t,y) is characterized with a single CLIP checkpoint (ViT-L/14) and with real/fake pairings based on ImageNet or LAION versus GenImage/Stable-Diffusion-family images. Figure 2b uses GenImage SD V1.5, the same generator family used for training and evaluation, so the layer-11 selection is partly in-family. Appendix A.2 explicitly states that the causes of the bias remain unclear. The strong CO-SPY results do not fully resolve this concern because CO-SPY is evaluated with the same CLIP encoder. I ask for cross-backbone bias measurements (e.g., ViT-B/32, ViT-B/16, RN50) and, if feasible, at least one InfoFD variant trained and evaluated with a different backbone, so that the bias can be shown to be a property of CLIP's text–image alignment rather than of one checkpoint.
  3. [§5.1, Evaluation Protocol 2] The role of the InterVL-generated prompts is not specified. The paper claims in Sections 1 and 4 that text and class labels are not required at inference, but the EP2 protocol introduces prompt generation for the CO-SPY assessment without explaining whether these prompts are used to construct r(z|t,y) during inference or only as auxiliary training information. If prompts are used at inference, the 'no text at inference' claim is contradicted; if they are not, the protocol should state which text features, if any, are used during evaluation. This distinction is needed for reproducibility and for assessing the practical claim of text-free deployment.
minor comments (7)
  1. [§5.3, Table 3] The ablation table rows consist only of checkmarks with no row labels or legend, making it impossible to know which component configuration each row represents; please label each row explicitly.
  2. [Throughout] The method name is capitalized inconsistently ('infoFD' vs. 'InfoFD'); please unify the spelling.
  3. [Figure 2(a) caption] The caption says 'All images are sourced from ImageNet [3]', but the figure includes fake images from GenImage generators; the caption should distinguish the sources of real and fake images.
  4. [§3] The sentence beginning 'However, Wu et al. [8] found...' is a sentence fragment; please revise so that the two questions in the numbered list are properly introduced.
  5. [Appendix C.2] The reference to Gandelsman et al. [63] as 'awesome work' is informal; please remove this expression in a journal-style revision.
  6. [Eq. (2)] The equation typesets the log-ratio as a fraction with numerator and denominator on separate lines, which is hard to read; please reformat so that the expression is unambiguous.
  7. [Table 7 discussion] The text 'We both provide accuracy and Diff in the table' is awkward and should be rephrased.

Circularity Check

0 steps flagged · score 2.0 of 10

No load-bearing circularity: the detector is validated on external benchmarks and the text-anchored bottleneck is an auxiliary training signal, not a renamed fit.

full rationale

InfoFD's derivation chain rests on an empirical observation (CLIP text features exhibit higher cosine similarity to fake images than to real images, strongest at layer 11) and on an auxiliary training objective that pulls image representations toward class-conditioned text anchors. Neither step is defined in terms of the final accuracy. The LMMD term (Eq. 5) compares the encoder mean to the text-derived r(z|t,y); DTO (Eq. 8) pools and orthogonalizes CLIP text features. These are construction choices, not predictions, and the final classifier is trained with cross-entropy and evaluated on held-out generator categories. The CO-SPY results (Table 2) provide an external check that is not used in any of the bias measurements, and text is not required at inference. The GenImage evaluation is slightly weakened because the layer-11 choice is motivated by Figure 2b, which includes GenImage/SD V1.5, one of the Table 1 test categories; this is a model-selection caveat, not a reduction of the result to its inputs. The only same-author citation (Ref. [39]) appears in a related-work list and is not load-bearing. Section A.2 explicitly admits that the cause of the bias remains unclear, which is an honest limitation rather than a circular justification. Overall, the central claims have independent empirical content.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

No new physical or model-level entities are introduced; DTO and TGCIB are software modules, not postulated mechanisms requiring independent evidence. The free parameters are all tuning choices, and the axioms are the unstated or loosely stated premises behind the information bottleneck derivation and the bias-guided training.

free parameters (5)
  • beta (Lagrange multiplier) = 0.15
    Balances the MMD compression term and the cross-entropy classification term; tuned on the GenImage validation set.
  • Hidden dimension d = 64
    Dimension of the compressed representation z; chosen to compress CLIP features.
  • Lp (global pooling scale) = 512
    Controls how much historical text information DTO retains; set in the implementation details.
  • CGP noise scale = mu_e=0.5, Sigma_e=0.4 (or Sigma_s)
    Controls the intensity of Composite Gaussian Perturbation; hand-set hyperparameters.
  • CLIP layer index = 11
    Feature extraction layer chosen because the observed text-image bias is largest there (Fig 2b); this choice is part of the method.
assumptions (5)
  • domain assumption CLIP's pretrained image and text representations are informative for distinguishing real from AI-generated images.
    The whole method relies on CLIP features; no evidence is provided that the same bias exists in other embeddings.
  • domain assumption The bias (higher cosine similarity of text features to fake than real images) is a stable property across generative models and CLIP layers, strongest at layer 11.
    This is empirically observed in Section 3, but the underlying cause is not explained, and the paper's own Limitations (A.2) state the cause remains unclear.
  • ad hoc to paper The Markov chain conditions Z-X-Y and Z-T-Y hold for the model.
    Assumed to derive the objective in Eq (1), yet the encoder e(z|x,t) depends on both X and T, which is inconsistent with either chain.
  • ad hoc to paper The MMD between means is a faithful surrogate for the KL divergence between e(z|x,t) and r(z|t,y).
    Eq (5) replaces the KL term with a squared-distance on means, which is not a bound and discards variance information.
  • domain assumption CGP noise simulates domain shift across generators.
    Used to improve generalization; no formal guarantee that this augmentation matches real distribution shifts.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multimodal Conditional Information Bottleneck for Generalizable AI-Generated Image Detection." pith.science (2026). https://pith.science/paper/DCHVDZZ6

@misc{pith2026250515217,
  author       = {Pith},
  title        = {Pith review of: Multimodal Conditional Information Bottleneck for Generalizable AI-Generated Image Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DCHVDZZ6}},
  note         = {Machine review of arXiv:2505.15217}
}
read the original abstract

Although existing CLIP-based methods for detecting AI-generated images have achieved promising results, they are still limited by severe feature redundancy, which hinders their generalization ability. To address this issue, incorporating an information bottleneck network into the task presents a straightforward solution. However, relying solely on image-corresponding prompts results in suboptimal performance due to the inherent diversity of prompts. In this paper, we propose a multimodal conditional bottleneck network to reduce feature redundancy while enhancing the discriminative power of features extracted by CLIP, thereby improving the model's generalization ability. We begin with a semantic analysis experiment, where we observe that arbitrary text features exhibit lower cosine similarity with real image features than with fake image features in the CLIP feature space, a phenomenon we refer to as "bias". Therefore, we introduce InfoFD, a text-guided AI-generated image detection framework. InfoFD consists of two key components: the Text-Guided Conditional Information Bottleneck (TGCIB) and Dynamic Text Orthogonalization (DTO). TGCIB improves the generalizability of learned representations by conditioning on both text and class modalities. DTO dynamically updates weighted text features, preserving semantic information while leveraging the global "bias". Our model achieves exceptional generalization performance on the GenImage dataset and latest generative models. Our code is available at https://github.com/Ant0ny44/InfoFD.

Figures

Figures reproduced from arXiv: 2505.15217 by the authors.

Figure 1
Figure 1. Illustration of Multimodal Conditional Information Bottleneck and T-SNE visual￾ization of “bias”. (a) (I) The standard Information Bottleneck (IB) [1]. (II) The Conditional Information Bottleneck (CIB) [2]. (III) Our MultiModal Conditional Information Bottleneck (MM￾CIB). (b) Blue triangles: fake images. Green circles: real images. Red triangles: prompts for fake images. Yellow circles: prompts for real images. Imag… view at source ↗
Figure 2
Figure 2. (a) Mean cos(·, ·) of different datasets with the same pooling random text features t˜. Here, Ir denotes real image features encoded by layer11, If denotes fake image features, and t˜ consists of 1,000 randomly generated strings with lengths between 30 and 70, composed of randomly mixed uppercase and lowercase letters, using 3−σ error bars. All images are sourced from ImageNet [3]. For more detailed experimental res… view at source ↗
Figure 3
Figure 3. Pipeline of infoFD. InfoFD is composed of two components: prompts processing and image processing. During the text processing phase, the texts are first categorized into real and fake groups based on their corresponding image classes. DTO: Subsequently, we employ DTO for information integration, ultimately obtaining r(z | t, y). We directly utilize the first 12 layers of the ViT in CLIP as the image encoder, based o… view at source ↗
Figures from the paper (13 more)
Figure 4
Figure 4. Figure 4: (a) ID(X;Z) against ID(Z; Y ) on MMCIB and IB. The black line represents the trend of MMCIB obtained through triple curve fitting. The results are calculated on GenImage [18] Test set. (b) Illustration of feature redundancy. The darker the color, the greater the degree…
Figure 5
Figure 5. Figure 5: PCA Projection. Green: real image features; Red: fake image features; CLIP: CLIP feature projections; z: TGIB-extracted representations. PCA 2D Projection. We conducted PCA on the features extracted by the ViT and projected them into a 2D space, with the results shown …
Figure 6
Figure 6. Figure 6: Frequency histogram of cosine similarity between text features and image features. The left figure shows the frequency plot of cosine similarity between real images and t¯r and t¯f , while the right figure displays the frequency plot of cosine similarity between fake i…
Figure 7
Figure 7. Figure 7: BigGAN MJ SD V1.4 SD V1.5 VQDM Wukong GLIDE ADM 0.000 0.025 0.050 0.075 0.100 0.125 0.150 0.175 Cosine similarity cos (Ir, t¯r) cos Ir, t¯f  cos If , t¯r  cos If , t¯f  [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: Mean cosine similarity of different datasets with the same pooling real and fake text features on UniFD [7]. C.2 Layers Experiments Layer Diff. To further analyze, we recorded the [CLS] outputs of different blocks in the ViT-L/14 encoder of CLIP, performed projection t…
Figure 9
Figure 9. Figure 9: Mean cosine similarity of different datasets with the same pooling real and fake text features on GANs and Diffusions. Figure 2b indicates that the output of the last layer of the ViT encoder in CLIP is not optimal. The “bias” exhibits a more pronounced discriminative …
Figure 10
Figure 10. Figure 10: Frequency histograms of cosine similarity between the outputs of different layers of the ViT in CLIP and random text features for the DiffusionDB and LAION datasets. There are 50,000 real images and 50,000 fake images. All output image features were transformed using …
Figure 11
Figure 11. Figure 11: DFT results of intermediate feature maps from Stable Diffusion. We processed 200 fake images from Stable Diffusion and 200 real images from ImageNet, extracting intermediate layer outputs excluding [CLS] and reshaping them into 16x16 patches. Summing along channels pr…
Figure 12
Figure 12. Figure 12: Images with varying degrees of degradation processing. robustness to perturbations. To validate this hypothesis, we conducted additional experiments using the final layer (Layer23) as input. The results confirm that Layer23 indeed shows significantly less performance …
Figure 13
Figure 13. Figure 13: Comparison of ID(X;Z) against ID(Z; Y ) on MultiModal Conditional Information Bottleneck (MMCIB) and Information Bottleneck (IB). The black line represents the trend of MMCIB obtained through triple curve fitting. Discrete Fourier Transform. We conduct more visualizat…
Figure 14
Figure 14. Figure 14: DFT result plots for BigGAN, Wukong, and Midjourney. 22 [PITH_FULL_IMAGE:figures/full_fig_p022_14.png]
Figure 15
Figure 15. Figure 15: DFT result plots for ADM, GLIDE, and VQDM. 23 [PITH_FULL_IMAGE:figures/full_fig_p023_15.png]
Figure 16
Figure 16. Figure 16: Visualization of cosine similarity between the features of real and fake images generated by Wukong, VQDM, GLIDE, and ADM models, and random text features, across different layers of the ViT in CLIP. The features were transformed using the projection layer of the ViT.…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. VGIF-Score: Interpretable and Diagnostic Evaluation of Spatio-Temporal Instruction Following in Video Generation

    cs.CV 2026-07 conditional novelty 7.0 of 10

    VGIF-Score decomposes video prompts into dependency graphs and uses a VLM to diagnose which instruction constraints models satisfy, revealing strong failures on causal and late-prompt constraints.

  2. FlexiGrad: Adaptive Gradient Modulation for Hierarchical Fine-Grained Classification

    cs.CV 2026-07 conditional novelty 4.0 of 10

    FlexiGrad selectively removes conflicting and reinforces agreeing gradient components between hierarchy levels, improving multi-granularity accuracy on CUB, FGVC-Aircraft and Stanford Cars.

Reference graph

Works this paper leans on

68 extracted references · 54 canonical work pages · cited by 2 Pith papers

  1. [1]

    Deep variational information bottleneck

    Alexander A Alemi, Ian Fischer, Joshua V Dillon, and Kevin Murphy. Deep variational information bottleneck. In International Conference on Learning Representations, 2017

    work page 2017

  2. [2]

    The conditional entropy bottleneck

    Ian Fischer. The conditional entropy bottleneck. Entropy, 2020

    work page 2020

  3. [3]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR. IEEE, 2009

    work page 2009

  4. [4]

    Diffusiondb: A large-scale prompt gallery dataset for text-to-image genera- tive models

    Zijie J Wang, Evan Montoya, David Munechika, Haoyang Yang, Benjamin Hoover, and Duen Horng Chau. Diffusiondb: A large-scale prompt gallery dataset for text-to-image genera- tive models. In ACL, 2023

    work page 2023

  5. [5]

    Forgery-aware adaptive transformer for generalizable synthetic image detection

    Huan Liu, Zichang Tan, Chuangchuang Tan, Yunchao Wei, Jingdong Wang, and Yao Zhao. Forgery-aware adaptive transformer for generalizable synthetic image detection. In CVPR, 2024

    work page 2024

  6. [6]

    CLIPping the Deception: Adapting Vision- Language Models for Universal Deepfake Detection

    Sohail Ahmed Khan and Duc-Tien Dang-Nguyen. CLIPping the Deception: Adapting Vision- Language Models for Universal Deepfake Detection. In ICMR, 2024

    work page 2024

  7. [7]

    Towards Universal Fake Image Detectors that Generalize Across Generative Models

    Utkarsh Ojha, Yuheng Li, and Yong Jae Lee. Towards Universal Fake Image Detectors that Generalize Across Generative Models. In CVPR, 2023

    work page 2023

  8. [8]

    Generalizable synthetic image detection via language-guided contrastive learning

    Haiwei Wu, Jiantao Zhou, and Shile Zhang. Generalizable synthetic image detection via language-guided contrastive learning. arXiv preprint arXiv:2305.13800, 2023

    arXiv 2023

Show all 68 references
  1. [9]

    Raising the Bar of AI-generated Image Detection with CLIP

    Davide Cozzolino, Giovanni Poggi, Riccardo Corvi, Matthias Nießner, and Luisa Verdoliva. Raising the Bar of AI-generated Image Detection with CLIP. In CVPRW, 2024

    2024

  2. [10]

    A sanity check for ai-generated image detection

    Shilin Yan, Ouxiang Li, Jiayin Cai, Yanbin Hao, Xiaolong Jiang, Yao Hu, and Weidi Xie. A sanity check for ai-generated image detection. arXiv preprint arXiv:2406.19435, 2024

    2024 arXiv

  3. [11]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ICML. PMLR, 2021

    2021

  4. [12]

    Not All Features Matter: Enhancing Few-shot CLIP with Adaptive Prior Refinement

    Xiangyang Zhu, Renrui Zhang, Bowei He, Aojun Zhou, Dong Wang, Bin Zhao, and Peng Gao. Not All Features Matter: Enhancing Few-shot CLIP with Adaptive Prior Refinement. In ICCV. IEEE, 2023

    2023

  5. [13]

    A closer look at the explainability of contrastive language-image pre-training

    Yi Li, Hualiang Wang, Yiqun Duan, Jiheng Zhang, and Xiaomeng Li. A closer look at the explainability of contrastive language-image pre-training. Pattern Recognition, 2025

    2025

  6. [14]

    De-fake: Detection and attribution of fake images generated by text-to-image generation models

    Zeyang Sha, Zheng Li, Ning Yu, and Yang Zhang. De-fake: Detection and attribution of fake images generated by text-to-image generation models. InProceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, 2023

    2023

  7. [15]

    Multimodal information bottleneck: Learning minimal sufficient unimodal and multimodal representations

    Sijie Mai, Ying Zeng, and Haifeng Hu. Multimodal information bottleneck: Learning minimal sufficient unimodal and multimodal representations. IEEE Transactions on Multimedia, 2022

    2022

  8. [16]

    Dynamic Multimodal Information Bottleneck for Multimodality Classification

    Yingying Fang, Shuang Wu, Sheng Zhang, Chaoyan Huang, Tieyong Zeng, Xiaodan Xing, Simon Walsh, and Guang Yang. Dynamic Multimodal Information Bottleneck for Multimodality Classification. In WCAC, 2024. 10

    2024

  9. [17]

    Enhancing multimodal entity and relation extraction with variational information bottleneck

    Shiyao Cui, Jiangxia Cao, Xin Cong, Jiawei Sheng, Quangang Li, Tingwen Liu, and Jinqiao Shi. Enhancing multimodal entity and relation extraction with variational information bottleneck. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2024

    2024

  10. [18]

    Genimage: A million-scale benchmark for detecting ai-generated image

    Mingjian Zhu, Hanting Chen, Qiangyu Yan, Xudong Huang, Guanyu Lin, Wei Li, Zhijun Tu, Hailin Hu, Jie Hu, and Yunhe Wang. Genimage: A million-scale benchmark for detecting ai-generated image. NeurIPS, 2024

    2024

  11. [19]

    Co-spy: Combining semantic and pixel features to detect synthetic images by ai

    Siyuan Cheng, Lingjuan Lyu, Zhenting Wang, Vikash Sehwag, and Xiangyu Zhang. Co-spy: Combining semantic and pixel features to detect synthetic images by ai. In CVPR, 2025

    2025

  12. [20]

    Large scale gan training for high fidelity natural image synthesis

    Andrew Brock. Large scale gan training for high fidelity natural image synthesis. In ICLR, 2019

    2019

  13. [21]

    A style-based generator architecture for generative adversarial networks

    Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In CVPR, 2019

    2019

  14. [22]

    Analyzing and improving the image quality of stylegan

    Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improving the image quality of stylegan. In CVPR, 2020

    2020

  15. [23]

    Alias-free generative adversarial networks

    Tero Karras, Miika Aittala, Samuli Laine, Erik Härkönen, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Alias-free generative adversarial networks. NeurIPS, 2021

    2021

  16. [24]

    Stylegan-xl: Scaling stylegan to large diverse datasets

    Axel Sauer, Katja Schwarz, and Andreas Geiger. Stylegan-xl: Scaling stylegan to large diverse datasets. In ACM SIGGRAPH, 2022

    2022

  17. [25]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. NeurIPS, 2020

    2020

  18. [26]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In ICLR, 2021

    2021

  19. [27]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In CVPR, 2022

    2022

  20. [28]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In ICCV, 2023

    2023

  21. [29]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. In ICLR, 2024

    2024

  22. [30]

    Cnn- generated images are surprisingly easy to spot

    Sheng-Yu Wang, Oliver Wang, Richard Zhang, Andrew Owens, and Alexei A Efros. Cnn- generated images are surprisingly easy to spot... for now. In CVPR, 2020

    2020

  23. [31]

    FaceForensics++: Learning to Detect Manipulated Facial Images

    Andreas Rossler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Niessner. FaceForensics++: Learning to Detect Manipulated Facial Images. In ICCV, 2019

    2019

  24. [32]

    Self-supervised adversarial training for robust face forgery detection

    Yueying Gao, Weiguo Lin, Junfeng Xu, Wanshan Xu, and Peibin Chen. Self-supervised adversarial training for robust face forgery detection. In BMVC, 2023

    2023

  25. [33]

    Intrigu- ing properties of synthetic images: From generative adversarial networks to diffusion models

    Riccardo Corvi, Davide Cozzolino, Giovanni Poggi, Koki Nagano, and Luisa Verdoliva. Intrigu- ing properties of synthetic images: From generative adversarial networks to diffusion models. In CVPRW, 2023

    2023

  26. [34]

    Leveraging frequency analysis for deep fake image recognition

    Joel Frank, Thorsten Eisenhofer, Lea Schönherr, Asja Fischer, Dorothea Kolossa, and Thorsten Holz. Leveraging frequency analysis for deep fake image recognition. In ICML, 2020

    2020

  27. [35]

    Frequency domain image translation: More photo-realistic, better identity-preserving

    Mu Cai, Hong Zhang, Huijuan Huang, Qichuan Geng, Yixuan Li, and Gao Huang. Frequency domain image translation: More photo-realistic, better identity-preserving. In ICCV, 2021

    2021

  28. [36]

    Dire for diffusion-generated image detection

    Zhendong Wang, Jianmin Bao, Wengang Zhou, Weilun Wang, Hezhen Hu, Hong Chen, and Houqiang Li. Dire for diffusion-generated image detection. In CVPR, 2023. 11

    2023

  29. [37]

    DRCT: Diffusion reconstruction contrastive training towards universal detection of diffusion generated images

    Baoying Chen, Jishen Zeng, Jianquan Yang, and Rui Yang. DRCT: Diffusion reconstruction contrastive training towards universal detection of diffusion generated images. In ICML, 2024

    2024

  30. [38]

    How to trace latent generative model generated images without artificial watermark? In ICML, 2024

    Zhenting Wang, Vikash Sehwag, Chen Chen, Lingjuan Lyu, Dimitris N Metaxas, and Shiqing Ma. How to trace latent generative model generated images without artificial watermark? In ICML, 2024

    2024

  31. [39]

    Fakereasoning: Towards generalizable forgery detection and reasoning

    Yueying Gao, Dongliang Chang, Bingyao Yu, Haotian Qin, Lei Chen, Kongming Liang, and Zhanyu Ma. Fakereasoning: Towards generalizable forgery detection and reasoning. arXiv preprint arXiv:2503.21210, 2025

    2025 arXiv

  32. [40]

    C2p-clip: Injecting category common prompt in clip to enhance generalization in deepfake detection

    Chuangchuang Tan, Renshuai Tao, Huan Liu, Guanghua Gu, Baoyuan Wu, Yao Zhao, and Yunchao Wei. C2p-clip: Injecting category common prompt in clip to enhance generalization in deepfake detection. In AAAI, 2025

    2025

  33. [41]

    Emergence of invariance and disentanglement in deep representations

    Alessandro Achille and Stefano Soatto. Emergence of invariance and disentanglement in deep representations. Journal of Machine Learning Research, 2018

    2018

  34. [42]

    Variational discriminator bottleneck: Improving imitation learning, inverse rl, and gans by constraining information flow

    Xue Bin Peng, Angjoo Kanazawa, Sam Toyer, Pieter Abbeel, and Sergey Levine. Variational discriminator bottleneck: Improving imitation learning, inverse rl, and gans by constraining information flow. In ICLR, 2019

    2019

  35. [43]

    Learning efficient multi-agent communication: An information bottleneck approach

    Rundong Wang, Xu He, Runsheng Yu, Wei Qiu, Bo An, and Zinovi Rabinovich. Learning efficient multi-agent communication: An information bottleneck approach. In ICML. PMLR, 2020

    2020

  36. [44]

    Information Bottleneck based Representation Learning for Multimodal Sentiment Analysis

    Tonghui Zhang, Haiying Zhang, Shuke Xiang, and Tong Wu. Information Bottleneck based Representation Learning for Multimodal Sentiment Analysis. In ICCE, 2022

    2022

  37. [45]

    Unpaired Image-to-Speech Synthesis With Multi- modal Information Bottleneck

    Shuang Ma, Daniel Mcduff, and Yale Song. Unpaired Image-to-Speech Synthesis With Multi- modal Information Bottleneck. In ICCV, 2019

    2019

  38. [46]

    Visual explanations of image-text repre- sentations via multi-modal information bottleneck attribution

    Ying Wang, Tim GJ Rudner, and Andrew G Wilson. Visual explanations of image-text repre- sentations via multi-modal information bottleneck attribution. Advances in Neural Information Processing Systems, 2023

    2023

  39. [47]

    Conditional information bottleneck approach for time series imputation

    MinGyu Choi and Changhee Lee. Conditional information bottleneck approach for time series imputation. In ICLR, 2023

    2023

  40. [48]

    Na, Sungwon Kim, Junseok Lee, and Chanyoung Park

    Namkyeong Lee, Dongmin Hyun, Gyoung S. Na, Sungwon Kim, Junseok Lee, and Chanyoung Park. Conditional Graph Information Bottleneck for Molecular Relational Learning. In ICML, 2023

    2023

  41. [49]

    Information Bottleneck Disentanglement for Identity Swapping

    Gege Gao, Huaibo Huang, Chaoyou Fu, Zhaoyang Li, and Ran He. Information Bottleneck Disentanglement for Identity Swapping. In CVPR, 2021

    2021

  42. [50]

    Infogcn: Representation learning for human skeleton-based action recognition

    Hyung-gun Chi, Myoung Hoon Ha, Seunggeun Chi, Sang Wan Lee, Qixing Huang, and Karthik Ramani. Infogcn: Representation learning for human skeleton-based action recognition. In CVPR, 2022

    2022

  43. [51]

    Training generative neural networks via maximum mean discrepancy optimization

    Gintare Karolina Dziugaite, Daniel M Roy, and Zoubin Ghahramani. Training generative neural networks via maximum mean discrepancy optimization. In Proceedings of the Thirty-First Conference on Uncertainty in Artificial Intelligence, 2015

    2015

  44. [52]

    Infovae: Information maximizing variational autoencoders

    Shengjia Zhao, Jiaming Song, and Stefano Ermon. Infovae: Information maximizing variational autoencoders. arXiv preprint arXiv:1706.02262, 2017

    2017 arXiv

  45. [53]

    Auto-encoding variational bayes

    Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013

    2013 arXiv

  46. [54]

    Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In CVPR, 2024. 12

    2024

  47. [55]

    Frequency-aware deepfake detection: Improving generalizability through frequency space domain learning

    Chuangchuang Tan, Yao Zhao, Shikui Wei, Guanghua Gu, Ping Liu, and Yunchao Wei. Frequency-aware deepfake detection: Improving generalizability through frequency space domain learning. In AAAI, 2024

    2024

  48. [56]

    Fusing global and local features for generalized ai-synthesized image detection

    Yan Ju, Shan Jia, Lipeng Ke, Hongfei Xue, Koki Nagano, and Siwei Lyu. Fusing global and local features for generalized ai-synthesized image detection. In ICIP. IEEE, 2022

    2022

  49. [57]

    Rethink- ing the up-sampling operations in cnn-based generative network for generalizable deepfake detection

    Chuangchuang Tan, Yao Zhao, Shikui Wei, Guanghua Gu, Ping Liu, and Yunchao Wei. Rethink- ing the up-sampling operations in cnn-based generative network for generalizable deepfake detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p...

    2024

  50. [58]

    Mine: mutual information neural estimation

    Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeswar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and R Devon Hjelm. Mine: mutual information neural estimation. arXiv preprint arXiv:1801.04062, 2018

    2018 arXiv

  51. [59]

    Assessing neural network representations during training using noise-resilient diffusion spectral entropy

    Danqi Liao, Chen Liu, Benjamin W Christensen, Alexander Tong, Guillaume Huguet, Guy Wolf, Maximilian Nickel, Ian Adelstein, and Smita Krishnaswamy. Assessing neural network representations during training using noise-resilient diffusion spectral entropy. In CISS. IEEE, 2024

    2024

  52. [60]

    A benchmark suite for evaluating neural mutual information estimators on unstructured datasets

    Kyungeun Lee and Wonjong Rhee. A benchmark suite for evaluating neural mutual information estimators on unstructured datasets. In NeurIPS Datasets and Benchmarks Track, 2023

    2023

  53. [61]

    Learning deep representations by mutual information estimation and maximization

    R Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio. Learning deep representations by mutual information estimation and maximization. In ICLR, 2019

    2019

  54. [62]

    Laion-400m: Open dataset of clip-filtered 400 million image-text pairs

    Christoph Schuhmann, Robert Kaczmarczyk, Aran Komatsuzaki, Aarush Katta, Richard Vencu, Romain Beaumont, Jenia Jitsev, Theo Coombes, and Clayton Mullis. Laion-400m: Open dataset of clip-filtered 400 million image-text pairs. In NeurIPS Workshop Datacentric AI, 2021

    2021

  55. [63]

    Interpreting clip’s image representa- tion via text-based decomposition

    Yossi Gandelsman, Alexei A Efros, and Jacob Steinhardt. Interpreting clip’s image representa- tion via text-based decomposition. In ICLR, 2024

    2024

  56. [64]

    Progressive growing of gans for improved quality, stability, and variation

    Tero Karras. Progressive growing of gans for improved quality, stability, and variation. InICLR, 2018

    2018

  57. [65]

    Patchcraft: Exploring texture patch for efficient ai-generated image detection

    Nan Zhong, Yiran Xu, Sheng Li, Zhenxing Qian, and Xinpeng Zhang. Patchcraft: Exploring texture patch for efficient ai-generated image detection. arXiv preprint arXiv:2311.12397, 2023

    2023 arXiv

  58. [66]

    Detecting generated images by real images

    Bo Liu, Fan Yang, Xiuli Bi, Bin Xiao, Weisheng Li, and Xinbo Gao. Detecting generated images by real images. In ECCV. Springer, 2022

    2022

  59. [67]

    Learning on Gradients: Generalized Artifacts Representation for GAN-Generated Images Detection

    Chuangchuang Tan, Yao Zhao, Shikui Wei, Guanghua Gu, and Yunchao Wei. Learning on Gradients: Generalized Artifacts Representation for GAN-Generated Images Detection. In CVPR, 2023. 13 A Additional Discussion A.1 Impact Statement Our work aims to advance the field of machine le...

    2023

  60. [68]

    bias” we identified before. We attribute Layer11’s superior performance to its balance between feature extraction and “bias

    predominantly encapsulate fine-grained image details, which are particularly advantageous for AI-generated detection. Conversely, as Gandelsman et al. [63] point out in their awesome work, the subsequent layers (e.g., layer 20 to 23) concentrate more on high-level semantic inf...

    2000

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.