REVIEW 4 major objections 5 minor 38 references
SUMI-IFL: An Information-Theoretic Framework for Image Forgery Localization with Sufficiency and Minimality Constraints
T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read SUMI-IFL claims that sufficiency and minimality constraints on forgery features—derived from mutual information and the information bottleneck—produce more comprehensive and accurate tamper localization than existing methods.
desk verdict Strong empirical gains undercut by a broken sufficiency-view derivation; the advertised theory doesn't support the losses, but the experiments deserve a referee. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is a pair of loss terms placed on two stages of the network. For sufficiency, since direct estimation of $I(M;F)$ is intractable, the paper optimizes a variational lower bound that compares the prediction distribution with and without each individual view: $L_{SU} = \min \exp(-D_{KL}[P_F \| P_{F \setminus f_i}])$, where $P_F = p(y|F)$ and $F \setminus f_i$ is the fusion of all views except $f_i$. For minimality, the paper adapts the conditional entropy bottleneck and uses variational inference to obtain $L_{MI} = E_{p(f)p(z|f)}[KL(p(z|f) \| q(z|m))]$, where $q(z|m)$ is obtained by encoding the ground-truth mask into the forgery feature space. These constraints are applied around a feature extractor of three attention U-Nets (channel, spatial, and pixel views) fused by a learnable layer $B_\phi$, and a mask-guided encoder-decoder that produces the final concise feature and an auxiliary noise-perturbed mask prediction.
What would settle it
Train two models identical except that one includes the sufficiency loss and one does not, then estimate $I(M;F)$ directly on a small test set using a neural mutual information estimator; if the model with the loss does not show higher estimated mutual information, the loss is not doing what the theory claims. Alternatively, search for a simple counterexample to the inequality in Eq. (4) by sampling from a toy distribution where the KL term is large but the conditional mutual information is small, which would invalidate the theoretical link.
Extended reading notes
Core claim
The central claim is that imposing two losses on the forgery feature pipeline makes the learned representation both comprehensive and concise. The sufficiency loss, $L_{SU} = \min \exp(-D_{KL}[P_F \| P_{F \setminus f_i}])$, is applied to the feature extraction network and is claimed to lower-bound the task-related mutual information $I(M;F)$, ensuring that no forgery clue is missed. The minimality loss, $L_{MI} = E_{p(f)p(z|f)}[KL(p(z|f) \| q(z|m))]$, is applied to the feature reasoning network and is claimed to remove task-unrelated information by pulling the feature distribution toward a mask-encoded variational distribution. The paper asserts that this information-theoretic grounding yields a latent forgery feature that is both sufficient and minimal, and it reports that the resulting model outperforms seven state-of-the-art methods in in-dataset, cross-dataset, and robustness evaluations.
Load-bearing premise
The load-bearing premise is that the sufficiency loss really does maximize how much the latent feature tells us about the forgery mask; the paper's derivation relies on an inequality about statistical distances that is not generally true.
Editorial extensions
If this is right
- Localization models trained with both constraints should be more robust to JPEG compression and Gaussian blur, because task-unrelated artifacts are explicitly suppressed.
- The sufficiency constraint offers a direct way to combine multiple forensic views without requiring orthogonality, so complementary clues can be pooled.
- The minimality constraint shows how ground-truth masks can guide feature compression through a learned variational mapping, reducing false alarms from unrelated image content.
- The reported cross-dataset results suggest that the constraints improve generalization to unseen tampering types and unseen dataset distributions.
- The two losses are modular and could be added to other forgery localization architectures as auxiliary training objectives.
Reading between the lines
- If the sufficiency loss is replaced by a direct mutual-information estimator on small-scale problems, the performance gap could reveal whether the theoretical bound or the multi-view architecture drives the gain.
- The same two-stage sufficiency/minimality scheme could transfer to other dense prediction tasks, such as anomaly localization or medical image segmentation, where multi-view features are fused and label-guided compression is desirable.
- Because the fusion layer uses a single scalar weight per view, the framework implicitly assumes that one global weighting is enough; per-pixel or per-channel fusion might further improve comprehensiveness.
- The noise-perturbed auxiliary mask training suggests a broader principle: injecting structured noise into the label path can make the variational encoder more robust, a technique that may generalize beyond forgery localization.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes SUMI-IFL, an image forgery localization framework with three attention U-Nets that extract channel, spatial, and pixel features, followed by a learned fusion layer and a feature reasoning network. Two information-theoretic losses are introduced: a sufficiency-view constraint, derived from maximizing the mutual information between the latent forgery feature and the ground-truth mask, and a minimality-view constraint, derived from the conditional entropy bottleneck. The model is trained on a compound dataset and evaluated with in-dataset, cross-dataset, and robustness experiments, reporting substantial gains over several state-of-the-art baselines.
Significance. If the theoretical derivation were valid, the paper would offer a principled way to constrain multi-view forgery features for localization, and the empirical results are indeed strong: Table 2 shows F1 improvements of roughly 8-15 percentage points over the best baselines on DEFACTO-12 and SSRGFD, Table 3 shows a 9.7-point F1 gain on NIST16, and the ablation study in Table 4 indicates that both proposed losses contribute to the final performance. The paper also evaluates generalization and robustness, which is a useful strength. However, the central theoretical claim is not supported as written: the derivation of the sufficiency loss rests on a false inequality and on an appendix proof that proves a different statement. The significance of the contribution is therefore conditional on either correcting the derivation or repositioning the losses as heuristic components within an empirically strong system.
major comments (4)
- [Method, 'Sufficiency-view constraint', Eq. (3)] The inequality in Eq. (3), I(M; f1,...,fn) = sum_i I(fi; M | f1,...,f_{i-1}) <= sum_i I(fi; M | F\fi), is not a consequence of mutual information monotonicity. Conditional mutual information is not monotone under arbitrary enlargement of the conditioning set; for example, if M, f1, and f2 are identical fair bits, the left-hand side is 1 bit while the right-hand side is 0. Since the three backbones process the same input image and no independence or diversity assumption is stated, this step is unsubstantiated and it is the load-bearing bridge from the mutual information objective to the sufficiency loss.
- [Method, Eq. (4); Appendix, Eqs. (21)-(24)] Eq. (4) is asserted with the proof deferred to the appendix, but the appendix does not prove it. The appendix's Eq. (23) bounds the conditional mutual information by the joint KL divergence DKL[p(M, Fi) || p(M, F\fi)] plus a nonnegative conditional-entropy term, whereas Eq. (5) and Eq. (24) implement exp(-DKL[PF || PF\fi]) with PF = p(y|F) and PF\fi = p(y|F\fi). The joint KL and the conditional KL are different quantities, and no argument shows that the joint-KL bound implies the conditional-KL inequality in Eq. (4). Thus the main text does not establish that minimizing exp(-DKL[PF || PF\fi]) maximizes I(M; F).
- [Appendix, Eqs. (18)-(22)] The appendix derivation contains algebra and notation errors that prevent it from repairing the main text. Eq. (18) writes the conditional mutual information as a sum over p(M, Fi) while retaining p(F) and p(M, F\fi), conflating distinct marginals; the derivation of Q1 in Eqs. (19)-(20) attaches H(fi | F\fi) to a sum over M without the required joint distribution; and Eq. (21) defines Q2 as a joint KL that is not the conditional KL appearing in the implemented loss. As written, the supplementary proof is not internally consistent.
- [Method, Eq. (5)] Even if Eq. (4) were established, the loss LSU = min exp(-DKL[PF || PF\fi]) encourages each individual feature fi to alter the posterior label distribution, which is a non-redundancy or necessity property rather than an obvious comprehensiveness property of the fused feature F. The paper does not explain why maximizing the posterior shift caused by removing fi guarantees that F contains maximal task-related information; this conceptual gap should be addressed or the claims should be tempered.
minor comments (5)
- [Method, Overview] The text reads 'The sufficient-view constraint'; this should be 'sufficiency-view' for consistency with the rest of the paper.
- [Table 3 and 'Cross-dataset comparisons'] The claim that SUMI-IFL 'consistently ranks among the top two' is not supported on the CIMD AUC row, where WSCL achieves 0.6273 and SUMI-IFL achieves 0.5513, placing SUMI-IFL third on that metric.
- [Ablation study, Table 4] The sentence 'resulting in an F1 increase of 9.8% and 5.1% on DEFACTO-12 and SSRGFD, respectively' does not match the table: removing LSU drops F1 from 0.9249 to 0.8335 on DEFACTO-12 and from 0.7995 to 0.6783 on SSRGFD, while removing LMI drops F1 from 0.9249 to 0.8774 on DEFACTO-12 and from 0.7995 to 0.6784 on SSRGFD; the reported percentages should be reconciled.
- [Method, Eq. (9)] The expectation notation E_{p(f,m)p(z|m)} in Eq. (9) appears to be a typo; the subsequent equations use E_{p(f)p(z|f)}, which is presumably the intended measure.
- [Author block] The email address for the second affiliation appears garbled as 'luoxy ieu@sina.comthird'.
Circularity Check
No significant circularity: empirical benchmarks are external, and the theoretical gaps are correctness concerns rather than self-referential reductions.
full rationale
The paper's central empirical claim is that the proposed SUMI-IFL framework outperforms prior methods on in-dataset and cross-dataset benchmarks. That claim is grounded in comparisons against external methods (MMFusion, EITL-Net, HiFi-IFDL, WSCL, IF-OSN, MVSS-Net, PSCC-Net) on standard datasets (DEFACTO-12, SSRGFD, CASIAv2, CIMD, NIST16), so the performance evidence is independently falsifiable and not circular. The sufficiency-view and minimality-view losses are training objectives derived from information-theoretic considerations; they are not fitted parameters renamed as predictions, and the reported gains are not guaranteed by the loss definitions alone. The theoretical derivation in Eqs. (3)-(5) and the appendix contains an unjustified inequality and a mismatch between the conditional KL used in Eq. (5) and the joint KL derived in Eq. (23). However, these are mathematical correctness/soundness concerns, not instances of a result being equivalent to its input by construction. The only self-citation (Sheng, Yin, and Lu 2025) appears in the related-work section and is not load-bearing for any derivation or experimental claim. No uniqueness theorem from the authors is invoked, and no ansatz is smuggled in via self-citation. Therefore no circular step is present; the appropriate score is 0.
Assumptions & free parameters
free parameters (5)
- lambda_1 =
0.1
- lambda_2 =
1
- lambda_3 =
0.1
- gamma =
not specified
- gamma_phi =
not specified
assumptions (4)
- ad hoc to paper The inequality sum_i I(fi; M | F\fi) >= sum_i DKL[PF || PF\fi] holds for the distributions defined in Eqs. (3)-(4).
- standard math The cross-entropy loss L_CE is a valid variational lower bound for I(Z;M) in the CEB objective.
- domain assumption The predicted distributions p(y|F) and p(y|F\fi) are accurate enough to estimate the KL divergence in the sufficiency loss.
- domain assumption The ground-truth mask M is available during training and can be mapped to the feature space via the mask encoder q(z|m).
Cite this review
Pith. "Pith review of SUMI-IFL: An Information-Theoretic Framework for Image Forgery Localization with Sufficiency and Minimality Constraints." pith.science (2026). https://pith.science/paper/P3UOKYQ4
@misc{pith2026241209981,
author = {Pith},
title = {Pith review of: SUMI-IFL: An Information-Theoretic Framework for Image Forgery Localization with Sufficiency and Minimality Constraints},
year = {2026},
howpublished = {\url{https://pith.science/paper/P3UOKYQ4}},
note = {Machine review of arXiv:2412.09981}
}
read the original abstract
Image forgery localization (IFL) is a crucial technique for preventing tampered image misuse and protecting social safety. However, due to the rapid development of image tampering technologies, extracting more comprehensive and accurate forgery clues remains an urgent challenge. To address these challenges, we introduce a novel information-theoretic IFL framework named SUMI-IFL that imposes sufficiency-view and minimality-view constraints on forgery feature representation. First, grounded in the theoretical analysis of mutual information, the sufficiency-view constraint is enforced on the feature extraction network to ensure that the latent forgery feature contains comprehensive forgery clues. Considering that forgery clues obtained from a single aspect alone may be incomplete, we construct the latent forgery feature by integrating several individual forgery features from multiple perspectives. Second, based on the information bottleneck, the minimality-view constraint is imposed on the feature reasoning network to achieve an accurate and concise forgery feature representation that counters the interference of task-unrelated features. Extensive experiments show the superior performance of SUMI-IFL to existing state-of-the-art methods, not only on in-dataset comparisons but also on cross-dataset comparisons.
Figures
Figures from the paper (5 more)
Reference graph
Works this paper leans on
-
[1]
Alemi, A. A.; Fischer, I.; Dillon, J. V.; and Murphy, K. 2016. Deep variational information bottleneck. arXiv preprint arXiv:1612.00410
arXiv 2016
-
[2]
Ba, Z.; Liu, Q.; Liu, Z.; Wu, S.; Lin, F.; Lu, L.; and Ren, K. 2024. Exposing the deception: Uncovering more forgery clues for deepfake detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 719--728
2024
-
[3]
Cui, C.; Ren, Y.; Pu, J.; Li, J.; Pu, X.; Wu, T.; Shi, Y.; and He, L. 2024. A novel approach for effective multi-view clustering with information-theoretic perspective. Advances in Neural Information Processing Systems, 36
work page 2024
-
[4]
Dong, C.; Chen, X.; Hu, R.; Cao, J.; and Li, X. 2022. MVSS-Net : Multi-view multi-scale supervised networks for image manipulation detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(3): 3539--3553
work page 2022
-
[5]
Dong, J.; Wang, W.; and Tan, T. 2013. Casia image tampering detection evaluation database. In IEEE China Summit and International Conference on Signal and Information Processing, 422--426
work page 2013
-
[6]
Fischer, I. 2020. The conditional entropy bottleneck. Entropy, 22(9): 999
work page 2020
-
[7]
N.; Delgado, A.; Zhou, D.; Kheyrkhah, T.; Smith, J.; and Fiscus, J
Guan, H.; Kozak, M.; Robertson, E.; Lee, Y.; Yates, A. N.; Delgado, A.; Zhou, D.; Kheyrkhah, T.; Smith, J.; and Fiscus, J. 2019. MFC datasets: Large-scale benchmark datasets for media forensic challenge evaluation. In IEEE Winter Applications of Computer Vision Workshops, 63--72
work page 2019
-
[8]
Guo, K.; Zhu, H.; and Cao, G. 2024. Effective image tampering localization via enhanced transformer and co-attention fusion. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 4895--4899
work page 2024
Show all 38 references
-
[9]
Guo, X.; Liu, X.; Ren, Z.; Grosz, S.; Masi, I.; and Liu, X. 2023. Hierarchical fine-grained image forgery detection and localization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3155--3165
2023
-
[10]
Hu, X.; Zhang, Z.; Jiang, Z.; Chaudhuri, S.; Yang, Z.; and Nevatia, R. 2020. SPAN: Spatial pyramid attention network for image manipulation localization. In The 16th European Conference on Computer Vision, 312--328
2020
-
[11]
Kuang, H.; Liu, H.; Wu, Y.; Satoh, S.; and Ji, R. 2024. Improving adversarial robustness via information bottleneck distillation. Advances in Neural Information Processing Systems, 36
2024
-
[12]
Kwon, M.-J.; Nam, S.-H.; Yu, I.-J.; Lee, H.-K.; and Kim, C. 2022 a . Learning jpeg compression artifacts for image manipulation detection and localization. International Journal of Computer Vision, 130(8): 1875--1895
2022
-
[13]
Kwon, M.-J.; Nam, S.-H.; Yu, I.-J.; Lee, H.-K.; and Kim, C. 2022 b . Learning jpeg compression artifacts for image manipulation detection and localization. International Journal of Computer Vision, 130(8): 1875--1895
2022
-
[14]
Li, D.; Zhu, J.; Wang, M.; Liu, J.; Fu, X.; and Zha, Z.-J. 2023 a . Edge-aware regional message passing controller for image forgery localization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8222--8232
2023
-
[15]
M.; and Zhang, L
Li, F.; Zhang, H.; Liu, S.; Guo, J.; Ni, L. M.; and Zhang, L. 2022. Dn-DETR : Accelerate detr training by introducing query denoising. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 13619--13627
2022
-
[16]
Li, H.; Zhu, C.; Zhang, Y.; Sun, Y.; Shui, Z.; Kuang, W.; Zheng, S.; and Yang, L. 2023 b . Task-specific fine-tuning via variational information bottleneck for weakly-supervised pathology whole slide image classification. In Proceedings of the IEEE/CVF Conference on Computer V...
2023
-
[17]
Lin, T.-Y.; Maire, M.; Belongie, S.; Hays, J.; Perona, P.; Ramanan, D.; Doll \'a r, P.; and Zitnick, C. L. 2014. Microsoft coco: Common objects in context. In The 13th European Conference on Computer Vision, 740--755
2014
-
[18]
Liu, H.; Tan, Z.; Tan, C.; Wei, Y.; Wang, J.; and Zhao, Y. 2024. Forgery-aware adaptive transformer for generalizable synthetic image detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 10770--10780
2024
-
[19]
Liu, X.; Liu, Y.; Chen, J.; and Liu, X. 2022. PSCC-Net : Progressive spatio-channel correlation network for image manipulation detection and localization. IEEE Transactions on Circuits and Systems for Video Technology, 32(11): 7505--7517
2022
-
[20]
Luo, J.; Liu, L.; Xu, W.; Yin, Q.; Lin, C.; Liu, H.; and Lu, W. 2022. Stereo super-resolution images detection based on multi-scale feature extraction and hierarchical feature fusion. Gene Expression Patterns, 45: 119266
2022
-
[21]
L.; and Pic, M
Mahfoudi, G.; Tajini, B.; Retraint, F.; Morain-Nicolier, F.; Dugelay, J. L.; and Pic, M. 2019. DEFACTO : Image and Face Manipulation Dataset. In 27th European Signal Processing Conference, 1--5
2019
-
[22]
Poole, B.; Ozair, S.; Van Den Oord, A.; Alemi, A.; and Tucker, G. 2019. On variational bounds of mutual information. In International Conference on Machine Learning, 5171--5180
2019
-
[23]
Rao, Y.; Ni, J.; Zhang, W.; and Huang, J. 2022. Towards JPEG-Resistant Image Forgery Detection and Localization Via Self-Supervised Domain Adaptation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1--12
2022
-
[24]
Ronneberger, O.; Fischer, P.; and Brox, T. 2015. U-Net : Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted intervention, 234--241
2015
-
[25]
Sheng, Z.; Yin, C.; and Lu, W. 2025. Exploring multi-scale forgery clues for stereo super-resolution image forgery localization. Pattern Recognition, 161: 111230
2025
-
[26]
Sun, Z.; Jiang, H.; Wang, D.; Li, X.; and Cao, J. 2023. SAFL-Net : Semantic-agnostic feature learning network with auxiliary plugins for image manipulation detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 22424--22433
2023
-
[27]
C.; and Bialek, W
Tishby, N.; Pereira, F. C.; and Bialek, W. 2000. The information bottleneck method. arXiv preprint physics/0004057
2000 arXiv
-
[28]
Triaridis, K.; and Mezaris, V. 2024. Exploring multi-modal fusion for image manipulation detection and localization. In International Conference on Multimedia Modeling, 198--211
2024
-
[29]
Wang, H.; Wang, J.; Hu, X.; Hu, B.; Yin, Q.; Luo, X.; Ma, B.; and Sun, J. 2024. Detecting Double Mixed Compressed Images Based on Quaternion Convolutional Neural Network. Chinese Journal of Electronics, 33(3): 657--671
2024
-
[30]
Wang, J.; Wu, Z.; Chen, J.; Han, X.; Shrivastava, A.; Lim, S.-N.; and Jiang, Y.-G. 2022. Objectformer for image manipulation detection and localization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2364--2373
2022
-
[31]
Wu, H.; Zhou, J.; Tian, J.; Liu, J.; and Qiao, Y. 2022. Robust image forgery detection against transmission over online social networks. IEEE Transactions on Information Forensics and Security, 17: 443--456
2022
-
[32]
Yin, C.; Zhang, B.; Luo, J.; Zhu, C.; and Lu, W. 2023. SSRGFD : stereo super-resolution general forensic dataset. Journal of Image and Graphics, 28(11): 3386--3399
2023
-
[33]
Zhai, Y.; Luan, T.; Doermann, D.; and Yuan, J. 2023. Towards generic image manipulation detection with weakly-supervised self-consistency learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 22390--22400
2023
-
[34]
Zhang, L.; Xu, M.; Li, D.; Du, J.; and Wang, R. 2024. CatmullRom Splines-Based Regression for Image Forgery Localization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 7196--7204
2024
-
[35]
Zhang, Z.; Li, M.; and Chang, M.-C. 2024. A New Benchmark and Model for Challenging Image Manipulation Detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 7405--7413
2024
-
[36]
Zhuo, L.; Tan, S.; Li, B.; and Huang, J. 2022. Self-adversarial training incorporating forgery attention for image forgery localization. IEEE Transactions on Information Forensics and Security, 17: 819--834
2022
-
[37]
, " * write output.state after.block = add.period write newline
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...
-
[38]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.