REVIEW 3 major objections 4 minor 33 references
Now You See the Hate: Adaptive View Retrieval for Hidden Hateful Illusions
T0 review · 3 major / 4 minor · reviewed 2026-08-01 · deepseek-v4-flash
Pith's one-line read Hate hidden inside optical illusions can be reliably detected by adaptively retrieving the concealed message from a bank of known templates, rather than by classifying the original view.
desk verdict Closed-set result is solid and well-analyzed, but the paper never quantifies the admitted drop on unseen symbol families, which is the number that matters. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is Adaptive View Retrieval, a retrieve-and-calibrate architecture. Its core object is a complementary view bank: seven deterministic image operators (original, contrast-enhanced, closure-edge, three figure-ground variants, and low-pass) applied identically to the query and every template, so retrieval compares matched views in a shared embedding space. A frozen CLIP encoder produces view embeddings, branch MLPs project them, and a learned gate reallocates trust from view to view depending on target type and visibility. Template scoring is a temperature-scaled, gated sum of cosine similarities, and a separate calibration MLP reads the gated fusion to produce the har
What would settle it
Held-out discovery test: take the HatefulIllusion test set, remove all templates belonging to one hate-symbol family (e.g., all variants of one symbol) from the bank, and measure balanced accuracy on those images. If accuracy collapses or those images get mislabeled as a different known template, the closed-set dependence is confirmed; if the system flags them as uncertain or unknown, the retrieval can in principle support open-set moderation. A second observation: report the exact 'stricter target holdout' number the authors acknowledge but do not provide.
Extended reading notes
Core claim
The paper's central claim is that hidden hateful illusions can be detected by reframing detection as perceptual retrieval. Instead of a K-way label head on the original image, the authors maintain a hidden-message template bank and retrieve the message identity through matched-view comparisons: the same set of seven classical perceptual views is generated for the input and for every template, a learned gate weights the views per image, and a frozen CLIP encoder embeds each view so retrieval can compare aligned evidence. The retrieved identity then feeds a calibration head that decides whether the evidence is harmful. The paper reports 93.2% balanced accuracy (mean ±3.1 over three seeds) on t
Load-bearing premise
The load-bearing premise is that every hidden message at test time has a reference template in the bank: retrieval picks the best-scoring known template, so an unseen hate string or an unseen symbol family is at best labeled as its closest known neighbor, and the paper explicitly notes performance drops under stricter target holdout on unseen symbol families.
Editorial extensions
If this is right
- Platforms that maintain template libraries of known slurs and symbols could add new threats without retraining the detector, and could audit a decision by inspecting which view and which template drove it.
- No single classical transform is a sufficient safety filter; only a complementary bank with learned gating keeps performance stable on hard low-visibility and benign look-alike cases.
- Retrieval-only training collapses moderation to 47.8%, so any practical deployment needs joint identity-retrieval and harm-calibration training.
- The same view bank transfers to non-hate hidden content (digits, fashion, animals, text), so the mechanism is a general perceptual-recovery module rather than a hate-specific classifier.
- On HC-Bench, multi-view questioning recovers hidden text and objects where resolution-losing zoom-out fails (98.2% vs 2.7%), suggesting adaptive multi-view input can replace or supplement downsampling in VLM pipelines.
Reading between the lines
- Editorial inference: Because retrieval is argmax over a fixed bank, the method's practical ceiling is template coverage; a natural extension is a rejection or 'unknown message' option based on retrieval confidence or a threshold on the top similarity score, which the paper does not implement.
- Editorial inference: The gate's visibility-dependent reallocation (more low-pass weight when signals are faint) suggests visibility labels could be used as weak supervision or as a conditioning variable, potentially improving low-visibility recall further.
- Editorial inference: The same matched-view bank could be tested on adversarial evasions—images crafted specifically to steer the gate toward a misleading view—since the failure analysis already shows deterministic figure-ground segmentation can break on textured carriers.
- Editorial inference: The HC-Bench result with a small 2B VLM raises a testable hypothesis: multiplicative gains may come from giving VLMs multiple observation strategies rather than trading resolution for zoom; a direct comparison on a much larger VLM would clarify whether the view bank is a substitute for scale or only a fallback when scale is unavailable.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Adaptive View Retrieval, a retrieve-and-calibrate framework for detecting hidden hateful messages in optical illusions. The method builds a seven-view perceptual bank (original, contrast, closure-edge, three figure-ground, low-pass), applies the same transforms to query images and to templates in a known-message library, embeds all views with a frozen CLIP encoder, uses an adaptive gate to weight views, retrieves the best-matching template, and calibrates a harmful/benign decision. On HatefulIllusion the authors report 93.2% balanced accuracy under a message-level held-out split, with ablations showing the view bank, adaptive gating, and joint retrieval/calibration training all contribute. The same view bank is transferred to IllusoryVQA (fine-tuned CLIP) and to HC-Bench (VLM multi-view questioning), where it outperforms single-view baselines. The paper includes component ablations, hyperparameter sensitivity, error analysis, and a public code release.
Significance. If the closed-set results are taken at face value, the paper demonstrates a substantial improvement over original-view classifiers and fixed-filter baselines on the HatefulIllusion benchmark, and the adaptive gate's dependence on target type and visibility is an interesting and interpretable finding. The component ablations (original view only 79.5%, uniform gate 89.1%, full model 93.2%) and the three-seed protocol support the central claim within the benchmark. The release of code and the explicit error analysis are also strengths. However, the practical significance is limited by the closed-set retrieval setting: the template bank contains a reference for every test message identity, and the paper acknowledges but does not quantify the drop under target holdout on unseen symbol families. The HC-Bench result is also scored as a best-of-seven oracle, which overstates the adaptive nature of the method. These concerns are addressable but are load-bearing for the generalization claims.
major comments (3)
- [Discussion, Implications and Limitations; Eq. (1)] The method's decision rule is argmax over template bank M (Eq. 1), and the authors state that performance 'also drops under stricter target holdout on unseen symbol families' but never report the magnitude of that drop. Because the headline 93.2% is a closed-set retrieval result—every test message has a template in M—the practical benefit for real-world moderation, where new slangs and symbols emerge, is unquantified. I request a target-holdout experiment: train templates exclude an entire symbol family (or several), and report balanced accuracy on those unseen families. This is needed to distinguish graceful degradation from collapse and directly addresses the paper's stated limitation.
- [Experiment 3, Table 2] The HC-Bench result is scored as 'counted correct if any transformed view yields a valid answer under the official rules.' This is a best-of-seven oracle, not an adaptive decision, and the comparison to original image or zoom-out (single views) is therefore not apples-to-apples. The reported 98.2% is an upper bound. Please also report per-view accuracies, and ideally an adaptive view-selection accuracy using the gate or a similar mechanism. Otherwise the claim that 'Adaptive View Retrieval' outperforms zoom-out on HC-Bench is overstated.
- [Training and inference; Supplementary Table 2] The hyperparameter sweep in Supplementary Table 2 reports balanced accuracy on the test split. If the default hyperparameters were selected using test-split accuracy, the 93.2% is not a clean held-out estimate. Please clarify whether the sweep was conducted on the validation split or the test split. If the test split was used, rerun the sweep on the validation split (or report validation accuracy) and confirm the default remains the same.
minor comments (4)
- [Method, Eq. (1)] The temperature τ in Eq. (1) is not defined. Is it a learned scalar initialized to some value? Please specify; this matters for the scale of the retrieval scores.
- [Experiment 2, Table 1] The IllusoryVQA results are reported as single point estimates without variance across seeds. Since Experiment 1 uses three seeds, please add mean±std for Table 1 as well, particularly because the method uses full fine-tuning there.
- [Supplementary Material] The main text references 'Supplementary Table 1' and 'Supplementary Figure 3,' but the supplementary file labels the relevant items as Table 3 and Figure 10. Please renumber consistently.
- [Experiment 3] The HC-Bench experiment uses a fixed VLM and multi-view questioning, but does not use the gate or calibration head from Experiments 1–2. Please clarify in the text that this is a view-bank ablation under the SemVink protocol, not the full Adaptive View Retrieval architecture.
Circularity Check
No circularity: the 93.2% result is a supervised, closed-gallery retrieval evaluation and the paper explicitly disclaims open-vocabulary generalization.
full rationale
The derivation chain is self-contained. The retrieval score S_j(x)=τ∑_k α_k(x)⟨z_x^k,z_j^k⟩ and decision m_hat=argmax_{m_j∈M} S_j(x) are defined over a fixed template bank; the calibration head d(x) is a separately trained binary head. Each component is trained with cross-entropy and BCE losses on the train split and evaluated on a held-out message-level test split over three seeds, so no fitted quantity is renamed as a prediction. The template bank is a candidate gallery, and the paper explicitly acknowledges it is not open-vocabulary ('not open-vocabulary discovery of unseen hate strings'), so the closed-set nature is disclosed rather than hidden. There are no author self-citations in the reference list; the benchmarks cited (Qu et al. 2025; Rostamkhani et al. 2025; Li et al. 2025) are external. The one-at-a-time hyperparameter sweep reported on the test split is an evaluation-hygiene concern (tuning on test), not a circular reduction. Experiment 3's 'any transformed view yields a valid answer' scoring is a lenient oracle but is an explicit protocol choice, not an equivalence to the method's inputs. The paper's limitation statement that performance drops on unseen symbol families is a scope restriction; it lowers external validity but does not make the reported result equivalent to its inputs by construction. Verdict: no significant circularity.
Assumptions & free parameters
free parameters (5)
- temperature τ =
learned, not reported
- detection loss weight λ_det =
0.5
- gate entropy weight λ_ent =
0.01
- learning rate / epochs =
1e-3 / 10
- view-bank transform constants =
unspecified
assumptions (5)
- domain assumption Frozen CLIP ViT-B/32 embeddings of matched views are semantically aligned for retrieval.
- domain assumption The seven deterministic transforms reveal enough hidden structure for all target types and carriers.
- domain assumption The template bank M covers all test message identities.
- domain assumption HatefulIllusion labels and visibility annotations are correct.
- standard math Otsu thresholding, Gaussian blur, and morphological operators behave as standard computer-vision primitives.
Cite this review
Pith. "Pith review of Now You See the Hate: Adaptive View Retrieval for Hidden Hateful Illusions." pith.science (2026). https://pith.science/paper/NAKTUR42
@misc{pith2026260719061,
author = {Pith},
title = {Pith review of: Now You See the Hate: Adaptive View Retrieval for Hidden Hateful Illusions},
year = {2026},
howpublished = {\url{https://pith.science/paper/NAKTUR42}},
note = {Machine review of arXiv:2607.19061}
}
read the original abstract
Hateful optical illusions expose a serious gap in current multimodal safety systems. On original-view hateful illusions, previous work shows that six moderation classifiers achieve at most 20.9 to 24.5% accuracy and nine state-of-the-art VLMs remain at or below 10.2% with illusion-aware prompting, leaving most hidden hate undetected. We formulate hidden hateful illusion detection as a perceptual retrieval problem and propose Adaptive View Retrieval. This retrieve-and-calibrate framework assembles a complementary view bank for the image and hidden-message templates, adaptively selects which views to trust, retrieves hidden-message identities, and calibrates whether the recovered evidence is harmful. On HatefulIllusion with a frozen CLIP encoder, Adaptive View Retrieval reaches 93.2% balanced accuracy on the held-out test split. It substantially outperforms original-view baselines and fixed single-transform filters across hate slangs, hate symbols, and visibility levels. The same design also surpasses official fine-tuned CLIP baselines, matches or exceeds human performance on IllusionMNIST, IllusionFashionMNIST, and IllusionAnimals, and outperforms zoom-out preprocessing on HC-Bench under the SemVink protocol. Together, these results show that robust multimodal moderation requires recovering hidden meaning before deciding whether it is harmful.
Figures
Figures from the paper (8 more)
Reference graph
Works this paper leans on
-
[1]
ACM SIGGRAPH 2024 Conference Papers , pages=
Diffusion illusions: Hiding images in plain sight , author=. ACM SIGGRAPH 2024 Conference Papers , pages=
2024
-
[2]
Proceedings of the ACM on Web Conference 2025 , pages=
Illusioncaptcha: A captcha based on visual illusion , author=. Proceedings of the ACM on Web Conference 2025 , pages=
2025
-
[3]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
Visual anagrams: Generating multi-view optical illusions with diffusion models , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
-
[4]
Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=
Convolutional neural networks can be deceived by visual illusions , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=
-
[5]
Vision Research , volume=
Color illusions also deceive CNNs for low-level vision tasks: Analysis and implications , author=. Vision Research , volume=. 2020 , publisher=
2020
-
[6]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
The art of deception: Color visual illusions and diffusion models , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
-
[7]
Guan, Tianrui and Liu, Fuxiao and Wu, Xiyang and Xian, Ruiqi and Li, Zongxia and Liu, Xiaoyu and Wang, Xijun and Chen, Lichang and Huang, Furong and Yacoob, Yaser and others , booktitle=
-
[8]
Advances in Neural Information Processing Systems , volume=
Hidden in plain sight: Evaluating abstract shape recognition in vision-language models , author=. Advances in Neural Information Processing Systems , volume=
Show all 33 references
-
[9]
arXiv preprint arXiv:2603.24730 , year=
A Framework for Generating Semantically Ambiguous Images to Probe Human and Machine Perception , author=. arXiv preprint arXiv:2603.24730 , year=
-
[10]
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=
SemVink: Advancing VLMs’ Semantic Understanding of Optical Illusions via Visual Global Thinking , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=
2025
-
[11]
Stevens' handbook of experimental psychology: Sensation and perception (vol
Perceptual organization in vision , author=. Stevens' handbook of experimental psychology: Sensation and perception (vol. 1, 3rd edn., pp. 177–234), John Wiley and Sons Inc, Hoboken, NJ, USA , year=
-
[12]
Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics , pages=
Evaluating vision-language models on bistable images , author=. Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics , pages=
-
[13]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
Do vision models perceive illusory motion in static images like humans? , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
-
[14]
Proceedings of the Computer Vision and Pattern Recognition Conference , pages=
Illusory VQA: Benchmarking and enhancing multimodal models on visual Illusions , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=
-
[15]
Shahgir, Haz Sameen and Sayeed, Khondker Salman and Bhattacharjee, Abhik and Ahmad, Wasi Uddin and Dong, Yue and Shahriyar, Rifat , journal=
-
[16]
arXiv preprint arXiv:2506.05765 , year=
Do Large Vision-Language Models Distinguish between the Actual and Apparent Features of Illusions? , author=. arXiv preprint arXiv:2506.05765 , year=
-
[17]
Sun, Xiaoxiao and Li, Mingyang and Yuan, Kun and Sun, Min Woo and Endo, Mark and Wu, Shengguang and Li, Changlin and Zhang, Yuhui and Wang, Zeyu and Yeung-Levy, Serena , booktitle=. Do
-
[18]
Perception , volume=
What are visual illusions? , author=. Perception , volume=. 2020 , publisher=
2020
-
[19]
arXiv preprint arXiv:2412.18613 , year=
The illusion-illusion: Vision language models see illusions where there are none , author=. arXiv preprint arXiv:2412.18613 , year=
-
[20]
Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=
Hate in Plain Sight: On the Risks of Moderating AI-Generated Hateful Illusions , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=
-
[21]
Illusions in humans and
Yang, Jianyi and Ye, Junyi and Dash, Ankan and Wang, Guiling , journal=. Illusions in humans and
-
[22]
Proceedings of the 2023 conference on empirical methods in natural language processing , pages=
Grounding visual illusions in language: Do vision-language models perceive illusions like humans? , author=. Proceedings of the 2023 conference on empirical methods in natural language processing , pages=
2023
-
[23]
2025 , organization=
Zhang, Yiming and Zhang, Zicheng and Wei, Xinyi and Liu, Xiaohong and Zhai, Guangtao and Min, Xiongkuo , booktitle=. 2025 , organization=
2025
-
[24]
2026 , eprint=
A Survey of Multimodal Hallucination Evaluation and Detection , author=. 2026 , eprint=
2026
-
[25]
2026 , eprint=
Beyond Hate: Differentiating Uncivil and Intolerant Speech in Multimodal Content Moderation , author=. 2026 , eprint=
2026
-
[26]
SURE : Safety Understanding and Reasoning Enhancement for Multimodal Large Language Models
Gou, Yuxin and Dong, Xiaoning and Li, Qin and Gu, Shishen and Hong, Richang and Hu, Wenbo. SURE : Safety Understanding and Reasoning Enhancement for Multimodal Large Language Models. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025. ...
2025 doi
-
[27]
2021 , journal=
Intriguing Properties of Vision Transformers , author=. 2021 , journal=
2021
-
[28]
Stochastic relativistic advection diffusion equation from the Metropolis algorithm , volume=
Başar, Gökçe and Bhambure, Jay and Singh, Rajeev and Teaney, Derek , year=. Stochastic relativistic advection diffusion equation from the Metropolis algorithm , volume=. Physical Review C , publisher=. doi:10.1103/physrevc.110.044903 , number=
-
[29]
2021 , eprint=
Better Aggregation in Test-Time Augmentation , author=. 2021 , eprint=
2021
-
[30]
Proceedings of the IEEE/CVF international conference on computer vision , pages=
Better aggregation in test-time augmentation , author=. Proceedings of the IEEE/CVF international conference on computer vision , pages=
-
[31]
2022 , eprint=
Trion gas, electron-hole liquid, and metal-insulator transition in doped heterostructures based on transition metal dichalcogenides , author=. 2022 , eprint=
2022
-
[32]
Perceptual grouping and figure--ground organization
A century of Gestalt psychology in visual perception: I. Perceptual grouping and figure--ground organization. , author=. Psychological bulletin , volume=. 2012 , publisher=
2012
-
[33]
, author=
Laws of organization in perceptual forms. , author=. 1938 , journal=
1938
Reviewed August 1, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.