{"id":"71c6d061-1139-4438-95a4-5eb9e3bd8191","arxiv_id":"2412.10122","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"DDIM inversion in diffusion models produces brightness and color shifts that track human visual illusions, and a diffusion-based optimizer can generate new illusions in realistic images that fool human observers.","lead":"Diffusion models, which generate images by removing noise, also shift brightness and color in their intermediate steps in ways that match how humans see classic visual illusions. The authors use this to generate new realistic images with embedded illusion targets that human observers misjudge.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. 8's absolute-change criterion is direction-blind and degenerate: identity output would score perfect PAS, so Table 1 does not establish illusion prediction.","rationale":"The reader's weakest_assumption is precisely the load-bearing concern: Eq. 8 uses mean absolute intensity change, which cannot encode the direction of the percept, and the accompanying text is internally inconsistent about what lower tau means. My read of the manuscript confirms this is the single most serious threat to the central claim. The strongest claim—that intermediate denoising steps introduce shifts aligned with human perception—rests almost entirely on the PAS numbers, and those numbers are generated by a metric that rewards doing nothing as much as it rewards the correct illusion. The generation contribution is backed by a psychophysical experiment with 15 observers and control images, so it is not similarly threatened; that is why I do not recommend rejection. A conditional acceptance requiring a signed metric, the identity control, and released evaluation code is the appropriate outcome, matching the reader's verdict. I found no separate concern that would move the verdict further.","tokens_in":12000,"tokens_out":4048,"duration_ms":42393,"concrete_test":"Recompute the BRI3L rows of Table 1 with a signed, direction-aware score: for each target mask compute d = mean(Iout) - mean(Iin), and count an image as aligned only when d is negative (darkening) with magnitude at least a small epsilon, and optionally require d to exceed the corresponding human-effect threshold. Add the identity model Iout = Iin as a control: under Eq. 8 it should score 100% at every tau, exposing the degeneracy; under the signed metric it should score 0% if no mask darkens. If DDIM's signed PAS falls to near chance or to VAE levels, the illusion-prediction claim is not supported by the current evidence.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The quantitative backbone of the replication claim is Eq. 8: Delta I = mean(|Iout - Iin|) over a target mask, with alignment declared when Delta I < tau * Iint. This metric has two fatal properties. First, it is direction-blind: an output that brightens a target as much as humans darken it receives the same Delta I, so the PAS cannot tell whether DDIM reproduces the human-perceived direction of the illusion. Second, it is degenerate: the identity transform Iout = Iin gives Delta I = 0, which satisfies the condition for every positive tau. Thus a model that simply copies the input would score 100% PAS at all thresholds. The text in Section 4.3 compounds the problem by stating that lowering tau 'requires the model to achieve greater darkness in the target regions'; under Eq. 8 lowering tau requires the output to stay closer to the input, and no change is always acceptable. Because BRI3L masks denote regions of expected darkening, the reported DDIM PAS values (e.g., 84-100% at tau = 0.8-1.0) could be driven by near-identity outputs or by brightening rather than by human-like darkening. The qualitative demonstrations and the psychophysical generation experiment provide independent, plausible evidence, but the central claim that diffusion models predict brightness/color illusions is not established by the current quantitative metric.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper claims that DDIM inversion of diffusion models produces intermediate representations whose brightness/color shifts align with human visual illusion perception, and uses this to (1) predict illusions on BRI3L and VQA datasets and (2) generate new illusions with text-to-image diffusion models, the latter being validated by a psychophysical experiment. The quantitative replication claim rests on Eq. (8), a mean absolute intensity-change metric, and on manually masked VQA evaluations; the generation method optimizes a mean-intensity loss in the latent space.","tokens_in":1452,"tokens_out":1233,"duration_ms":40296,"significance":"If the quantitative replication claim were sound, this would be an interesting and nontrivial observation: diffusion models trained only on natural images might encode low-level perceptual biases, and their latent trajectories could serve as a computational model of brightness/color illusions. The paper has genuine strengths: the qualitative demonstrations in Fig. 2 and 4 are suggestive; the psychophysical experiment in Sec. 5.2 provides independent and credible evidence that the generated illusions fool humans; and no parameter is fitted to the benchmark labels. However, the main quantitative tool used to support the replication claim is direction-blind and degenerate, so the central claim is not currently established.","major_comments":[{"comment":"The metric Delta-I = mean(|I_out - I_in|) over the target mask is direction-blind: an output that brightens the target exactly as much as humans darken it yields the same Delta-I, so the PAS values in Table 1 cannot distinguish human-like darkening from brightening. More seriously, the identity output I_out = I_in gives Delta-I = 0 and satisfies Delta-I < tau * I_int for every positive tau, so a model that simply copies the input would score 100% PAS at all thresholds. The text's statement that lowering tau \"requires the model to achieve greater darkness in the target regions\" is inconsistent with the equation; lowering tau actually requires the output to stay closer to the input, and no change is always acceptable. Because BRI3L masks denote expected darkening regions, the reported DDIM PAS values (e.g., 84-100% at tau = 0.8-1.0) could be driven by near-identity outputs rather than by human-like perceptual shifts. The replication claim therefore needs a signed, direction-aware metric (and ideally a no-change baseline) before Table 1 can be interpreted.","section":"Sec. 4.3, Eq. (8)"},{"comment":"The evaluation on IllusionVQA, GVIL, and HallusionBench is underspecified: the paper states that the authors \"manually create segmentation masks\" and consider the answer aligned \"if it matches the expected intensity\" for the referenced regions, but it does not define how the expected intensity is derived, how ambiguous multiple-choice answers are scored, or how the masks are constructed and validated. Table 2 reports PAS values (e.g., DDIM 5 steps 90.40% on IllusionVQA) without specifying the question-answer matching procedure. Since these numbers are part of the central claim that diffusion models predict color illusions, the protocol must be detailed enough to be reproduced and checked for potential experimenter bias.","section":"Sec. 4.3, VQA-based evaluation"},{"comment":"The psychophysical experiment is a strong point, but the paper does not report statistical significance of the difference between the illusion images (64% mean \"different\") and control images (13% mean \"different\"). A confidence interval or hypothesis test over the 15 observers would strengthen the claim that the generation method reliably fools humans beyond chance.","section":"Sec. 5.2"}],"minor_comments":[{"comment":"The notation is inconsistent: Eq. (8) uses |M| for the mask size while Eq. (3) uses M; please unify and define the symbols clearly.","section":"Sec. 4.3, Eq. (8)"},{"comment":"The caption says \"Herman-grid\" but the correct spelling is \"Hermann grid\"; also please ensure the reference [16] is formatted consistently.","section":"Fig. 4"},{"comment":"There is a typo \"responsable\" in Sec. 4.2; it should be \"responsible\".","section":"Sec. 4.2"},{"comment":"References [42] and [43] appear to refer to the same BRI3L dataset (one as a 2023 citation and one as a 2024 arXiv preprint); please consolidate them.","section":"References"},{"comment":"The intuitive notion of \"Gaussianization\" of the latent space is used informally; a brief formal definition or a reference to the iterative Gaussianization literature would help readers outside the specific field.","section":"Sec. 3.2"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a topic within the journal's scope, and the generation part is genuinely novel and supported by psychophysical evidence. The replication claim, however, relies on a metric that is direction-blind and trivially satisfied by identity outputs, so the main quantitative tables cannot be accepted as they stand. I believe a revision that corrects Eq. (8) to a signed, region-specific metric and re-runs the benchmarks, and that specifies the VQA evaluation protocol, would be within the authors' capability and the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know: the paper's central quantitative claim about diffusion models predicting visual illusions is not established by its own metric. The observation itself is novel and worth taking seriously, though.\n\nWhat's new: the authors show that intermediate DDIM inversion states of Stable Diffusion and DeepFloyd introduce brightness/color shifts that align with human illusory perception. This goes beyond prior work that measured perception in classifiers, restoration networks, or likelihood flows. The qualitative demonstrations are compelling, especially the divergence of physically identical patches along the trajectory. The generation method, which optimizes target regions during sampling and validates results with a psychophysical experiment, is a real contribution and looks reproducible.\n\nThe soft spot is the quantitative backbone. Eq. 8 defines Delta I as the mean absolute intensity change in the target mask and declares alignment when Delta I < tau * I_in. That metric is direction-blind: brightening a region that humans darken yields the same Delta I as darkening it. It is also degenerate: the identity transform gives Delta I = 0 and would score 100% PAS at every threshold. The text in Sec. 4.3 says lowering tau requires \"greater darkness,\" but under Eq. 8 it just requires staying closer to the input. So Table 1's numbers could be driven by near-identity outputs or wrong-direction changes, not by human-like prediction. The VQA evaluation is also underspecified: manually created masks, no protocol for determining expected intensity, and no released code or data.\n\nI don't think the idea is sunk. The qualitative evidence is suggestive and consistent, and the psychophysical generation experiment provides independent support, though it shares the same mean-intensity proxy used to score replication, so that part is mildly self-referential. Fixing the metric with a signed, direction-aware measure and releasing the artifacts would likely settle the question.\n\nThe paper deserves a serious referee. It reports a fresh observation with a plausible mechanism and a useful generation method, but it needs major revision before I'd trust the numbers. I'd bring it to a reading group as a maybe—the idea is worth discussing even if the current evidence is weak.","headline":"Novel observation, but the headline metric is direction-blind; the paper needs a signed, direction-aware metric before the replication claim holds.","tokens_in":12777,"tokens_out":2103,"would_cite":false,"duration_ms":22309,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Diffusion models reproduce human brightness and color illusions in their denoising trajectories and can generate new ones.","keywords":["visual illusions","diffusion models","DDIM inversion","brightness perception","color perception","text-to-image generation","psychophysics"],"falsifier":"Compute a signed, direction-aware version of the paper's $\\Delta I$ on a brightness-illusion dataset: instead of $|I_{\\mathrm{out}} - I_{\\mathrm{in}}|$, measure $I_{\\mathrm{out}} - I_{\\mathrm{in}}$ and count only cases where the sign matches the illusion's expected darkening. If the model's predicted shifts are actually direction-specific, this score should stay close to the absolute-value PAS; if it drops substantially, the claim that the trajectory mirrors human perception is not supported by the current metric.","tokens_in":11830,"feed_emoji":"👁️","tokens_out":6118,"duration_ms":62843,"temperature":0.7,"pith_summary":"Diffusion models trained only on natural photographs appear to carry human-like low-level vision in their latent space. The paper tries to establish that the intermediate steps of DDIM inversion — the deterministic path that maps an image back toward noise — progressively shift physically identical patches in the same direction that humans perceive them, so the trajectory itself can be read as a perceptual prediction. The authors then turn this observation into a generation method: by adding a perceptual loss on target regions during text-to-image sampling, they plant brightness/color illusions into realistic generated images, and a psychophysical experiment confirms that human observers see the generated targets as different. If correct, this would mean diffusion priors are a usable model of human perception and a practical tool for creating perceptual effects.","feed_headline":"Diffusion models see color illusions like humans do","feed_subtitle":"Their latent trajectories shift identical patches the way human perception does, enabling new illusions to be generated.","key_machinery":"The load-bearing object is the DDIM inversion trajectory: the deterministic reverse mapping that sends an image $z_0$ to a sequence of latent states $z_t$ via the learned noise predictor. Instead of reading perception from the model's final output, the paper reads the decoded intermediate latents at small step counts, when gaussianization of the input is incomplete. The theoretical lens is that the forward diffusion attempts to Gaussianize images; illusory stimuli lie off the natural-image manifold, so the trajectory shows systematic 'corrections' that mirror human perceptual biases. A second mechanism is the generation loss $L = \\gamma L_{\\mathrm{VI}} + \\beta L_{\\mathrm{sim}}$, which pushes noisy latents so that target regions reach user-specified colors while staying harmonized with the background.","core_discovery":"The central claim is that visual illusions are encoded in the intermediate representations of diffusion models. For an input image $z_0$, DDIM inversion produces a trajectory $z_1, \\dots, z_T$; decoding the early states and measuring mean per-channel intensities inside illusion-relevant regions shows that regions humans judge as darker, lighter, or color-shifted are altered by the inversion in exactly that direction. The paper interprets this as the model mapping an out-of-distribution illusory stimulus toward the natural-image manifold, so the transformation is analogous to human adaptation to natural statistics. It supports the claim by reporting high Perception Accuracy Scores on a brightness-illusion dataset and on three color-illusion visual-question-answering datasets, and by showing that the effect is absent when only the autoencoder is used without the diffusion process. It then uses the same latent readout as a loss to generate new illusions and validates them with human observers.","pith_inferences":["One testable extension is to treat the signed latent drift along the trajectory as a continuous perceptual metric and compare it with human discrimination thresholds; the paper's own metric is unsigned, so it cannot yet separate darkening from brightening.","The same gaussianization argument could be tested on other low-level dimensions, such as contrast sensitivity, spatial frequency, or motion aftereffects, where human biases are also thought to reflect natural-image statistics.","If diffusion priors genuinely mirror human perception, then optimizing images to maximize latent drift could generate adversarial perceptual probes: stimuli that strongly fool humans but not models would reveal exactly where the learned prior diverges from human adaptation."],"forward_implications":["A pretrained diffusion model can act as a vision model: perceptual judgments are obtained by DDIM inversion and read out from intermediate latents, without training or fine-tuning.","The same tool captures brightness and color illusions across laboratory stimuli, published illusion images, and natural photographs, without per-illusion parameter tuning.","Text-to-image generation can embed an illusion into a specified region and control its color, producing images that fool human observers in psychophysical testing.","Because the effect is tied to training on natural-image statistics, the latent shift can serve as a probe for what the model considers out-of-distribution."],"supporting_citations":[{"why":"Defines DDIM inversion, the deterministic trajectory that the paper uses to read out perceptual shifts.","marker":"[46]"},{"why":"Provides Stable Diffusion, the pretrained text-to-image model used for the replication experiments and inversion.","marker":"[41]"},{"why":"Supplies the BRI3L brightness-illusion dataset with 22,366 images and masks, used for quantitative PAS evaluation.","marker":"[42]"},{"why":"Defines the canonical Brightness Contrast stimulus used to motivate and illustrate the observed latent shifts.","marker":"[5]"},{"why":"Supplies the wholly-empirical framework linking perception to natural-image statistics that motivates the interpretation of the shifts.","marker":"[20]"},{"why":"Provides the parameter settings used to calibrate the classical vision models (ODOG, CIWaM) compared in the benchmark.","marker":"[3]"},{"why":"Shows CNNs can be deceived by visual illusions, providing the prior replication baselines DNnet and RestoreNet.","marker":"[12]"},{"why":"One of the color-illusion visual-question-answering datasets used to evaluate illusion replication.","marker":"[44]"},{"why":"Another visual-question-answering dataset, HallusionBench, used as an evaluation set in the replication experiments.","marker":"[15]"}],"fun_headline_variants":["Diffusion models encode human-like color illusions in latent space","Diffusion models predict visual illusions humans see","Diffusion models generate new color illusions that trick humans","Latent diffusion reveals the art of color deception","Diffusion models' latent space mimics human color perception"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The quantitative argument depends on treating the mean absolute intensity change inside a target region as a faithful measure of a human-like illusion, which assumes that any change in the right direction is what the model produces and does not penalize changes in the opposite direction.","fun_headline_variants_meta":{"raw":{"variants":["Diffusion models encode human-like color illusions in latent space","Diffusion models predict visual illusions humans see","Diffusion models generate new color illusions that trick humans","Latent diffusion reveals the art of color deception","Diffusion models' latent space mimics human color perception"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001062,"raw_usage":{"total_tokens":4422,"prompt_tokens":884,"completion_tokens":3538,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":500,"completion_tokens_details":{"reasoning_tokens":3464}},"tokens_in":500,"tokens_out":3538,"duration_ms":25780,"temperature":1.0,"reasoning_tokens":3464,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T16:18:47.673865+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute a signed, direction-aware version of the paper's $\\Delta I$ on a brightness-illusion dataset: instead of $|I_{\\mathrm{out}} - I_{\\mathrm{in}}|$, measure $I_{\\mathrm{out}} - I_{\\mathrm{in}}$ and count only cases where the sign matches the illusion's expected darkening. If the model's predicted shifts are actually direction-specific, this score should stay close to the absolute-value PAS; if it drops substantially, the claim that the trajectory mirrors human perception is not supported by the current metric.","supporting_citations":[{"cited_title":"Bri3l: A brightness illusion image dataset for identification and localization of regions of illusory perception","cited_arxiv_id":null,"evidence_quote":"Supplies the BRI3L brightness-illusion dataset with 22,366 images and masks, used for quantitative PAS evaluation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the canonical Brightness Contrast stimulus used to motivate and illustrate the observed latent shifts."},{"cited_title":"Lightness, brightness and trans- parency: A quarter century of new ideas, captivating demon- strations and unrelenting controversy","cited_arxiv_id":null,"evidence_quote":"Supplies the wholly-empirical framework linking perception to natural-image statistics that motivates the interpretation of the shifts."},{"cited_title":"Noise masking of white’s illusion exposes the weakness of current spatial filtering models of lightness perception","cited_arxiv_id":null,"evidence_quote":"Provides the parameter settings used to calibrate the classical vision models (ODOG, CIWaM) compared in the benchmark."},{"cited_title":"Convolutional neural networks can be deceived by visual illusions","cited_arxiv_id":null,"evidence_quote":"Shows CNNs can be deceived by visual illusions, providing the prior replication baselines DNnet and RestoreNet."},{"cited_title":"Hallusionbench: an advanced diagnos- tic suite for entangled language hallucination and visual il- lusion in large vision-language models","cited_arxiv_id":null,"evidence_quote":"Another visual-question-answering dataset, HallusionBench, used as an evaluation set in the replication experiments."}],"review_version":1}