{"id":"5bc44be7-7bfb-40f6-9678-325d4819293a","arxiv_id":"2505.12660","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"F-SUM, an image-computable score combining foveated vision and vision-language models, correlates with human scene comprehension times (r=0.47) and saccade counts (r=0.51) across 277 images.","lead":"A new image-computable score, F-SUM, predicts how long people need to understand a scene by combining simulated foveated vision with AI-generated scene descriptions. The score correlates with human response times and eye movements better than existing clutter, complexity, or ambiguity measures, suggesting the interaction of foveation with the spatial spread of informative details is a key driver of comprehension difficulty.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eqs. 2-3 as written would assign high F-SUM difficulty scores to easy scenes with uniformly high similarity, contradicting the paper's own examples and the reported positive RT correlation; an unstated inversion is required for the headline results to hold.","rationale":"The reader's weakest_assumption focuses on the VLM proxy for human foveated perception. That concern is legitimate, but the most load-bearing problem is more elementary: as written, the aggregation metric has the wrong sign. Section 2.2 says higher F-SUM entries mean more informative fixations; Section 2.3 says a higher weighted-Ripley score means a more difficult scene. For a scene where all fixations are informative, Eqs. 2-3 therefore produce an especially high difficulty score, yet the paper's easy examples are assigned negative F-SUM z-scores. The only way the reported positive correlations with RT and saccade counts can hold is if the implementation inverts the map (e.g., w = 1 - M) without stating it, or uses a different definition not given in the text. Without code or data, the reader cannot tell which version produced Tables 1-3, so the central claim is not reproducible from the published specification. I would therefore move the verdict from CONDITIONAL to UNVERDICTED: the discrepancy could in principle be a presentation error, but until the authors reconcile the equations or release the implementation, the headline correlations are uninterpretable. The VLM-proxy validation and code release suggested by the reader remain important once the aggregation rule is fixed.","tokens_in":12089,"tokens_out":10530,"duration_ms":118848,"concrete_test":"Reimplement Eqs. 1-3 exactly as written, with no hidden inversion, using the reported fixation grid and foveation parameters, and compute S for the two example scenes in Fig. 4 and ideally all 277 images. If S(easy) > S(hard), or if the Pearson r between S and mean human RT changes sign relative to Table 1, the stated equations are not the metric that produced the results. Then re-run the same pipeline with w_p = 1 - M_{m,n} and compare: if the reported correlations (r = 0.47, 0.51, -0.56) only appear under this unstated inversion, Section 2.3 must be corrected and all tables re-reported. Releasing code and per-image F-SUM maps would settle this directly.","verdict_should_be":"UNVERDICTED","load_bearing_attack":"Section 2.2 defines M_{m,n} as the similarity between foveated and gold-standard VLM descriptions, and states that higher values mean more informative fixations. Section 2.3 then defines K(r) = (1/N) sum_{p≠q} w_p w_q indicator(r-1 ≤ d_pq ≤ r) and S = sum_{r=1}^{R} K(r)/r divided by sum_{r=1}^{R} 1/r, stating that higher S means the scene is more difficult. If w_p and w_q are the entries of M, as the text says, a scene whose informative content is available everywhere (the easy baseball scene in Fig. 1 and top-left of Fig. 4) has large products w_p w_q for nearly all pairs and therefore a large S, i.e., it is predicted to be difficult. The reported F-SUM z-scores assign negative values to exactly such easy scenes, so the implementation must differ from the text (for example, using w_p = 1 - M_{m,n}) without stating it. This is an internal inconsistency in the central construction, not a modeling choice: as specified, Eqs. 2-3 cannot generate the headline correlations in Tables 1-3. The VLM-proxy concern is real but secondary, because even a perfect VLM proxy would not rescue a metric whose aggregation rule has the wrong sign.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces F-SUM, an image-computable metric for predicting how long humans take to comprehend a scene. The metric is constructed by (1) obtaining a 'gold standard' VLM description of the original image, (2) generating VLM descriptions of foveated versions of the image sampled at many fixation locations, (3) computing semantic similarities between the gold-standard and foveated descriptions, and (4) aggregating the resulting similarity map with a weighted Ripley's K function. The authors report that the aggregate F-SUM score correlates with mean human response times (r=0.47, N=17 participants, 277 scenes), with number of saccades (r=0.51), and negatively with description accuracy under 2- and 4-saccade viewing (r=-0.56 and -0.54), and that these correlations significantly exceed those of clutter, complexity, language-entropy, and directly prompted VLM baselines. The model has no fitted parameters.","tokens_in":12347,"tokens_out":7244,"duration_ms":80111,"significance":"If the reported correlations are reproducible, F-SUM would be a useful new image-computable predictor for scene-comprehension time and would provide evidence that the interaction between foveated vision and the spatial distribution of task-relevant information is a key bottleneck. The paper has several strengths: the construction is parameter-free, the metric is evaluated with two independent VLM backends, bootstrap confidence intervals are reported for the main correlations, and the comparison set includes multiple reasonable baselines. However, the central aggregation step as written is internally inconsistent with the reported results, and the core VLM-proxy assumption is not independently validated. The empirical correlations are moderate, and the advantages over the strongest baselines for response time are marginal (p=0.03, one-sided). These issues make the current manuscript unsuitable for acceptance without revision.","major_comments":[{"comment":"","section":"Section 2.3, Eqs. (2)-(3)"},{"comment":"","section":"Section 2.1 and Section 3.4"},{"comment":"","section":"Tables 1-2 and Section 3.4"},{"comment":"","section":"Section 2.3, Eq. (2)"}],"minor_comments":[{"comment":"","section":"Section 3.5"},{"comment":"","section":"Section 2.3"},{"comment":"","section":"Section 2.2, Eq. (1)"},{"comment":"","section":"Figure 4"},{"comment":"","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The sign inconsistency in Eqs. (2)-(3) is the most serious issue: it suggests the reported results may have been produced by a computation that differs from the one described in the text. I would ask the authors to provide the actual implementation or pseudocode as part of the revision, and to clarify whether the published equations are simply misstated or whether the reported numbers require a different aggregation. The VLM-proxy assumption is also central and should be addressed with a dedicated validation experiment rather than a post-hoc argument. The paper's novelty is real, but the current manuscript is not reproducible enough for acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this one has an interesting idea and real data, but the central math as written doesn't do what the text claims. The equations for the weighted Ripley's K aggregation have a sign inversion problem, and the paper doesn't state the fix. Until that's resolved, I wouldn't trust the headline correlations.\n\nWhat's genuinely new: F-SUM is the first image-computable metric I've seen that tries to predict scene comprehension time by combining foveated vision with VLM descriptions and a spatial aggregation. The absence of fitted parameters, the use of two VLM backends, and bootstrap confidence intervals all count in its favor. The human studies are real psychophysics, and the saccade-limited experiment is a nice touch. The correlations (r≈0.47 for RT, 0.51 for saccades, -0.56 for description accuracy) are moderate but beat clutter, complexity, and language entropy baselines.\n\nNow the problem. Section 2.3 defines M as the similarity between foveated and gold-standard descriptions, with higher M meaning more informative fixations. Then Eq. 2 sets w_p and w_q to the pixel values of M, and Eq. 3 sums products w_p w_q over pairs. For an easy scene where every fixation gives a high similarity (the baseball example in Fig. 1), every product is large, so S is large. But the text says a higher S means the scene is harder, and the paper's own examples show easy scenes with negative F-SUM z-scores. The headline result is a positive correlation between S and RT. That can't hold with w=M. The implementation must be using an inversion (e.g., w=1-M) that is nowhere stated. This is an internal contradiction in the central construction, not a modeling nuance.\n\nThat's the load-bearing issue. The VLM-proxy assumption (that a VLM's foveated description matches human perception) is a genuine concern but secondary. The low human-human RT correlation (r=0.22) also makes me wonder how reliable the ground truth is. No code or data is released, and the ~4% trial exclusion threshold is arbitrary. All fixable in principle.\n\nBottom line: the idea is worth a serious referee, but the paper as submitted cannot be accepted. The authors need to (1) correct the sign inconsistency explicitly, (2) validate the VLM proxy against human foveated perception, and (3) release code and data. If the sign issue turns out to be a simple typo, the paper could be a solid contribution. As it stands, I wouldn't cite it or build on it.","headline":"F-SUM is an intriguing new image-computable metric with real psychophysics behind it, but the central aggregation equations as written have a sign inversion that makes the reported correlations unsupported by the text.","tokens_in":12936,"tokens_out":3679,"would_cite":false,"duration_ms":36517,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The time to comprehend a scene is predictable from an image-computable score that combines human-like foveated blurring with vision-language descriptions.","keywords":["foveated vision","response time prediction","vision-language models","scene understanding","weighted Ripley's K","spatial information distribution","saccades","image-computable metric"],"falsifier":"Have human observers describe scenes rendered with exactly the paper's foveation model under gaze-contingent single-fixation display and compare their descriptions to the VLM's foveated descriptions; if human comprehension at a fixation does not track VLM similarity to the gold standard across locations, the F-SUM mechanism is measuring VLM behavior rather than human vision, and the reported correlations should collapse when the proxy is corrected.","tokens_in":11816,"feed_emoji":"👁️","tokens_out":7305,"duration_ms":63723,"temperature":0.7,"pith_summary":"This paper tries to establish that the main reason some scenes take longer to understand is the interplay between human foveated vision—high detail only at the point of gaze, blurred periphery—and where the task-relevant information sits in the image. To test this, the authors build a fully image-computable score, F-SUM, that foveates an image at many fixation points, asks a vision-language model to describe each blurred view, and compares each description with a gold-standard full-image description. The aggregate score correlates with average human response times (r = 0.47), number of saccades (r = 0.51), and description accuracy under 2- and 4-saccade viewing (r = -0.56), and it beats clutter, complexity, and language-entropy baselines. If the claim holds, response-time variability in scene comprehension can be predicted from an image alone, without eye tracking or fitted parameters.","feed_headline":"Foveation-aware metric predicts scene understanding time","feed_subtitle":"Blurs images like the human eye, then uses vision-language descriptions to rank scenes by the fixations they demand.","key_machinery":"The central object is the Foveated Scene Understanding Map (F-SUM): for each of 108–136 fixation locations on a grid, the image is passed through a gaze-contingent foveation model (multi-level Gaussian pyramid blur matched to human acuity falloff), a VLM (GPT-4o or Ovis2-8B) produces five descriptions, and the mean cosine similarity between their embeddings and gold-standard full-image description embeddings is stored as the map value. The aggregation step is a weighted Ripley's K-function, which sums products of map values at nearby locations with weights 1/r, so scenes whose informative regions are clustered receive higher difficulty scores. The combination carries the argument: the foveation step encodes the human visual constraint, the VLM supplies semantic content, and the spatial statistic converts the map into a single score that can be correlated with behavior.","core_discovery":"F-SUM is a 2D map whose entry at each fixation location is the mean cosine similarity between embeddings of descriptions of the foveated image at that location and embeddings of descriptions of the original unfoveated scene, which serve as the gold standard for full comprehension. The map is aggregated by a weighted Ripley's K-function that rewards both overall information and spatial clustering of informative gaze locations, yielding a difficulty score. The paper reports that this score predicts average human response time (r = 0.47, bootstrapped 95% CI 0.37–0.56) and saccade count (r = 0.51, CI 0.42–0.59) across 277 scenes with 17 observers, and predicts description accuracy in saccade-limited viewing with 16 observers (r = -0.56 for 2 saccades, r = -0.54 for 4 saccades). The authors interpret the pattern as evidence that the bottleneck in scene understanding is the interaction between foveated visual limitations and the spatial distribution of comprehension-critical information, not raw clutter, complexity, or description uncertainty.","pith_inferences":["The paper does not test whether a vision-language model's foveated descriptions genuinely mirror human peripheral perception; a human single-fixation description study would be the natural check, and if the proxy fails, the reported correlations might reflect VLM artifacts rather than human vision.","The F-SUM map itself could be reused as a fixation-suitability map for active vision or gaze-planning applications, since it scores every location by how much scene understanding it recovers; this use is an extension the authors do not pursue.","Combining F-SUM with language entropy or complexity scores might improve response-time prediction further, since those baselines carry partially independent information; the authors note they did not explore such combinations.","Because the metric is computed from a fixed grid rather than from human gaze, individual differences in peripheral acuity or viewing strategy are not captured; incorporating observer-specific foveation parameters is a plausible next step the authors mention as a limitation."],"forward_implications":["F-SUM gives a parameter-free, training-free image-computable predictor of scene comprehension time, so response time can be estimated for arbitrary new images with no eye-tracking data.","The significant gains over clutter, complexity, and language-entropy baselines support the paper's hypothesis that foveation interacting with spatial layout, rather than low-level density or description uncertainty, drives comprehension difficulty.","Scenes with critical details concentrated around one area or identifiable in the periphery should be predicted to be understood quickly, whereas scenes with informative regions widely scattered should require more saccades and longer times."],"supporting_citations":[{"why":"Supplies the gaze-contingent foveation method used to blur images according to distance from fixation.","marker":"Perry and Geisler, 2002"},{"why":"Provides the foveation implementation parameters and saliency context used to render foveated images.","marker":"Jiang et al., 2015"},{"why":"Basis for the closed-source VLM backend (GPT-4o) that generates scene descriptions.","marker":"Achiam et al., 2023"},{"why":"Ovis2-8B is the open-source VLM used for the fully open-source F-SUM variant and the language-entropy baseline.","marker":"Lu et al., 2024b"},{"why":"Gemini embedding model used to embed scene descriptions and compute similarity against gold standards.","marker":"Lee et al., 2025"},{"why":"Source of the distilled open-source embedding model (stella) used in the open-source F-SUM pipeline.","marker":"Zhang et al., 2024"},{"why":"Defines the spatial statistic that F-SUM adapts into a weighted measure of informativeness clustering.","marker":"Ripley, 1976"},{"why":"Provides the predictive-entropy formulation used as the language-entropy baseline.","marker":"Malinin and Gales, 2020"},{"why":"Feature congestion, a clutter metric that serves as one of the baselines F-SUM must beat.","marker":"Rosenholtz et al., 2005"},{"why":"ICNet image-complexity baseline that provides one of the strongest alternative predictors.","marker":"Feng et al., 2022"}],"fun_headline_variants":["Foveation maps predict scene comprehension time","Where you fixate predicts scene understanding time","F-SUM maps gaze to scene comprehension time","Gaze-based metric predicts scene understanding speed","Foveation score predicts time to understand scenes"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a vision-language model's description of an image that is sharp only where a person looks and blurred elsewhere correctly captures what a human can perceive from that exact fixation point; the paper gives no independent test of that proxy against human peripheral perception.","fun_headline_variants_meta":{"raw":{"variants":["Foveation maps predict scene comprehension time","Where you fixate predicts scene understanding time","F-SUM maps gaze to scene comprehension time","Gaze-based metric predicts scene understanding speed","Foveation score predicts time to understand scenes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001164,"raw_usage":{"total_tokens":4882,"prompt_tokens":1071,"completion_tokens":3811,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":687,"completion_tokens_details":{"reasoning_tokens":3743}},"tokens_in":687,"tokens_out":3811,"duration_ms":24907,"temperature":1.0,"reasoning_tokens":3743,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:28:49.348853+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Have human observers describe scenes rendered with exactly the paper's foveation model under gaze-contingent single-fixation display and compare their descriptions to the VLM's foveated descriptions; if human comprehension at a fixation does not track VLM similarity to the gold standard across locations, the F-SUM mechanism is measuring VLM behavior rather than human vision, and the reported correlations should collapse when the proxy is corrected.","supporting_citations":[],"review_version":1}