{"id":"09d1ca35-8ac2-403b-813d-08a67e32f6b3","arxiv_id":"2504.19682","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"The paper quantifies and visualizes the layer-wise evolution of Vision GNN graphs, showing a shift from local, visually similar connections to long-range, class-specific connections that do not align with human perception.","lead":"This paper measures how the graph structure inside a Vision GNN changes layer by layer when classifying images. It shows that early layers connect similar nearby patches, while deeper layers connect distant patches in ways that diverge from human perception, and it offers metrics and heatmaps to track this.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Embedding-similarity metric is circular: edges are chosen by KNN on the same embeddings, so S_emb measures self-similarity, not semantic coherence.","rationale":"The reader's weakest assumption (mask quality) is a valid concern for Q, but Q is one secondary metric and the ImageNet-a comparison is not the core of the central claim. The circularity of S_emb is more load-bearing because it undermines the paper's primary semantic-coherence evidence and the 'conviction spike' narrative that bridges graph structure to decisions. This is an internal validity issue, not an external benchmark issue, and it can be resolved by explicit controls. I agree with the reader's overall CONDITIONAL verdict; the circularity is already among their requested revisions, so the verdict remains conditional. Credit is due where the paper has independent support: the S_vis and D metrics are directly measurable, and the qualitative heatmaps are illustrative; these parts survive. The modularity and mask concerns also remain, but the circularity of the embedding-similarity metric should be addressed first.","tokens_in":8749,"tokens_out":14085,"duration_ms":145903,"concrete_test":"Recompute Table 1 with control graphs at each layer: (a) random rewiring of the ViG-selected edges preserving the degree sequence, and (b) a fixed spatial-neighbor graph (e.g., 8-connected grid) on the same layer features. Plot S_emb for the ViG-selected edges versus both controls. If the control curves reproduce the same late-layer spike, then the spike is a feature-geometry artifact of KNN selection and cannot be used to support class-specific decision consolidation. If the ViG curve clearly exceeds the controls, the circularity concern is resolved. Report the controls in the same format as Table 1 and in the diagrams of Fig. 2 and Fig. 3(c).","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing assumption for the semantic part of the central claim is that S^l_emb (Eq. 2) measures how semantically related the patches the model connects are. But ViG's dynamic graph construction (Sec. 3.2) selects edges at layer l as the K nearest neighbors in the embedding space of the same features x^l, using cosine similarity. Eq. 2 then averages cosine similarity over exactly those selected edges; it is the mean of each node's top-K similarities. S^l_emb is therefore a self-similarity statistic of the KNN graph, not an independent measurement of whether the model 'links semantically related regions' or of 'semantic coherence' (Sec. 4.2). The late-layer spike (Table 1: 0.842 to 0.900; Fig. 2) is exactly what is expected when features become more clustered as the classifier sharpens; it does not demonstrate a decision-making process. The conclusion that 'the most confident predictions emerge from representations that diverge significantly from human visual intuition' relies on the contrast between S_vis and S_emb, but the S_emb side of that contrast is partly fixed by construction. Other metrics (S_vis, D) are not circular, but they cannot carry the semantic-coherence interpretation alone.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a white-box explainability analysis of the Vision GNN (ViG) image classifier. It defines five metrics—embedding similarity, spatial distance, visual similarity, layer-wise class probability, and object-based modularity—and reports their evolution across layer pairs on a 10,000-image ImageNet subset and on 7,500 ImageNet-a adversarial images. The main descriptive finding is that as layers deepen, connected patches become more spatially distant and less visually similar, while embedding similarity rises sharply in the final layers, coinciding with increasing classification confidence. The authors also provide heatmap visualizations and claim that their analysis reveals ViG's decision-making can be explained and that its later-layer reasoning diverges from human perception.","tokens_in":8980,"tokens_out":7476,"duration_ms":67375,"significance":"If validated, this would be a useful contribution to interpretability for graph-based vision models, introducing a suite of quantitative metrics and heatmap visualizations that could be reused by the community. The paper is honest about its white-box assumption and provides code. The non-circular metrics (D and S_vis) give a clean, reproducible description of the model's increasing receptive field, and the comparison between in-domain and adversarial images is a relevant robustness question. However, the paper's central semantic-coherence claim rests on an embedding-similarity metric that is in part a tautology of the KNN graph construction, and the absence of statistical validation and mask-quality checks limits the strength of the additional conclusions.","major_comments":[{"comment":"The metric S^l_emb is circular with respect to the graph construction. ViG builds E^l by connecting each node to its K nearest neighbors in the same embedding space using cosine similarity (Sec. 3.2), so Eq. (2) simply averages the top-K cosine similarities of each node. The late-layer increase in S_emb (Table 1: 0.842 to 0.900; Fig. 2) is therefore expected as features cluster and does not, by itself, show that the model 'links semantically related regions' (Sec. 4.2) or that confident predictions diverge from human visual intuition (Sec. 5.2). To support the semantic interpretation, the authors should compare S_emb to a baseline (e.g., random edges or non-neighbor pairs) or validate the selected edges with an external semantic signal.","section":"Sec. 3.2, Eq. (2)"},{"comment":"All quantitative conclusions are based on single point estimates. There are no error bars, standard deviations, or significance tests, yet the text makes comparative claims such as a 'sudden increase' in S_emb and 'significantly lower' modularity for ImageNet-a (Q 0.095 vs 0.236). For a dataset of 10,000 and 7,500 images, the authors should report per-image variation and appropriate significance tests (paired or bootstrap) for the layer-to-layer and dataset-to-dataset differences.","section":"Table 1, Sec. 5.1"},{"comment":"The ImageNet subset is described only as '10,000 validation images' without details of sampling, seed, or class balance, making selection bias and reproducibility impossible to assess. Please specify the exact subset construction or use the full validation set. Also correct the statement in Sec. 5.1 that results are across 'all three datasets', since only ImageNet and ImageNet-a are used.","section":"Sec. 4.1"},{"comment":"The modularity formula is not the standard directed-graph modularity and is not derived. For a directed graph, modularity is usually defined with k_in and k_out in the null model (e.g., Leicht and Newman), and the form used here, with (k_in^c k_out^c)/(2|E|)^2, is not justified. In addition, the object/background masks from GroundingDINO and SAM are taken as ground truth with no quality validation; for multi-object images the 'main object' is ambiguous, and the adversarial versus standard difference in Q could be an artifact of segmentation failures. The authors should either adopt the correct modularity definition and validate masks (e.g., IoU on a human-annotated subset) or show that their qualitative conclusions are robust to these choices.","section":"Sec. 4.2, Eq. (6)"},{"comment":"The paper analyzes only ViG-Small, yet the title, abstract, and conclusion refer to 'Vision GNNs' and 'GNN-based vision models' in general. To substantiate the plural claim, the authors should add at least one additional architecture (e.g., MobileViG or GreedyViG) or explicitly restrict the scope and revise the abstract and conclusion accordingly.","section":"Sec. 5.1 and abstract"}],"minor_comments":[{"comment":"The phrase 'Explanations from standard and adversarial settings are also compared to assess whether they reflect the classifiers' robustness' is unclear; the comparison is between metric values, not between 'explanations' as objects.","section":"Abstract"},{"comment":"The star and dagger markers for ImageNet S_emb are inconsistent with the stated rule; the third-highest value (0.900, layers 15-16) is not marked, while 0.845 in layers 13-14 is marked as second highest. Please correct the markers.","section":"Table 1"},{"comment":"The symbol L_c is used before being defined; please define it as the number of intra-community edges.","section":"Eq. (6)"},{"comment":"References [6] and [7] are duplicates of the same paper (Hendrycks et al.); remove one.","section":"References"},{"comment":"Fig. 2 and Fig. 3 are referenced in the text but appear to be missing from the arXiv preprint; ensure the final submission includes all figures.","section":"Figures"},{"comment":"If the ImageNet subset is made available, please provide a download link or the exact selection procedure.","section":"Sec. 4.1"}],"recommendation":"major_revision","confidential_remarks":"The circularity of S_emb is the main technical obstacle, but it is fixable with appropriate baseline comparisons or external semantic validation. The lack of statistical rigor and the non-standard modularity formula also need attention. If the authors address these points, the paper could be a reasonable contribution to the interpretability literature. There is no sign of novelty misrepresentation, but the plural claims about 'Vision GNNs' should be tempered given that only one small variant is tested."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a useful but shallow diagnostic study of a Vision GNN. The descriptive local-to-global trend is real and the code is public, but the main 'semantic coherence' metric is partly circular, and the numbers have no error bars or significance tests.\n\nWhat's actually new: applying simple layer-wise metrics—cosine similarity, Manhattan distance, modularity—to a ViG model and documenting how they evolve. That specific application hasn't been done for this architecture, and the public code helps. Table 1 gives a clear picture: early layers favor spatially nearby, pixel-similar patches; deeper layers connect distant patches with lower visual similarity and higher embedding similarity. That is a genuine empirical description of this model.\n\nThe main soft spot is the embedding similarity S_emb. Edges are constructed at each layer by KNN on the same cosine similarity in the same feature space, so S_emb is the average of each node's top-K similarities. A high value means the features are clustered—which is interesting—but it does not independently establish that the model 'links semantically related regions.' The late-layer spike is exactly what you'd expect as the classifier sharpens. The authors need to either reframe the claim as a self-similarity statistic or compare against an external semantic ground truth. The other metrics (D, S_vis) aren't circular but they can't carry the semantic interpretation.\n\nOther issues are fixable. The ImageNet '10,000 validation images' subset is not described—random, class-balanced? No error bars or significance tests anywhere. Only one model (ViG-Small) is analyzed. The modularity metric depends on binary masks from GroundingDINO and SAM, and there is no validation that those masks actually isolate the main object. For multi-object images, 'the object' is ambiguous. The comparison between ImageNet and ImageNet-a is suggestive but statistically thin.\n\nAlso, the novelty claim is overstated. Calling it the 'first explainability analysis of GNN-based image classification approaches' goes too far; there's already work on GNN explainability and on interpreting graph-based vision models. What is new is the specific layer-wise metric suite on ViG, not the idea of explaining such models.\n\nCredit where due: the authors are honest about the white-box constraint, the code is available, and the metrics are clearly defined. The core observable pattern—increasing receptive field and decreasing pixel similarity—holds up as a description of this model.\n\nWho it's for: someone working on GNN explainability or on debugging ViG-style models would find this a reasonable starting point. It's a solid workshop or short-paper candidate after revisions. I'd support peer review, not desk rejection, but I'd expect major revisions. I wouldn't cite it in my own work yet, though I might bring it to a reading group to discuss what layer-wise graph statistics can and can't tell us.\n\nRecommendation: send it to a serious referee; it deserves the time, with the expectation that the circularity and statistical issues get addressed.","headline":"Useful but flawed diagnostic: the local-to-global trend in ViG is real, but the 'semantic coherence' metric is circular and the analysis lacks statistics.","tokens_in":9504,"tokens_out":3180,"would_cite":false,"duration_ms":31346,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A Vision GNN's own graph shows its reasoning diverges from human perception in deep layers.","keywords":["Vision GNN","graph neural networks","image classification","explainability","graph modularity","embedding similarity","adversarial examples","semantic consistency"],"falsifier":"Recompute $Q^l$ on a few hundred ImageNet images using human-annotated ground-truth object masks instead of the automatic GroundingDINO/SAM masks, and also on a control set of random binary masks. If the modularity gap between ImageNet and ImageNet-a (0.236 versus 0.095) shrinks or disappears, the object-separation claim is an artifact of the mask generation; if random masks produce similar values, the metric is not measuring object structure.","tokens_in":8574,"feed_emoji":"🕸️","tokens_out":4311,"duration_ms":40453,"temperature":0.7,"pith_summary":"The paper tries to establish that a Vision GNN's internal graph can be read as a self-documenting record of how an image classification decision forms, and that reading it reveals a process that diverges from human perception. On standard images, early layers connect nearby patches that look alike, while deeper layers link distant patches that the model considers embedding-similar even when they look nothing alike to a human. A late spike in embedding similarity coincides with a jump in classification confidence, suggesting that the model commits to a class through semantic convergence rather than visual coherence. On natural adversarial images, the object-background separation in the graph is weaker from the start and the final convergence spike is muted, matching the model's low accuracy. A sympathetic reader would take away that graph structure itself is a usable explanation surface for these models, even when it shows the model reasoning unlike a human.","feed_headline":"Vision GNNs diverge from human perception in deep layers","feed_subtitle":"Layer-by-layer graph metrics show long-range semantic links, not visual coherence, drive final classification.","key_machinery":"The central object is the per-layer dynamic graph $G^l = (V, E^l)$ built by connecting each of 196 image patches to its $K$ nearest neighbors in embedding space using cosine similarity. The analysis instruments this graph with five metrics—embedding similarity $S^l_{\\text{emb}}$, spatial distance $D^l$, visual similarity $S^l_{\\text{vis}}$, layer-wise ground-truth probability $p^l$, and object-based modularity $Q^l$ computed with binary object masks from GroundingDINO and SAM—plus heatmaps showing incoming edges of selected patches. The metrics are what carry the argument: they transform the dynamic graph update into a measurable story about locality, semantic coherence, and object separation across layers.","core_discovery":"On the paper's own terms, the decision-making of Vision GNN image classifiers can be effectively explained by examining the graphs formed at each layer, and those graphs do not behave like human perception. Across layers there is a consistent progression: visual similarity of connected patches falls from 0.700 to 0.306, spatial distance grows from 3.5 to 8.9 grid steps, embedding similarity dips then spikes to 0.900 in the final layers, and this spike aligns with the largest increase in ground-truth class probability. On ImageNet-a adversarial images, graph modularity starts at 0.095 versus 0.236 on ImageNet and stays lower throughout, and the convergence spike is reduced, indicating weaker object/background separation and less coherent class-specific representations. The explanations are therefore effective as diagnostics of the model's own reasoning, not as evidence of human-aligned reasoning.","pith_inferences":["Editorial inference: The same five metrics could serve as a layer-wise diagnostic for other graph-based vision models, but the paper only studies ViG, so transfer is not established.","Editorial inference: The final embedding-similarity spike may partly be an artifact of k-NN graph construction in deep embedding space, where mutually nearest neighbors become mechanically similar; the paper does not separate this structural effect from genuine semantic convergence.","Editorial inference: If the central claim is right, a regularizer that forces deep-layer edges to be visually similar to human perception should reduce accuracy; that is a direct testable consequence not run in the paper."],"forward_implications":["Graph edges of a ViG become a faithful explanation artifact: one can trace a patch's receptive field and see the model's shift from local visual features to global semantic ones.","The final-layer embedding-similarity spike is a marker of decision consolidation, so monitoring it could serve as a confidence or failure detector without extra supervision.","Low graph modularity on adversarial inputs could act as an early signal of likely misclassification, since it correlates with near-zero accuracy on ImageNet-a.","Explanations should not be expected to match human intuition in deep layers; methods that align machine explanations to human perception may fight against the model's actual behavior.","The dynamic graph construction in ViG is inherently more transparent than fixed receptive fields, because each edge documents a learned relevance relation."],"supporting_citations":[{"why":"Defines the ViG architecture and its dynamic k-NN graph construction, the object of the entire analysis.","marker":"[4]"},{"why":"Supplies the ImageNet validation subset on which the standard-image metrics and baselines are computed.","marker":"[16]"},{"why":"Supplies the ImageNet-a natural adversarial images used for the robustness comparison.","marker":"[6]"},{"why":"Detects the ground-truth object in each image, producing the boxes that seed the binary masks for the modularity metric.","marker":"[11]"},{"why":"Produces the pixel-level object masks from the detected boxes, which the modularity calculation depends on.","marker":"[9]"},{"why":"Provides the max-relative graph convolution used in ViG's message passing, shaping the graph updates under study.","marker":"[10]"}],"fun_headline_variants":["Vision GNNs: explainable, yet diverge from human perception","Deep layers of Vision GNNs favor semantic links over visual coherence","Vision GNN reasoning lacks human-like visual structure in deep layers","Adversarial images show Vision GNNs rely on long-range semantics","Vision GNNs' decisions are traceable, but not human-plausible"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The binary object masks produced by GroundingDINO and SAM are assumed to segment the ground-truth object correctly for every image, and the object-based modularity scores, including the conclusion that adversarial images have weaker object separation, would shift if those masks are wrong.","fun_headline_variants_meta":{"raw":{"variants":["Vision GNNs: explainable, yet diverge from human perception","Deep layers of Vision GNNs favor semantic links over visual coherence","Vision GNN reasoning lacks human-like visual structure in deep layers","Adversarial images show Vision GNNs rely on long-range semantics","Vision GNNs' decisions are traceable, but not human-plausible"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000243,"raw_usage":{"total_tokens":1514,"prompt_tokens":916,"completion_tokens":598,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":532,"completion_tokens_details":{"reasoning_tokens":503}},"tokens_in":532,"tokens_out":598,"duration_ms":6097,"temperature":1.0,"reasoning_tokens":503,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:45:29.654126+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute $Q^l$ on a few hundred ImageNet images using human-annotated ground-truth object masks instead of the automatic GroundingDINO/SAM masks, and also on a control set of random binary masks. If the modularity gap between ImageNet and ImageNet-a (0.236 versus 0.095) shrinks or disappears, the object-separation claim is an artifact of the mask generation; if random masks produce similar values, the metric is not measuring object structure.","supporting_citations":[{"cited_title":"International Journal of Computer Vision115, 211 – 252 (2014),https://api.semanticscholar.org/CorpusID:2930547","cited_arxiv_id":null,"evidence_quote":"Supplies the ImageNet validation subset on which the standard-image metrics and baselines are computed."},{"cited_title":"In: European Conference on Computer Vision","cited_arxiv_id":null,"evidence_quote":"Detects the ground-truth object in each image, producing the boxes that seed the binary masks for the modularity metric."},{"cited_title":"9266–9275 (2019), https://api.semanticscholar.org/CorpusID:201070021","cited_arxiv_id":null,"evidence_quote":"Provides the max-relative graph convolution used in ViG's message passing, shaping the graph updates under study."}],"review_version":1}