REVIEW 4 major objections 5 minor 1 cited by
Image Segmentation: Inducing graph-based learning
T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read Inserting a graph neural network bottleneck into U-Net produces the best reported IoU on fisheye and skin-lesion segmentation among the compared models.
desk verdict The reported 0.933 mIoU on WoodScape is not supported by the paper's own per-class IoUs, which average about 0.61; the central evidence is internally contradictory. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the GNN bottleneck built on a k-nearest-neighbor graph with relative positional encoding. Given the deepest encoder feature map $F_L \in \mathbb{R}^{H_L \times W_L \times C_L}$, each spatial location is a node; each node is connected to its $k$ closest nodes in a warped coordinate space $P'_{xy} = P_{xy} + R_{xy}$, where $R_{xy}$ is a sine-cosine positional offset. A graph convolution $h_i^{(t)} = \sigma(\sum_{j \in \mathcal{N}(v_i)} W h_j^{(t-1)} + b)$ then refines the features by aggregating neighbors. This construction is what lets the network relate regions that are far apart in the regular pixel grid but close in the distorted image geometry, which is the paper's explanation for the WoodScape gains.
What would settle it
Retrain U-Net, U-Net++, SwinUNet, and UNet-GNN on the same WoodScape split with the same preprocessing, augmentation, optimizer, learning-rate schedule, and epoch count, then compare mean IoU; if the 0.933 versus 0.878 gap shrinks to noise, the claimed advantage of the GNN bottleneck is not demonstrated.
Extended reading notes
Core claim
On its own terms, the paper's central discovery is that a graph-based bottleneck can do what local convolutions and global self-attention do not fully do: it can make segmentation robust to the non-uniform geometry of fisheye images. The authors replace the plain U-Net bottleneck with graph convolutional layers. Each node is a spatial cell of the deepest feature map, edges are drawn to k-nearest neighbors under a warped coordinate system that includes a learned relative positional offset, and node features are updated by aggregating neighbor features. The reported numbers—mean IoU 0.933 on WoodScape, 0.833 on ISIC2016, and 0.774 on PascalVOC—are offered as evidence that the GNN bottleneck raises accuracy over U-Net, U-Net++, and SwinUNet on distorted and boundary-heavy imagery while remaining competitive on a standard natural-image benchmark.
Load-bearing premise
The paper's causal reading of the numbers assumes all compared models were trained and evaluated under identical conditions, with the only difference being the GNN bottleneck; the authors do not report the training configuration that would make this verifiable.
Editorial extensions
If this is right
- If the WoodScape result holds, GNN bottlenecks are a plausible drop-in upgrade for U-Net-style segmenters in fisheye camera pipelines, where convolutions alone lose accuracy near the image periphery.
- The same bottleneck could help other tasks that share the same failure mode, such as instance segmentation or depth estimation on wide-angle cameras, because it does not depend on the segmentation head.
- The gap over SwinUNet on WoodScape suggests that explicit relational modeling over a distortion-aware graph can beat global self-attention when the image geometry is non-uniform.
- On medical images such as ISIC2016, the reported gains indicate that graph-based feature refinement can help delineate lesions with highly variable shape and boundary, not just distorted driving scenes.
- The method is architecture-agnostic at the encoder and decoder level, so stronger convolutional backbones could be combined with the graph bottleneck to push accuracy further.
Reading between the lines
- The paper leaves training details unspecified, so a fair reading is that the reported improvements are conditional on the comparison being controlled; if the baselines were trained with different schedules or augmentation, part of the gap could come from factors other than the graph module.
- The warped-coordinate k-NN construction suggests a direct test: distort a standard benchmark with a synthetic fisheye model and measure how much of the gain is explained by the positional offset $R_{xy}$ versus the graph aggregation alone.
- Because the GNN bottleneck operates on a comparatively small feature map, it may be a cheaper route to global context than SwinUNet's shifted-window attention; a parameter and latency comparison would tell whether the accuracy gain comes with a computational win.
- If the Generalized Wasserstein Dice Loss proposed in the limitations section is incorporated, the model's confusion between rider and person and between bicycle and motorcycle would be the concrete place to look for further gains.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes UNet-GNN, a U-Net variant with a graph neural network (GNN) bottleneck for semantic segmentation. The authors evaluate it on PascalVOC, WoodScape, and ISIC2016 against U-Net, U-Net++, and SwinUNet, reporting higher IoU on WoodScape (0.933) and ISIC2016 (0.833) and competitive performance on PascalVOC (0.774). The method constructs a k-NN graph on the encoder bottleneck features using a warped coordinate space with relative positional encodings, and it applies graph convolution to refine features before decoding. The paper concludes that the GNN bottleneck improves segmentation under geometric distortion and irregular boundaries.
Significance. The core idea—inserting a GNN bottleneck into a U-Net—is a reasonable and potentially useful architectural variant for segmentation under non-Euclidean or distorted image conditions, and the authors make a good-faith attempt to evaluate it across diverse domains. The work also offers a concrete implementation direction (code claimed on GitHub) and an explicit limitation statement. However, the paper's scientific value hinges entirely on the reported numerical results, and those results are internally inconsistent in the main table. As presented, the central claim that UNet-GNN outperforms the baselines on WoodScape is not supported by the manuscript's own evidence. Because the experimental reporting lacks essential training details and statistical grounding, the contribution is currently more a proposal than a validated empirical study.
major comments (4)
- [Results, Table 1] Table 1 contains a direct arithmetic contradiction between per-class IoU values and the reported average mIoU. For UNet-GNN on WoodScape, the per-class IoUs are 0.97, 0.94, 0.69, 0.60, 0.44, 0.37, 0.90, 0.43, 0.60, and 0.17; the mean of these values is 0.611, not the reported 0.93. The same problem occurs for U-Net (per-class mean 0.494 vs. reported 0.87), U-Net++ (0.470 vs. 0.81), and SwinUNet (0.552 vs. 0.89). The Average Accuracy row (0.99/0.98/0.98/0.99) is also inconsistent with per-class accuracies that go as low as 0.10 for Traffic Sign. Since the central claim of the paper—the 0.933 IoU on WoodScape—rests on this average row, the evidence for the headline result is invalid as written. The authors must either correct the per-class values or the averages; without this, no comparison between methods can be trusted.
- [Results, PascalVOC] The PascalVOC paragraph states, 'SwinUNet attains a higher IoU of 0.774', but the text immediately before gives UNet-GNN an IoU of 0.774 as well. 'Higher' is therefore inaccurate; the two values are equal. This is a minor wording issue, but it also signals a lack of care in reporting that is consistent with the larger Table 1 inconsistency. Please verify all comparative statements against the actual numbers.
- [Methodology, Graph Construction and Relative Positional Encoding] The relative positional encoding is described as a 'learned offset Rxy' in the Graph Construction section, yet the next paragraph states that Rxy is 'computed via sine-cosine functions as proposed in [21]'. Sine-cosine positional encodings are typically fixed functions of position, not learned offsets. The paper never clarifies whether Rxy is learned or fixed, and if learned, how it is trained. Since the entire motivating argument for the GNN bottleneck is that warping coordinates helps with fisheye distortion, this ambiguity is not purely cosmetic. Please specify the exact form of Rxy and whether it is an independent learned parameter or a deterministic function.
- [Results and Methodology (overall experimental reporting)] The paper reports single-run IoU scores with no error bars, no standard deviations, and no information about training configuration: no optimizer, learning rate, batch size, number of epochs, loss function, data splits, preprocessing, augmentation, or evaluation protocol. Without this information, the reader cannot determine whether the reported differences (e.g., 0.933 vs. 0.893 on WoodScape) are statistically meaningful or whether all models were trained under identical conditions. Since the paper claims that the GNN bottleneck is the cause of the improvement, experimental control is essential. A proper ablation (e.g., U-Net with and without the GNN bottleneck under matched training) would also be needed to isolate the contribution of the graph module.
minor comments (5)
- [Table 1] The table header 'Sr . # Categories' contains an extra space, and the rows 'A verage mIoU' and 'A verage Accuracy' have inconsistent spacing; these are typographical issues that should be corrected.
- [Results, WoodScape] The text reports SwinUNet's WoodScape IoU as 0.893, but Table 1 lists 0.89. Please ensure consistency between prose and tables.
- [Limitations and Future Work] The generalized Wasserstein Dice Loss equation is not properly typeset and lacks the normalization constants typically present in the original formulation. Please provide a complete, correctly formatted equation or a proper citation to the source.
- [Overall] The paper states 'Code Available at GitHub' but does not provide a URL or repository identifier. Since reproducibility is a stated goal (and one of the contributions), please provide a working link.
- [Introduction] The reference to '[5]' as a general claim about CNNs being sensitive to geometric distortion is fine, but several statements in the Introduction and Related Work are over-broad and would benefit from more specific citations (e.g., the claim that 'CNNs are sensitive to geometric distortions' is not quantified).
Circularity Check
No circularity found: the paper's claims are empirical comparisons against external benchmarks; no derived result reduces to fitted inputs or self-citations.
full rationale
The paper is an empirical architecture-comparison study. Its central claims (UNet-GNN IoU values on PascalVOC, WoodScape, ISIC2016) are reported measurements from training/evaluation, not quantities derived from the model's defining equations or from fitted parameters that are then renamed as predictions. The GNN bottleneck is described by explicit equations (graph construction, k-NN with relative positional encoding, graph convolution Eq. 2, decoder Eqs. 3-4), and there is no step in which an output is shown to be identical to an input by construction. The only self-citations are refs [5] (Kumar et al., including co-author Eising) and [13] (Manzoor et al., including authors Singh and Eising), and these support background claims about fisheye-camera perception and deformable convolution; they are not the evidence for the reported segmentation scores, which are benchmarked against U-Net, U-Net++, and SwinUNet on standard datasets. There is no uniqueness theorem imported from the authors and no ansatz smuggled in via self-citation: the relative positional encoding is attributed to the standard Transformer paper [21]. The Limitations section openly acknowledges class-confusion weaknesses, which is a substantive admission rather than a circular justification. The reader-identified concern about missing training configuration is a reproducibility/experimental-rigor issue, and the WoodScape Table 1 average-vs-per-class discrepancy is an internal arithmetic inconsistency, but neither constitutes derivation-by-construction circularity. Therefore the circularity score is 0.
Assumptions & free parameters
free parameters (3)
- k in k-NN graph construction
- number of GNN layers
- learned offset scale for relative positional encoding
assumptions (3)
- domain assumption The U-Net encoder-decoder with skip connections and GNN bottleneck is a sufficient architecture for the task.
- domain assumption The sine-cosine relative positional encoding can capture fisheye distortion geometry.
- domain assumption Graph convolution aggregation over k-NN in warped space provides global context.
Cite this review
Pith. "Pith review of Image Segmentation: Inducing graph-based learning." pith.science (2026). https://pith.science/paper/5UA6RE5C
@misc{pith2026250103765,
author = {Pith},
title = {Pith review of: Image Segmentation: Inducing graph-based learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/5UA6RE5C}},
note = {Machine review of arXiv:2501.03765}
}
read the original abstract
This study explores the potential of graph neural networks (GNNs) to enhance semantic segmentation across diverse image modalities. We evaluate the effectiveness of a novel GNN-based U-Net architecture on three distinct datasets: PascalVOC, a standard benchmark for natural image segmentation, WoodScape, a challenging dataset of fisheye images commonly used in autonomous driving, introducing significant geometric distortions; and ISIC2016, a dataset of dermoscopic images for skin lesion segmentation. We compare our proposed UNet-GNN model against established convolutional neural networks (CNNs) based segmentation models, including U-Net and U-Net++, as well as the transformer-based SwinUNet. Unlike these methods, which primarily rely on local convolutional operations or global self-attention, GNNs explicitly model relationships between image regions by constructing and operating on a graph representation of the image features. This approach allows the model to capture long-range dependencies and complex spatial relationships, which we hypothesize will be particularly beneficial for handling geometric distortions present in fisheye imagery and capturing intricate boundaries in medical images. Our analysis demonstrates the versatility of GNNs in addressing diverse segmentation challenges and highlights their potential to improve segmentation accuracy in various applications, including autonomous driving and medical image analysis.
Figures
Forward citations
Cited by 1 Pith paper
-
Graph Neural Network Reveals the Cortical Morphology of Local Brain Aging in Normal Cognition and Alzheimer's Disease
A graph neural network trained on cortical surface morphometry produces vertex-level local brain age maps that show prefrontal/parietal aging in normal cognition and parahippocampal/temporal aging in Alzheimer's disease.
Reference graph
Works this paper leans on
-
[21]
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, Attention Is All You Need, arXiv preprint arXiv:1706.03762, 2023. Available at: https://arxiv.org/abs/1706.03762
arXiv 2023
-
[1]
LeCun, Y ., Bottou, L., Bengio, Y ., and Haffner, P. (1998). Gradient-based learning applied to document recognition . Proceedings of the IEEE. Available at: https://doi.org/ 10.1109/5.726791. doi: 10.1109/5.726791
doi:10.1109/5.726791 1998
-
[2]
O. Ronneberger, P. Fischer, and T. Brox, U-Net: Convolu- tional Networks for Biomedical Image Segmentation , arXiv preprint arXiv:1505.04597 , 2015. Available at: https:// arxiv.org/abs/1505.04597
arXiv 2015
-
[3]
Z. Zhou, M. M. R. Siddiquee, N. Tajbakhsh, and J. Liang, UNet++: A Nested U-Net Architecture for Medical Image Segmentation, arXiv preprint arXiv:1807.10165, 2018. Avail- able at: https://arxiv.org/abs/1807.10165
arXiv 2018
-
[4]
H. Cao, Y . Wang, J. Chen, D. Jiang, X. Zhang, Q. Tian, and M. Wang, Swin-Unet: Unet-like Pure Trans- former for Medical Image Segmentation , arXiv preprint arXiv:2105.05537, 2021. Available at: https://arxiv. org/abs/2105.05537
arXiv 2021
-
[5]
V . R. Kumar, C. Eising, C. Witt, and S. Yogamani, Surround-view Fisheye Camera Perception for Automated Driving: Overview, Survey and Challenges , arXiv preprint arXiv:2205.13281, 2023. Available at: https://arxiv. org/abs/2205.13281
work page Pith review arXiv 2023
-
[6]
C., Hagenbuchner, M., and Monfardini, G
Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M., and Monfardini, G. (2009). The Graph Neural Network Model . IEEE Transactions on Neural Networks, 20(1), 61–80. doi: 10.1109/TNN.2008.2005605
arXiv 2009
-
[7]
Han, K., Wang, Y ., Guo, J., Tang, Y ., and Wu, E. (2022). Vision GNN: An Image is Worth Graph of Nodes . arXiv preprint arXiv:2206.00272. Available at: https://arxiv. org/abs/2206.00272
arXiv 2022
Show all 25 references
-
[8]
Griffiths and D
R. Griffiths and D. G. Dansereau, Adapting CNNs for Fisheye Cameras without Retraining , arXiv preprint arXiv:2404.08187, Apr. 2024. Available at: https:// arxiv.org/abs/2404.08187
2024 arXiv
-
[9]
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., and Zisserman, A. (2010). The Pascal Visual Object Classes (VOC) Challenge. International Journal of Computer Vision, 88(2), 303–338
2010
-
[10]
Yogamani, C
S. Yogamani, C. Hughes, J. Horgan, G. Sistu, P. Varley, D. O’Dea, M. Uricar, S. Milz, M. Simon, K. Amende, C. Witt, H. Rashed, S. Chennupati, S. Nayak, S. Man- soor, X. Perroton, and P. Perez, WoodScape: A Multi- task, Multi-camera Fisheye Dataset for Autonomous Driv- ing, arX...
1905 arXiv
-
[11]
Gutman, N
D. Gutman, N. C. F. Codella, E. Celebi, B. Helba, M. Marchetti, N. Mishra, and A. Halpern, Skin Lesion Anal- ysis toward Melanoma Detection: A Challenge at the In- ternational Symposium on Biomedical Imaging (ISBI) 2016, hosted by the International Skin Imaging Collaboration (...
2016 arXiv
-
[12]
Muhammad, T
K. Muhammad, T. Hussain, H. Ullah, J. Del Ser, M. Rezaei, N. Kumar, M. Hijji, P. Bellavista, and V . H. C. de Albu- querque, Vision-Based Semantic Segmentation in Scene Un- derstanding for Autonomous Driving: Recent Achievements, Challenges, and Outlooks, IEEE Transactions on ...
-
[13]
Manzoor, A
A. Manzoor, A. Singh, G. Sistu, R. Mohandas, E. Grua, A. Scanlan, and C. Eising, Deformable Convolution Based Road Scene Semantic Segmentation of Fisheye Images in Au- tonomous Driving, arXiv preprint arXiv:2407.16647 , 2024. Available at: https://arxiv.org/abs/2407.16647
2024 arXiv
-
[14]
Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo, Swin Transformer: Hierarchical Vi- sion Transformer using Shifted Windows , arXiv preprint arXiv:2103.14030, 2021. Available at: https://arxiv. org/abs/2103.14030
2021 arXiv
-
[15]
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N. (2021). An Image is Worth 16x16 Words: Transformers for Im- age Recognition at Scale . arXiv preprint arXi...
2021 arXiv
-
[16]
Q. Zhao, C. Zhu, F. Dai, Y . Ma, G. Jin, and Y . Zhang, Distortion-aware CNNs for Spherical Images , in Proceed- ings of the International Joint Conference on Artificial In- telligence (IJCAI), pp. 1198–1204, 2018
2018
-
[17]
Walker, V
T. Walker, V . Anand, and P. Andreadis, Spherical Fea- ture Pyramid Networks For Semantic Segmentation , arXiv preprint arXiv:2307.02658 , 2023. Available at: https:// arxiv.org/abs/2307.02658
2023 arXiv
-
[18]
Scaramuzza, A
D. Scaramuzza, A. Martinelli, and R. Siegwart, A Toolbox for Easily Calibrating Omnidirectional Cameras, in Proceed- ings of the 2006 IEEE/RSJ International Conference on In- telligent Robots and Systems , pp. 5695–5701, 2006. DOI: 10.1109/IROS.2006.282372
2006
-
[19]
N., and Welling, M
Kipf, T. N., and Welling, M. (2016). Semi-Supervised Classification with Graph Convolutional Networks . CoRR, abs/1609.02907. Available at: http://arxiv.org/abs/ 1609.02907
2016 arXiv
-
[20]
M. M. Bronstein, J. Bruna, Y . LeCun, A. Szlam, and P. Van- dergheynst, Geometric Deep Learning: Going beyond Eu- clidean data , IEEE Signal Processing Magazine , vol. 34, no. 4, pp. 18–42, Jul. 2017. DOI: 10.1109/MSP.2017. 2693418. Available at: http://dx.doi.org/10.1109/ MSP...
2017
-
[22]
H. Wang, H. Yin, M. Zhang, and P. Li, Equivariant and Stable Positional Encoding for More Powerful Graph Neural Networks, arXiv preprint arXiv:2203.00199, 2022. Available at: https://arxiv.org/abs/2203.00199
2022 arXiv
-
[23]
F. Lan, C. Yang, G. Cheung, and J. Z. G. Tan, Joint De- mosaicking / Rectification of Fisheye Camera Images using Multi-color Graph Laplacian Regularization, arXiv preprint arXiv:2006.11636, 2020. Available at: https://arxiv. org/abs/2006.11636
2006 arXiv
-
[24]
Fidon, W
L. Fidon, W. Li, L. C. Garcia-Peraza-Herrera, J. Ekanayake, N. Kitchen, S. Ourselin, and T. Vercauteren, Generalised Wasserstein Dice Score for Imbalanced Multi-class Segmen- tation Using Holistic Convolutional Networks , DOI: 10. 1007/978-3-319-75238-9_6 . Available at: http:...
-
[2022]
DOI: 10.1109/TITS.2022.3207665
2022
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.