Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Image Segmentation: Inducing graph-based learning

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Pith's one-line read Inserting a graph neural network bottleneck into U-Net produces the best reported IoU on fisheye and skin-lesion segmentation among the compared models.

desk verdict The reported 0.933 mIoU on WoodScape is not supported by the paper's own per-class IoUs, which average about 0.61; the central evidence is internally contradictory. read the letter →

arxiv 2501.03765 v2 pith:5UA6RE5C submitted 2025-01-07 cs.CV eess.IV

classification cs.CVeess.IV
keywords semanticsegmentationgraphneuralnetworkU-NetfisheyecameraWoodScapePascalVOCISIC2016relativepositionalencoding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that inserting a graph neural network (GNN) at the bottleneck of a U-Net improves semantic segmentation across image domains, and that the improvement is largest where geometry is most distorted. The authors construct a graph from the deepest CNN feature map, connecting each spatial location to its k-nearest neighbors in a coordinate space warped by learned positional offsets, then refine the features with graph convolutions before decoding. On the fisheye WoodScape dataset they report a mean IoU of 0.933, above U-Net (0.878), U-Net++ (0.810), and SwinUNet (0.893); on ISIC2016 they report 0.833 versus 0.771 for SwinUNet. On PascalVOC both the proposed model and SwinUNet reach 0.774 IoU, exceeding the convolutional baselines. If the result is right, a relatively small architectural change gives convolutional segmenters an explicit mechanism for long-range, non-local relationships, which matters for autonomous driving with wide-angle cameras and for medical images with irregular boundaries.

What carries the argument

The load-bearing mechanism is the GNN bottleneck built on a k-nearest-neighbor graph with relative positional encoding. Given the deepest encoder feature map $F_L \in \mathbb{R}^{H_L \times W_L \times C_L}$, each spatial location is a node; each node is connected to its $k$ closest nodes in a warped coordinate space $P'_{xy} = P_{xy} + R_{xy}$, where $R_{xy}$ is a sine-cosine positional offset. A graph convolution $h_i^{(t)} = \sigma(\sum_{j \in \mathcal{N}(v_i)} W h_j^{(t-1)} + b)$ then refines the features by aggregating neighbors. This construction is what lets the network relate regions that are far apart in the regular pixel grid but close in the distorted image geometry, which is the paper's explanation for the WoodScape gains.

What would settle it

Retrain U-Net, U-Net++, SwinUNet, and UNet-GNN on the same WoodScape split with the same preprocessing, augmentation, optimizer, learning-rate schedule, and epoch count, then compare mean IoU; if the 0.933 versus 0.878 gap shrinks to noise, the claimed advantage of the GNN bottleneck is not demonstrated.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that a graph-based bottleneck can do what local convolutions and global self-attention do not fully do: it can make segmentation robust to the non-uniform geometry of fisheye images. The authors replace the plain U-Net bottleneck with graph convolutional layers. Each node is a spatial cell of the deepest feature map, edges are drawn to k-nearest neighbors under a warped coordinate system that includes a learned relative positional offset, and node features are updated by aggregating neighbor features. The reported numbers—mean IoU 0.933 on WoodScape, 0.833 on ISIC2016, and 0.774 on PascalVOC—are offered as evidence that the GNN bottleneck raises accuracy over U-Net, U-Net++, and SwinUNet on distorted and boundary-heavy imagery while remaining competitive on a standard natural-image benchmark.

Load-bearing premise

The paper's causal reading of the numbers assumes all compared models were trained and evaluated under identical conditions, with the only difference being the GNN bottleneck; the authors do not report the training configuration that would make this verifiable.

Editorial extensions

If this is right

  • If the WoodScape result holds, GNN bottlenecks are a plausible drop-in upgrade for U-Net-style segmenters in fisheye camera pipelines, where convolutions alone lose accuracy near the image periphery.
  • The same bottleneck could help other tasks that share the same failure mode, such as instance segmentation or depth estimation on wide-angle cameras, because it does not depend on the segmentation head.
  • The gap over SwinUNet on WoodScape suggests that explicit relational modeling over a distortion-aware graph can beat global self-attention when the image geometry is non-uniform.
  • On medical images such as ISIC2016, the reported gains indicate that graph-based feature refinement can help delineate lesions with highly variable shape and boundary, not just distorted driving scenes.
  • The method is architecture-agnostic at the encoder and decoder level, so stronger convolutional backbones could be combined with the graph bottleneck to push accuracy further.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves training details unspecified, so a fair reading is that the reported improvements are conditional on the comparison being controlled; if the baselines were trained with different schedules or augmentation, part of the gap could come from factors other than the graph module.
  • The warped-coordinate k-NN construction suggests a direct test: distort a standard benchmark with a synthetic fisheye model and measure how much of the gain is explained by the positional offset $R_{xy}$ versus the graph aggregation alone.
  • Because the GNN bottleneck operates on a comparatively small feature map, it may be a cheaper route to global context than SwinUNet's shifted-window attention; a parameter and latency comparison would tell whether the accuracy gain comes with a computational win.
  • If the Generalized Wasserstein Dice Loss proposed in the limitations section is incorporated, the model's confusion between rider and person and between bicycle and motorcycle would be the concrete place to look for further gains.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes UNet-GNN, a U-Net variant with a graph neural network (GNN) bottleneck for semantic segmentation. The authors evaluate it on PascalVOC, WoodScape, and ISIC2016 against U-Net, U-Net++, and SwinUNet, reporting higher IoU on WoodScape (0.933) and ISIC2016 (0.833) and competitive performance on PascalVOC (0.774). The method constructs a k-NN graph on the encoder bottleneck features using a warped coordinate space with relative positional encodings, and it applies graph convolution to refine features before decoding. The paper concludes that the GNN bottleneck improves segmentation under geometric distortion and irregular boundaries.

Significance. The core idea—inserting a GNN bottleneck into a U-Net—is a reasonable and potentially useful architectural variant for segmentation under non-Euclidean or distorted image conditions, and the authors make a good-faith attempt to evaluate it across diverse domains. The work also offers a concrete implementation direction (code claimed on GitHub) and an explicit limitation statement. However, the paper's scientific value hinges entirely on the reported numerical results, and those results are internally inconsistent in the main table. As presented, the central claim that UNet-GNN outperforms the baselines on WoodScape is not supported by the manuscript's own evidence. Because the experimental reporting lacks essential training details and statistical grounding, the contribution is currently more a proposal than a validated empirical study.

major comments (4)
  1. [Results, Table 1] Table 1 contains a direct arithmetic contradiction between per-class IoU values and the reported average mIoU. For UNet-GNN on WoodScape, the per-class IoUs are 0.97, 0.94, 0.69, 0.60, 0.44, 0.37, 0.90, 0.43, 0.60, and 0.17; the mean of these values is 0.611, not the reported 0.93. The same problem occurs for U-Net (per-class mean 0.494 vs. reported 0.87), U-Net++ (0.470 vs. 0.81), and SwinUNet (0.552 vs. 0.89). The Average Accuracy row (0.99/0.98/0.98/0.99) is also inconsistent with per-class accuracies that go as low as 0.10 for Traffic Sign. Since the central claim of the paper—the 0.933 IoU on WoodScape—rests on this average row, the evidence for the headline result is invalid as written. The authors must either correct the per-class values or the averages; without this, no comparison between methods can be trusted.
  2. [Results, PascalVOC] The PascalVOC paragraph states, 'SwinUNet attains a higher IoU of 0.774', but the text immediately before gives UNet-GNN an IoU of 0.774 as well. 'Higher' is therefore inaccurate; the two values are equal. This is a minor wording issue, but it also signals a lack of care in reporting that is consistent with the larger Table 1 inconsistency. Please verify all comparative statements against the actual numbers.
  3. [Methodology, Graph Construction and Relative Positional Encoding] The relative positional encoding is described as a 'learned offset Rxy' in the Graph Construction section, yet the next paragraph states that Rxy is 'computed via sine-cosine functions as proposed in [21]'. Sine-cosine positional encodings are typically fixed functions of position, not learned offsets. The paper never clarifies whether Rxy is learned or fixed, and if learned, how it is trained. Since the entire motivating argument for the GNN bottleneck is that warping coordinates helps with fisheye distortion, this ambiguity is not purely cosmetic. Please specify the exact form of Rxy and whether it is an independent learned parameter or a deterministic function.
  4. [Results and Methodology (overall experimental reporting)] The paper reports single-run IoU scores with no error bars, no standard deviations, and no information about training configuration: no optimizer, learning rate, batch size, number of epochs, loss function, data splits, preprocessing, augmentation, or evaluation protocol. Without this information, the reader cannot determine whether the reported differences (e.g., 0.933 vs. 0.893 on WoodScape) are statistically meaningful or whether all models were trained under identical conditions. Since the paper claims that the GNN bottleneck is the cause of the improvement, experimental control is essential. A proper ablation (e.g., U-Net with and without the GNN bottleneck under matched training) would also be needed to isolate the contribution of the graph module.
minor comments (5)
  1. [Table 1] The table header 'Sr . # Categories' contains an extra space, and the rows 'A verage mIoU' and 'A verage Accuracy' have inconsistent spacing; these are typographical issues that should be corrected.
  2. [Results, WoodScape] The text reports SwinUNet's WoodScape IoU as 0.893, but Table 1 lists 0.89. Please ensure consistency between prose and tables.
  3. [Limitations and Future Work] The generalized Wasserstein Dice Loss equation is not properly typeset and lacks the normalization constants typically present in the original formulation. Please provide a complete, correctly formatted equation or a proper citation to the source.
  4. [Overall] The paper states 'Code Available at GitHub' but does not provide a URL or repository identifier. Since reproducibility is a stated goal (and one of the contributions), please provide a working link.
  5. [Introduction] The reference to '[5]' as a general claim about CNNs being sensitive to geometric distortion is fine, but several statements in the Introduction and Related Work are over-broad and would benefit from more specific citations (e.g., the claim that 'CNNs are sensitive to geometric distortions' is not quantified).

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the paper's claims are empirical comparisons against external benchmarks; no derived result reduces to fitted inputs or self-citations.

full rationale

The paper is an empirical architecture-comparison study. Its central claims (UNet-GNN IoU values on PascalVOC, WoodScape, ISIC2016) are reported measurements from training/evaluation, not quantities derived from the model's defining equations or from fitted parameters that are then renamed as predictions. The GNN bottleneck is described by explicit equations (graph construction, k-NN with relative positional encoding, graph convolution Eq. 2, decoder Eqs. 3-4), and there is no step in which an output is shown to be identical to an input by construction. The only self-citations are refs [5] (Kumar et al., including co-author Eising) and [13] (Manzoor et al., including authors Singh and Eising), and these support background claims about fisheye-camera perception and deformable convolution; they are not the evidence for the reported segmentation scores, which are benchmarked against U-Net, U-Net++, and SwinUNet on standard datasets. There is no uniqueness theorem imported from the authors and no ansatz smuggled in via self-citation: the relative positional encoding is attributed to the standard Transformer paper [21]. The Limitations section openly acknowledges class-confusion weaknesses, which is a substantive admission rather than a circular justification. The reader-identified concern about missing training configuration is a reproducibility/experimental-rigor issue, and the WoodScape Table 1 average-vs-per-class discrepancy is an internal arithmetic inconsistency, but neither constitutes derivation-by-construction circularity. Therefore the circularity score is 0.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim rests on an empirical comparison with no formal derivation. The free parameters listed are the key hyperparameters that are not specified. The axioms are assumptions about the architecture, the positional encoding, and the graph aggregation without experimental validation. No invented entities are introduced.

free parameters (3)
  • k in k-NN graph construction
    The number of neighbors per node is not specified; the graph construction and GNN bottleneck depend on this hyperparameter.
  • number of GNN layers
    The GNN bottleneck depth is undefined; the aggregation in Eq. (2) is shown for one layer but the actual depth is not given.
  • learned offset scale for relative positional encoding
    Rxy is described as a learned offset but computed by fixed sine-cosine functions; its scaling or weighting is not defined.
assumptions (3)
  • domain assumption The U-Net encoder-decoder with skip connections and GNN bottleneck is a sufficient architecture for the task.
    The paper assumes the proposed architectural combination will improve segmentation without theoretical proof or ablation.
  • domain assumption The sine-cosine relative positional encoding can capture fisheye distortion geometry.
    Adopted from transformers without validation on fisheye images; the distortion invariance claim is not measured.
  • domain assumption Graph convolution aggregation over k-NN in warped space provides global context.
    The paper assumes that k-NN graph over a bottleneck feature map captures long-range dependencies; no complexity analysis or comparison to other global context modules.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Image Segmentation: Inducing graph-based learning." pith.science (2026). https://pith.science/paper/5UA6RE5C

@misc{pith2026250103765,
  author       = {Pith},
  title        = {Pith review of: Image Segmentation: Inducing graph-based learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5UA6RE5C}},
  note         = {Machine review of arXiv:2501.03765}
}
read the original abstract

This study explores the potential of graph neural networks (GNNs) to enhance semantic segmentation across diverse image modalities. We evaluate the effectiveness of a novel GNN-based U-Net architecture on three distinct datasets: PascalVOC, a standard benchmark for natural image segmentation, WoodScape, a challenging dataset of fisheye images commonly used in autonomous driving, introducing significant geometric distortions; and ISIC2016, a dataset of dermoscopic images for skin lesion segmentation. We compare our proposed UNet-GNN model against established convolutional neural networks (CNNs) based segmentation models, including U-Net and U-Net++, as well as the transformer-based SwinUNet. Unlike these methods, which primarily rely on local convolutional operations or global self-attention, GNNs explicitly model relationships between image regions by constructing and operating on a graph representation of the image features. This approach allows the model to capture long-range dependencies and complex spatial relationships, which we hypothesize will be particularly beneficial for handling geometric distortions present in fisheye imagery and capturing intricate boundaries in medical images. Our analysis demonstrates the versatility of GNNs in addressing diverse segmentation challenges and highlights their potential to improve segmentation accuracy in various applications, including autonomous driving and medical image analysis.

Figures

Figures reproduced from arXiv: 2501.03765 by the authors.

Figure 1
Figure 1. Model Architecture. The network is composed of three main modules: an Encoder, a GNN Bottleneck, and a Decoder. Methodology We propose an enhanced U-Net architecture that incorpo￾rates Graph Neural Network (GNN) layers at the bottleneck. This design leverages the convolutional feature extraction capabilities of U-Net while introducing a graph-based mechanism to capture complex global interdependencies—particularly u… view at source ↗
Figure 2
Figure 2. Visualization of segmentation mask generated by different models under this study neighbors, we add a learned offset Rxy (computed via sine-cosine functions as proposed in [21]) to warp the coordinates: P ′ xy = Pxy +Rxy, where Pxy is the original spatial coordinate of node vxy and Rxy is its corresponding relative positional encoding. This warped co￾ordinate space preserves context about local distortions, allowing… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Graph Neural Network Reveals the Cortical Morphology of Local Brain Aging in Normal Cognition and Alzheimer's Disease

    q-bio.NC 2026-01 conditional novelty 5.0 of 10

    A graph neural network trained on cortical surface morphometry produces vertex-level local brain age maps that show prefrontal/parietal aging in normal cognition and parahippocampal/temporal aging in Alzheimer's disease.

Reference graph

Works this paper leans on

25 extracted references · 8 canonical work pages · cited by 1 Pith paper

  1. [21]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, Attention Is All You Need, arXiv preprint arXiv:1706.03762, 2023. Available at: https://arxiv.org/abs/1706.03762

  2. [1]

    LeCun, Y ., Bottou, L., Bengio, Y ., and Haffner, P. (1998). Gradient-based learning applied to document recognition . Proceedings of the IEEE. Available at: https://doi.org/ 10.1109/5.726791. doi: 10.1109/5.726791

  3. [2]

    Ronneberger, P

    O. Ronneberger, P. Fischer, and T. Brox, U-Net: Convolu- tional Networks for Biomedical Image Segmentation , arXiv preprint arXiv:1505.04597 , 2015. Available at: https:// arxiv.org/abs/1505.04597

  4. [3]

    Z. Zhou, M. M. R. Siddiquee, N. Tajbakhsh, and J. Liang, UNet++: A Nested U-Net Architecture for Medical Image Segmentation, arXiv preprint arXiv:1807.10165, 2018. Avail- able at: https://arxiv.org/abs/1807.10165

  5. [4]

    H. Cao, Y . Wang, J. Chen, D. Jiang, X. Zhang, Q. Tian, and M. Wang, Swin-Unet: Unet-like Pure Trans- former for Medical Image Segmentation , arXiv preprint arXiv:2105.05537, 2021. Available at: https://arxiv. org/abs/2105.05537

  6. [5]

    V . R. Kumar, C. Eising, C. Witt, and S. Yogamani, Surround-view Fisheye Camera Perception for Automated Driving: Overview, Survey and Challenges , arXiv preprint arXiv:2205.13281, 2023. Available at: https://arxiv. org/abs/2205.13281

  7. [6]

    C., Hagenbuchner, M., and Monfardini, G

    Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M., and Monfardini, G. (2009). The Graph Neural Network Model . IEEE Transactions on Neural Networks, 20(1), 61–80. doi: 10.1109/TNN.2008.2005605

  8. [7]

    Han, K., Wang, Y ., Guo, J., Tang, Y ., and Wu, E. (2022). Vision GNN: An Image is Worth Graph of Nodes . arXiv preprint arXiv:2206.00272. Available at: https://arxiv. org/abs/2206.00272

Show all 25 references
  1. [8]

    Griffiths and D

    R. Griffiths and D. G. Dansereau, Adapting CNNs for Fisheye Cameras without Retraining , arXiv preprint arXiv:2404.08187, Apr. 2024. Available at: https:// arxiv.org/abs/2404.08187

  2. [9]

    Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., and Zisserman, A. (2010). The Pascal Visual Object Classes (VOC) Challenge. International Journal of Computer Vision, 88(2), 303–338

  3. [10]

    Yogamani, C

    S. Yogamani, C. Hughes, J. Horgan, G. Sistu, P. Varley, D. O’Dea, M. Uricar, S. Milz, M. Simon, K. Amende, C. Witt, H. Rashed, S. Chennupati, S. Nayak, S. Man- soor, X. Perroton, and P. Perez, WoodScape: A Multi- task, Multi-camera Fisheye Dataset for Autonomous Driv- ing, arX...

  4. [11]

    Gutman, N

    D. Gutman, N. C. F. Codella, E. Celebi, B. Helba, M. Marchetti, N. Mishra, and A. Halpern, Skin Lesion Anal- ysis toward Melanoma Detection: A Challenge at the In- ternational Symposium on Biomedical Imaging (ISBI) 2016, hosted by the International Skin Imaging Collaboration (...

  5. [12]

    Muhammad, T

    K. Muhammad, T. Hussain, H. Ullah, J. Del Ser, M. Rezaei, N. Kumar, M. Hijji, P. Bellavista, and V . H. C. de Albu- querque, Vision-Based Semantic Segmentation in Scene Un- derstanding for Autonomous Driving: Recent Achievements, Challenges, and Outlooks, IEEE Transactions on ...

  6. [13]

    Manzoor, A

    A. Manzoor, A. Singh, G. Sistu, R. Mohandas, E. Grua, A. Scanlan, and C. Eising, Deformable Convolution Based Road Scene Semantic Segmentation of Fisheye Images in Au- tonomous Driving, arXiv preprint arXiv:2407.16647 , 2024. Available at: https://arxiv.org/abs/2407.16647

  7. [14]

    Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo, Swin Transformer: Hierarchical Vi- sion Transformer using Shifted Windows , arXiv preprint arXiv:2103.14030, 2021. Available at: https://arxiv. org/abs/2103.14030

  8. [15]

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N. (2021). An Image is Worth 16x16 Words: Transformers for Im- age Recognition at Scale . arXiv preprint arXi...

  9. [16]

    Q. Zhao, C. Zhu, F. Dai, Y . Ma, G. Jin, and Y . Zhang, Distortion-aware CNNs for Spherical Images , in Proceed- ings of the International Joint Conference on Artificial In- telligence (IJCAI), pp. 1198–1204, 2018

  10. [17]

    Walker, V

    T. Walker, V . Anand, and P. Andreadis, Spherical Fea- ture Pyramid Networks For Semantic Segmentation , arXiv preprint arXiv:2307.02658 , 2023. Available at: https:// arxiv.org/abs/2307.02658

  11. [18]

    Scaramuzza, A

    D. Scaramuzza, A. Martinelli, and R. Siegwart, A Toolbox for Easily Calibrating Omnidirectional Cameras, in Proceed- ings of the 2006 IEEE/RSJ International Conference on In- telligent Robots and Systems , pp. 5695–5701, 2006. DOI: 10.1109/IROS.2006.282372

  12. [19]

    N., and Welling, M

    Kipf, T. N., and Welling, M. (2016). Semi-Supervised Classification with Graph Convolutional Networks . CoRR, abs/1609.02907. Available at: http://arxiv.org/abs/ 1609.02907

  13. [20]

    M. M. Bronstein, J. Bruna, Y . LeCun, A. Szlam, and P. Van- dergheynst, Geometric Deep Learning: Going beyond Eu- clidean data , IEEE Signal Processing Magazine , vol. 34, no. 4, pp. 18–42, Jul. 2017. DOI: 10.1109/MSP.2017. 2693418. Available at: http://dx.doi.org/10.1109/ MSP...

  14. [22]

    H. Wang, H. Yin, M. Zhang, and P. Li, Equivariant and Stable Positional Encoding for More Powerful Graph Neural Networks, arXiv preprint arXiv:2203.00199, 2022. Available at: https://arxiv.org/abs/2203.00199

  15. [23]

    F. Lan, C. Yang, G. Cheung, and J. Z. G. Tan, Joint De- mosaicking / Rectification of Fisheye Camera Images using Multi-color Graph Laplacian Regularization, arXiv preprint arXiv:2006.11636, 2020. Available at: https://arxiv. org/abs/2006.11636

  16. [24]

    Fidon, W

    L. Fidon, W. Li, L. C. Garcia-Peraza-Herrera, J. Ekanayake, N. Kitchen, S. Ourselin, and T. Vercauteren, Generalised Wasserstein Dice Score for Imbalanced Multi-class Segmen- tation Using Holistic Convolutional Networks , DOI: 10. 1007/978-3-319-75238-9_6 . Available at: http:...

  17. [2022]

    DOI: 10.1109/TITS.2022.3207665

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.