REVIEW 3 major objections 7 minor 3 cited by
Distance Map Loss Penalty Term for Semantic Segmentation
T0 review · 3 major / 7 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read Distance-map loss penalty improves 3D knee MRI boundary segmentation.
desk verdict A plausible boundary-weighted loss with an honest comparison, but the reported 'significant improvement' does not survive a t-test on the given numbers. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the distance-map penalty term $\Phi$: a per-voxel weight field derived from the ground-truth segmentation by a distance transform, inverted so that boundary-adjacent voxels get high weight, with the additive $+1$ keeping the loss from vanishing in interiors. Combined with the Hadamard product in $\mathcal{L}=\frac{1}{N}\sum_i(1+\Phi)\odot\sum_j -y_{ij}\log\hat{y}_{ij}$, it redirects gradient effort toward hard, boundary-adjacent errors during the whole training run, rather than as a fine-tuning stage. The paper's argument is carried by comparing this loss against established alternatives on the same architecture and data, with boundary Dice and relaxed boundary Dice at tolerances of one to four voxels as the outcome metrics.
What would settle it
Re-run the same loss comparison and score the trained models with a shape metric that is not derived from ground-truth boundary-distance weighting, for example Hausdorff distance, mean surface distance, or downstream bone-surface curvature and thickness measurements, and see whether the proposed loss still wins. If the boundary Dice advantage disappears or reverses under such an independent metric, the central claim about shape preservation would be refuted.
Extended reading notes
Core claim
The central claim is that training a V-Net on 3D knee MRI with the loss $\mathcal{L}=\frac{1}{N}\sum_{i=1}^{N}(1+\Phi)\odot\sum_{j=1}^{K}-y_{ij}\log\hat{y}_{ij}$ produces segmentations with more accurate bone boundaries than the same architecture trained with Dice loss, focal loss, or a confident-predictions penalty. The map $\Phi$ is built by computing a distance transform on each ground-truth mask and inverting it, so voxels close to the bone surface receive the largest penalty weights; inner maps are computed separately per bone to compensate for size differences among femur, tibia, and patella. In the paper's experiments, the proposed loss reaches a boundary Dice of $28.83\pm4.45\%$, compared with $26.73\pm5.40\%$ for Dice loss, $26.70\pm4.27\%$ for focal loss, and $25.81\pm3.02\%$ for the confident-predictions penalty, with global Dice scores all near 95--96%. The authors interpret this as better shape preservation at bone boundaries and in partial-volume regions, and state their downstream goal of extracting more reliable shape biomarkers.
Load-bearing premise
The load-bearing premise is that Boundary Dice, a score that rewards voxels lying within a small tolerance of the ground-truth surface, actually captures the shape fidelity that matters for clinical biomarkers; if that metric is not measuring clinically relevant shape, the reported boundary improvement could fail to translate into better morphology estimates.
Editorial extensions
If this is right
- A boundary-weighted cross-entropy loss can be dropped into existing segmentation networks without architectural changes, so the same training pipeline can be reused across anatomies.
- Because the penalty is built directly from ground-truth distance maps, it extends naturally to multi-class 3D problems and to highly imbalanced datasets where boundary voxels are a tiny fraction of the volume.
- If the reported boundary Dice gain holds, shape-sensitive downstream measurements such as bone surface area, thickness, or curvature estimates should become more reproducible with the proposed loss.
- The comparison suggests that explicit boundary weighting outperforms attention strategies that suppress confident predictions globally, at least on knee-bone MRI.
Reading between the lines
- A natural extension not tested here is to anneal the weight of $\Phi$ over training, since keeping the boundary penalty at full strength throughout may over-constrain late-stage refinement.
- Because the training penalty and the boundary evaluation metric are both built from ground-truth edge distance, the numerical gain could be partly a metric-alignment effect; checking the same models with a metric that does not use boundary-distance weighting, such as Hausdorff distance or surface-to-surface error, would tell whether actual shape fidelity improves.
- The same distance-map construction could be used as a post-hoc weighting for other losses, or as a regularizer on feature maps, which might transfer the boundary benefit to architectures that cannot train with the multi-class cross-entropy term directly.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a distance-map-derived penalty term added to multi-class cross-entropy for 3D semantic segmentation of knee bones in MRI. The penalty uses distance transforms of ground-truth masks to up-weight voxels near bone boundaries, under the intuition that boundary regions are the hardest to segment. The method is evaluated on 40 manually segmented patients from the OAI dataset with a 25/5/10 train/validation/test split, comparing against soft-Dice loss, focal loss, and the confident-prediction penalty of Pereyra et al. The main reported result is an improvement in boundary Dice (B-DSC) of 28.83±4.45% versus 26.73±5.40% for Dice loss, with the authors describing this as significant and also reporting small gains in global Dice. The paper is a short MIDL extended abstract with an appendix of additional results.
Significance. If the reported improvement is real, the contribution is a simple, low-cost modification to a standard cross-entropy loss that improves boundary fidelity, which is directly relevant to shape-biomarker extraction in musculoskeletal imaging. The method is straightforward to implement, is formulated for multi-class 3D segmentation, and is compared against several established loss functions on a public dataset. The main strengths are the simplicity of the idea and the concrete experimental comparison; however, the evidence is thin. The dataset is small (40 patients), there is a single train/validation/test split with no repeated seeds, no statistical significance test is provided, and the central claim rests on a difference that may be within chance variation. The metric-alignment issue (the loss and the B-DSC metric are both built from boundary-distance information) is an additional concern that should be addressed with complementary metrics.
major comments (3)
- [§3, Results and Conclusions] The claim that "B-DSC of our proposed loss shows a significant improvement" is not supported by any statistical test. Using the reported means and standard deviations for B-DSC (28.83±4.45% vs 26.73±5.40%) with n=10 per group, an unpaired Welch's t-test gives t≈0.95, p≈0.36, which is far above conventional significance levels. Even with a paired design, the paper provides no per-subject data or correlation measure to establish significance. Because the entire central claim of the paper depends on this comparison, the authors must either provide a proper statistical analysis (e.g., paired test on per-subject scores, or results over multiple splits/runs) or downscale the claim to an observed trend. This is a load-bearing issue, not a presentation detail.
- [§2, Methods and Experiments] The experimental design uses only 40 unique patients with a single 25/5/10 split and no repeated training runs or cross-validation. With only 10 test patients, the reported 2-point B-DSC difference could easily arise from split selection or stochastic training variability. The authors should report per-subject scores, or results across multiple random seeds and/or splits, to demonstrate that the improvement is stable. Without this, the reliability of the headline result is unknown.
- [§3, Figure 3 and evaluation metrics] The evaluation metric B-DSC is computed from boundary-distance information that is essentially the same information used to construct the training penalty Φ in Eq. (2). The reported improvement may therefore reflect alignment between the training objective and the metric rather than a general improvement in segmentation quality or shape preservation. To establish generalizability, the paper should also report boundary-agnostic metrics (e.g., volumetric Dice, symmetric surface distance, Hausdorff distance) and, ideally, downstream shape-biomarker reproducibility. Even if the B-DSC improvement were statistically significant, this alignment would still limit the interpretation of that single metric.
minor comments (7)
- [Eq. (2)] The notation (1 + Φ) ⊙ Σ_j −y_j log ŷ_j is ambiguous because Φ is defined per voxel but the summation over classes is not; please clarify that Φ is a per-voxel weight and define N and the coordinate indexing explicitly.
- [§2, distance map construction] The description "by voxel-wise subtracting the binary segmentation from the mask overall max distance value" is unclear; rewriting this step with a concrete formula or pseudocode would improve reproducibility.
- [§3, baselines] The paper compares against Dice, focal, and Pereyra losses but does not include plain cross-entropy, despite the proposed loss being a weighted form of cross-entropy. Including a standard cross-entropy baseline would isolate the effect of the weighting.
- [Abstract and §3] The adjective "significant" is used without a test in both the abstract and the results section; either provide a test or replace it with a more neutral phrase such as "observed improvement".
- [Figure 2] Figure 2 shows error maps for a single test patient; a quantitative summary across all test patients, or at least a few representative examples, would be more informative.
- [References] The MATLAB citation "Users Guide Matlab. The mathworks. Inc., Natick, MA, 1992, 1760" appears malformed; please correct the year and title.
- [§2, bone size statement] The phrase "the femur being 1.6 and 16 times larger than tibia and patella respectively" is ambiguous; please clarify the reference comparisons (e.g., femur vs tibia and femur vs patella).
Circularity Check
No circularity: the boundary-aware loss and boundary Dice metric are aligned but not identical, and the reported comparison is on held-out test data.
full rationale
The paper does not derive a prediction from its inputs; it reports an empirical comparison of loss functions. Equation (2) defines a penalty term using distance maps derived from training ground-truth masks, while the primary evaluation metric B-DSC is computed from held-out test ground-truth boundaries. These are related in spirit, but the loss is not the same function as the metric, and the test-set B-DSC is an output rather than a fitted input. The only self-citation, Norman et al. 2018 (with overlapping authors Pedoia and Majumdar), is used for describing the OAI MR acquisition and prior cartilage/meniscus segmentation work; it is not load-bearing for the central claim about the proposed loss. The paper's assertion of a 'significant improvement' without a significance test is a statistical-validity concern, not a circularity concern, and therefore does not change the circularity score. The derivation chain is self-contained in the sense that the proposed loss is defined independently of the reported boundary Dice results, and the comparison is made on data not used to construct the loss.
Assumptions & free parameters
free parameters (3)
- Unit offset in loss weight (1 + Phi) =
1
- Distance map inversion scale =
max distance value of each mask
- Per-bone inner distance map combination weights =
Size ratios 1.6 and 16
assumptions (3)
- domain assumption Ground truth manual segmentations are accurate enough to define true boundaries and distance maps.
- domain assumption Boundary proximity, as encoded by the inverted distance transform, is a valid proxy for segmentation difficulty and clinical shape preservation.
- domain assumption V-Net with Adam and random in-plane rotation training reproduces the baselines reliably enough for comparison.
Cite this review
Pith. "Pith review of Distance Map Loss Penalty Term for Semantic Segmentation." pith.science (2026). https://pith.science/paper/4FCZV5GE
@misc{pith2026190803679,
author = {Pith},
title = {Pith review of: Distance Map Loss Penalty Term for Semantic Segmentation},
year = {2026},
howpublished = {\url{https://pith.science/paper/4FCZV5GE}},
note = {Machine review of arXiv:1908.03679}
}
read the original abstract
Convolutional neural networks for semantic segmentation suffer from low performance at object boundaries. In medical imaging, accurate representation of tissue surfaces and volumes is important for tracking of disease biomarkers such as tissue morphology and shape features. In this work, we propose a novel distance map derived loss penalty term for semantic segmentation. We propose to use distance maps, derived from ground truth masks, to create a penalty term, guiding the network's focus towards hard-to-segment boundary regions. We investigate the effects of this penalizing factor against cross-entropy, Dice, and focal loss, among others, evaluating performance on a 3D MRI bone segmentation task from the publicly available Osteoarthritis Initiative dataset. We observe a significant improvement in the quality of segmentation, with better shape preservation at bone boundaries and areas affected by partial volume. We ultimately aim to use our loss penalty term to improve the extraction of shape biomarkers and derive metrics to quantitatively evaluate the preservation of shape.
Figures
Forward citations
Cited by 3 Pith papers
-
Critical edge sets in vertex-critical graphs
Erdős's 1985 growth question for critical edge sets in k-vertex-critical graphs is answered affirmatively for every k > 4 via f_k(n) = Ω(n^(1/3)), and a first upper bound f_k(n) = O(n/(log n)^Ω(1)) is given for all k ≥ 4.
-
Learning Pore-scale Multi-phase Flow from Experimental Data with Graph Neural Network
LSE-MGN, a graph neural network with long- and short-edge message passing, predicts pore-scale gas/liquid evolution from experimental micro-CT data with roughly 9 to 10 percent surface-area error over short autoregres...
-
A Novel Shape Guided Transformer Network for Instance Segmentation in Remote Sensing Images
A shape-guided transformer with row and column global attention reaches state-of-the-art instance segmentation accuracy on three remote sensing datasets.
Reference graph
Works this paper leans on
-
[1]
Tensorflow: A system for large-scale machine learning
Mart \' n Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. Tensorflow: A system for large-scale machine learning. In OSDI, volume 16, pages 265--283, 2016
work page 2016
-
[2]
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770--778, 2016
2016
-
[3]
Boundary loss for highly unbalanced segmentation
Hoel Kervadec, Jihene Bouchtiba, Christian Desrosiers, \'E ric Granger, Jose Dolz, and Ismail Ben Ayed. Boundary loss for highly unbalanced segmentation. arXiv preprint arXiv:1812.07032, 2018
work page Pith review arXiv 2018
-
[4]
Adam: A method for stochastic optimization
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014
arXiv 2014
-
[5]
Focal loss for dense object detection
Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Doll \'a r. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, pages 2980--2988, 2017
2017
- [6]
-
[7]
V-net: Fully convolutional neural networks for volumetric medical image segmentation
Fausto Milletari, Nassir Navab, and Seyed-Ahmad Ahmadi. V-net: Fully convolutional neural networks for volumetric medical image segmentation. In 2016 Fourth International Conference on 3D Vision (3DV), pages 565--571. IEEE, 2016
work page 2016
-
[8]
Berk Norman, Valentina Pedoia, and Sharmila Majumdar. Use of 2d u-net convolutional neural networks for automated cartilage and meniscus segmentation of knee mr imaging data to determine relaxometry and morphometry. Radiology, 288 0 (1): 0 177--185, 2018
work page 2018
Show all 10 references
-
[9]
Regularizing neural networks by penalizing confident output distributions
Gabriel Pereyra, George Tucker, Jan Chorowski, ukasz Kaiser, and Geoffrey Hinton. Regularizing neural networks by penalizing confident output distributions. arXiv preprint arXiv:1701.06548, 2017
2017 arXiv
-
[10]
U-net: Convolutional networks for biomedical image segmentation
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention, pages 234--241. Springer, 2015
2015
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.