{"id":"025bc757-97a1-4e68-b0cf-e477d855c24e","arxiv_id":"1908.03679","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A distance-map-weighted cross-entropy loss improves boundary Dice in 3D knee bone segmentation compared with Dice, focal, and confidence-penalty losses.","lead":"This paper proposes a way to make image-segmentation networks pay more attention to object edges by weighting the standard loss with a distance map. The authors report better edge accuracy for bone segmentation in knee MRI scans compared with three common loss functions.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported boundary-Dice improvement (28.83 vs 26.73) is not supported by significance testing; with 10 test subjects and overlapping standard deviations, the difference is within chance (unpaired t ≈ 0.95, p ≈ 0.35).","rationale":"The reader's weakest assumption focuses on whether B-DSC captures clinically relevant shape preservation, given that both the loss and the metric use boundary-distance information. That is a valid secondary concern about interpretation and generalizability. However, the more load-bearing issue is statistical: the paper explicitly claims a 'significant improvement' but provides no significance test. The summary statistics themselves suggest the difference is not significant, so the empirical foundation of the central claim is shaky. This is a concrete, internally checkable issue rather than a matter of consensus. I therefore identify the lack of demonstrated statistical significance as the single most load-bearing concern. The proposed concrete test—obtaining per-subject values and running a paired permutation test—would settle whether the improvement is real. Because the reader already made CONDITIONAL the verdict and listed significance testing as a requirement, my analysis does not change the verdict; it reinforces the condition. Agreement is partial because the reader's stated weakest assumption is different, though the reader's rationale also mentions the absence of significance testing.","tokens_in":3601,"tokens_out":6407,"duration_ms":69151,"concrete_test":"Obtain the per-test-subject B-DSC values for the proposed loss and the Dice baseline from the authors (or re-run the experiment on the same 10 held-out subjects) and compute a paired permutation test on the difference. If the two-sided p-value exceeds 0.05, the claim of significant improvement is refuted. If per-subject data are unavailable, compute a Welch's t-test from the reported summary statistics, which already yields p≈0.36 and thereby fails to support the claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3 states 'B-DSC of our proposed loss shows a significant improvement' but provides no statistical test. Using the reported means and SDs for B-DSC (28.83±4.45 vs 26.73±5.40) with n=10 per group, an unpaired Welch's t-test gives t≈0.95, p≈0.36, far above 0.05. Even allowing for a paired design, no per-subject data or correlation is provided to establish significance. The central claim is the improvement from the proposed loss, and the entire empirical support rests on this comparison, so the claim is currently unsubstantiated. This is a correctness risk, not a disagreement with consensus. The metric-alignment issue (loss and B-DSC both use boundary distance) is secondary; the immediate problem is that the measured effect is not shown to be real. Additionally, a single 25/5/10 split with no repeated seeds means the variance of the training procedure is unknown, further undermining the reliability of the reported improvement.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a distance-map-derived penalty term added to multi-class cross-entropy for 3D semantic segmentation of knee bones in MRI. The penalty uses distance transforms of ground-truth masks to up-weight voxels near bone boundaries, under the intuition that boundary regions are the hardest to segment. The method is evaluated on 40 manually segmented patients from the OAI dataset with a 25/5/10 train/validation/test split, comparing against soft-Dice loss, focal loss, and the confident-prediction penalty of Pereyra et al. The main reported result is an improvement in boundary Dice (B-DSC) of 28.83±4.45% versus 26.73±5.40% for Dice loss, with the authors describing this as significant and also reporting small gains in global Dice. The paper is a short MIDL extended abstract with an appendix of additional results.","tokens_in":3805,"tokens_out":2815,"duration_ms":30580,"significance":"If the reported improvement is real, the contribution is a simple, low-cost modification to a standard cross-entropy loss that improves boundary fidelity, which is directly relevant to shape-biomarker extraction in musculoskeletal imaging. The method is straightforward to implement, is formulated for multi-class 3D segmentation, and is compared against several established loss functions on a public dataset. The main strengths are the simplicity of the idea and the concrete experimental comparison; however, the evidence is thin. The dataset is small (40 patients), there is a single train/validation/test split with no repeated seeds, no statistical significance test is provided, and the central claim rests on a difference that may be within chance variation. The metric-alignment issue (the loss and the B-DSC metric are both built from boundary-distance information) is an additional concern that should be addressed with complementary metrics.","major_comments":[{"comment":"The claim that \"B-DSC of our proposed loss shows a significant improvement\" is not supported by any statistical test. Using the reported means and standard deviations for B-DSC (28.83±4.45% vs 26.73±5.40%) with n=10 per group, an unpaired Welch's t-test gives t≈0.95, p≈0.36, which is far above conventional significance levels. Even with a paired design, the paper provides no per-subject data or correlation measure to establish significance. Because the entire central claim of the paper depends on this comparison, the authors must either provide a proper statistical analysis (e.g., paired test on per-subject scores, or results over multiple splits/runs) or downscale the claim to an observed trend. This is a load-bearing issue, not a presentation detail.","section":"§3, Results and Conclusions"},{"comment":"The experimental design uses only 40 unique patients with a single 25/5/10 split and no repeated training runs or cross-validation. With only 10 test patients, the reported 2-point B-DSC difference could easily arise from split selection or stochastic training variability. The authors should report per-subject scores, or results across multiple random seeds and/or splits, to demonstrate that the improvement is stable. Without this, the reliability of the headline result is unknown.","section":"§2, Methods and Experiments"},{"comment":"The evaluation metric B-DSC is computed from boundary-distance information that is essentially the same information used to construct the training penalty Φ in Eq. (2). The reported improvement may therefore reflect alignment between the training objective and the metric rather than a general improvement in segmentation quality or shape preservation. To establish generalizability, the paper should also report boundary-agnostic metrics (e.g., volumetric Dice, symmetric surface distance, Hausdorff distance) and, ideally, downstream shape-biomarker reproducibility. Even if the B-DSC improvement were statistically significant, this alignment would still limit the interpretation of that single metric.","section":"§3, Figure 3 and evaluation metrics"}],"minor_comments":[{"comment":"The notation (1 + Φ) ⊙ Σ_j −y_j log ŷ_j is ambiguous because Φ is defined per voxel but the summation over classes is not; please clarify that Φ is a per-voxel weight and define N and the coordinate indexing explicitly.","section":"Eq. (2)"},{"comment":"The description \"by voxel-wise subtracting the binary segmentation from the mask overall max distance value\" is unclear; rewriting this step with a concrete formula or pseudocode would improve reproducibility.","section":"§2, distance map construction"},{"comment":"The paper compares against Dice, focal, and Pereyra losses but does not include plain cross-entropy, despite the proposed loss being a weighted form of cross-entropy. Including a standard cross-entropy baseline would isolate the effect of the weighting.","section":"§3, baselines"},{"comment":"The adjective \"significant\" is used without a test in both the abstract and the results section; either provide a test or replace it with a more neutral phrase such as \"observed improvement\".","section":"Abstract and §3"},{"comment":"Figure 2 shows error maps for a single test patient; a quantitative summary across all test patients, or at least a few representative examples, would be more informative.","section":"Figure 2"},{"comment":"The MATLAB citation \"Users Guide Matlab. The mathworks. Inc., Natick, MA, 1992, 1760\" appears malformed; please correct the year and title.","section":"References"},{"comment":"The phrase \"the femur being 1.6 and 16 times larger than tibia and patella respectively\" is ambiguous; please clarify the reference comparisons (e.g., femur vs tibia and femur vs patella).","section":"§2, bone size statement"}],"recommendation":"major_revision","confidential_remarks":"The paper is a short MIDL extended abstract, so a modest amount of evidence may be acceptable for the venue, but the explicit claim of statistical significance is unsupported by the reported numbers. The authors should be encouraged to add per-subject data or repeated-split experiments, or to temper the claim. The metric-alignment concern is worth raising but is secondary to the missing significance testing."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nHere's the short take: this is a clean, honest extension of Kervadec et al.'s boundary loss to 3D multi-class segmentation, with a broader comparison than most short papers bother with. But the central claim—that the proposed loss gives a 'significant improvement' in boundary Dice—is not supported by the numbers as reported. The means and standard deviations for B-DSC (28.83±4.45 vs 26.73±5.40, n=10) give an unpaired t ≈ 0.95, p ≈ 0.36. No significance test is reported anywhere. That is the load-bearing issue.\n\nWhat the paper does well: it is straightforward about the relationship to prior work, the method is simple to reproduce (weighted cross-entropy with a distance-map weight), and the authors test against three reasonable baselines: Dice loss, focal loss, and a confidence penalty. The motivation—better boundary preservation for shape biomarkers—is plausible, and the qualitative error maps in Figure 2 support the direction of the effect, if not its magnitude.\n\nThe soft spots beyond the missing test: no plain cross-entropy baseline, so we cannot tell whether the weighting helps over the most basic loss; a single 25/5/10 split with no repeated seeds; and the metric-alignment issue you noticed—both the loss and B-DSC are built from boundary distance, so part of the gain may reflect design alignment rather than a general segmentation quality improvement. That last one is worth a sentence in a revision, but it is secondary to the statistical problem.\n\nWho should read this: people working on boundary-aware losses in medical imaging will want to know this exists as a data point. It is an extended abstract, so I would not cite it as strong evidence. But it deserves a serious referee: the idea is sensible, the comparison is useful, and the missing statistics are fixable. If I were the editor, I would send it to review rather than desk-reject, and ask the authors to add a proper significance test, a cross-entropy ablation, and details on how the split was chosen.\n\nBest,\n[Your name]","headline":"A plausible boundary-weighted loss with an honest comparison, but the reported 'significant improvement' does not survive a t-test on the given numbers.","tokens_in":4356,"tokens_out":2561,"would_cite":false,"duration_ms":25480,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Distance-map loss penalty improves 3D knee MRI boundary segmentation.","keywords":["penalized loss","bone segmentation","distance maps","knee MRI","magnetic resonance imaging","3D convolutional neural networks","boundary segmentation","Dice score"],"falsifier":"Re-run the same loss comparison and score the trained models with a shape metric that is not derived from ground-truth boundary-distance weighting, for example Hausdorff distance, mean surface distance, or downstream bone-surface curvature and thickness measurements, and see whether the proposed loss still wins. If the boundary Dice advantage disappears or reverses under such an independent metric, the central claim about shape preservation would be refuted.","tokens_in":3400,"feed_emoji":"🦴","tokens_out":6525,"duration_ms":66076,"temperature":0.7,"pith_summary":"This paper is trying to establish that a simple modification to the training loss can fix the weakest part of a deep-learning segmentation pipeline: the borders of anatomical structures. The proposed loss adds a distance-map penalty term, derived from the ground-truth masks, to a multi-class cross-entropy loss, so that prediction errors near bone boundaries are weighted more heavily than errors in the interior. Why this matters is clinical: knee MRI segmentation is used to track shape and morphology biomarkers, and those biomarkers are most sensitive to how well the network preserves surfaces, especially where partial-volume effects blur tissue boundaries. On a 40-patient knee-bone dataset, the paper reports that this boundary-weighted loss improves boundary Dice from $26.73\\pm5.40\\%$ with Dice loss to $28.83\\pm4.45\\%$, while leaving global Dice essentially unchanged.","feed_headline":"Distance-map penalty sharpens bone boundaries in 3D knee MRI","feed_subtitle":"Adding a distance-map penalty to the loss lifts boundary Dice on knee MRI while global Dice holds.","key_machinery":"The load-bearing object is the distance-map penalty term $\\Phi$: a per-voxel weight field derived from the ground-truth segmentation by a distance transform, inverted so that boundary-adjacent voxels get high weight, with the additive $+1$ keeping the loss from vanishing in interiors. Combined with the Hadamard product in $\\mathcal{L}=\\frac{1}{N}\\sum_i(1+\\Phi)\\odot\\sum_j -y_{ij}\\log\\hat{y}_{ij}$, it redirects gradient effort toward hard, boundary-adjacent errors during the whole training run, rather than as a fine-tuning stage. The paper's argument is carried by comparing this loss against established alternatives on the same architecture and data, with boundary Dice and relaxed boundary Dice at tolerances of one to four voxels as the outcome metrics.","core_discovery":"The central claim is that training a V-Net on 3D knee MRI with the loss $\\mathcal{L}=\\frac{1}{N}\\sum_{i=1}^{N}(1+\\Phi)\\odot\\sum_{j=1}^{K}-y_{ij}\\log\\hat{y}_{ij}$ produces segmentations with more accurate bone boundaries than the same architecture trained with Dice loss, focal loss, or a confident-predictions penalty. The map $\\Phi$ is built by computing a distance transform on each ground-truth mask and inverting it, so voxels close to the bone surface receive the largest penalty weights; inner maps are computed separately per bone to compensate for size differences among femur, tibia, and patella. In the paper's experiments, the proposed loss reaches a boundary Dice of $28.83\\pm4.45\\%$, compared with $26.73\\pm5.40\\%$ for Dice loss, $26.70\\pm4.27\\%$ for focal loss, and $25.81\\pm3.02\\%$ for the confident-predictions penalty, with global Dice scores all near 95--96%. The authors interpret this as better shape preservation at bone boundaries and in partial-volume regions, and state their downstream goal of extracting more reliable shape biomarkers.","pith_inferences":["A natural extension not tested here is to anneal the weight of $\\Phi$ over training, since keeping the boundary penalty at full strength throughout may over-constrain late-stage refinement.","Because the training penalty and the boundary evaluation metric are both built from ground-truth edge distance, the numerical gain could be partly a metric-alignment effect; checking the same models with a metric that does not use boundary-distance weighting, such as Hausdorff distance or surface-to-surface error, would tell whether actual shape fidelity improves.","The same distance-map construction could be used as a post-hoc weighting for other losses, or as a regularizer on feature maps, which might transfer the boundary benefit to architectures that cannot train with the multi-class cross-entropy term directly."],"forward_implications":["A boundary-weighted cross-entropy loss can be dropped into existing segmentation networks without architectural changes, so the same training pipeline can be reused across anatomies.","Because the penalty is built directly from ground-truth distance maps, it extends naturally to multi-class 3D problems and to highly imbalanced datasets where boundary voxels are a tiny fraction of the volume.","If the reported boundary Dice gain holds, shape-sensitive downstream measurements such as bone surface area, thickness, or curvature estimates should become more reproducible with the proposed loss.","The comparison suggests that explicit boundary weighting outperforms attention strategies that suppress confident predictions globally, at least on knee-bone MRI."],"supporting_citations":[{"why":"Introduces the earlier distance-based boundary loss that the proposed approach extends by training from the start in 3D multi-class settings.","marker":"Kervadec et al., 2018"},{"why":"Supplies the V-Net architecture and the Dice-loss baseline that the proposed penalty is compared against.","marker":"Milletari et al., 2016"},{"why":"Provides the focal-loss baseline used in the performance comparison.","marker":"Lin et al., 2017"},{"why":"Provides the confident-predictions penalizing loss baseline used in the performance comparison.","marker":"Pereyra et al., 2017"},{"why":"Documents the knee MRI acquisition protocol and prior segmentation work that define the data setting.","marker":"Norman et al., 2018"},{"why":"Establishes the encoder-decoder segmentation architecture context that V-Net builds on.","marker":"Ronneberger et al., 2015"}],"fun_headline_variants":["Distance-map penalty improves bone boundary Dice in knee MRI","Boundary-aware loss boosts knee MRI segmentation edges","New loss term sharpens segmentation boundaries on 3D knee MRI","Penalty loss preserves bone shape in knee MRI segmentation","Distance transform loss lifts boundary accuracy in medical imaging"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that Boundary Dice, a score that rewards voxels lying within a small tolerance of the ground-truth surface, actually captures the shape fidelity that matters for clinical biomarkers; if that metric is not measuring clinically relevant shape, the reported boundary improvement could fail to translate into better morphology estimates.","fun_headline_variants_meta":{"raw":{"variants":["Distance-map penalty improves bone boundary Dice in knee MRI","Boundary-aware loss boosts knee MRI segmentation edges","New loss term sharpens segmentation boundaries on 3D knee MRI","Penalty loss preserves bone shape in knee MRI segmentation","Distance transform loss lifts boundary accuracy in medical imaging"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000561,"raw_usage":{"total_tokens":2668,"prompt_tokens":952,"completion_tokens":1716,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":568,"completion_tokens_details":{"reasoning_tokens":1639}},"tokens_in":568,"tokens_out":1716,"duration_ms":11358,"temperature":1.0,"reasoning_tokens":1639,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T14:05:32.603135+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the same loss comparison and score the trained models with a shape metric that is not derived from ground-truth boundary-distance weighting, for example Hausdorff distance, mean surface distance, or downstream bone-surface curvature and thickness measurements, and see whether the proposed loss still wins. If the boundary Dice advantage disappears or reverses under such an independent metric, the central claim about shape preservation would be refuted.","supporting_citations":[{"cited_title":"Boundary loss for highly unbalanced segmentation","cited_arxiv_id":"1812.07032","evidence_quote":"Introduces the earlier distance-based boundary loss that the proposed approach extends by training from the start in 3D multi-class settings."},{"cited_title":"V-net: Fully convolutional neural networks for volumetric medical image segmentation","cited_arxiv_id":null,"evidence_quote":"Supplies the V-Net architecture and the Dice-loss baseline that the proposed penalty is compared against."},{"cited_title":"Use of 2d u-net convolutional neural networks for automated cartilage and meniscus segmentation of knee mr imaging data to determine relaxometry and morphometry","cited_arxiv_id":null,"evidence_quote":"Documents the knee MRI acquisition protocol and prior segmentation work that define the data setting."}],"review_version":1}