Pith. sign in

REVIEW 2 major objections 6 minor 22 references

HERMES: A Hybrid Ensemble for Head-and-Neck Tumor Segmentation, TN Staging, and Recurrence-Free Survival on PET/CT

T0 review · 2 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The N-stage signal in head-and-neck PET/CT lives in node size and number, not in radiomics texture.

desk verdict A careful HECKTOR challenge write-up whose main N-stage claim is honestly reported as a trend, with a real but non-fatal selection-on-evaluation caveat. read the letter →

arxiv 2607.26498 v2 pith:KVZP4MMW submitted 2026-07-29 cs.CV physics.med-ph

Kai Wang , Meixu Chen , Elie Nasr , Ryan Lanning , Moyed Miften This is my paper
classification cs.CVphysics.med-ph
keywords headandneckcancerPET/CTtumorsegmentationTNMstagingsurvivalpredictionradiomicsgeometryfeaturesconcordanceloss
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that features used for downstream clinical tasks should mirror the definitions of those tasks. For neck-lymph-node staging, it replaces a 30-dimensional intensity/texture radiomics vector with seven geometry features read from the predicted segmentation mask—node count, largest-node size, total burden, and related terms—that align with the size and number axes of the 7th-edition radiological N categories. On internal cross-validation this raises N-stage balanced accuracy from 0.691 to 0.720, a favorable but not statistically significant gain; the same comparison on ground-truth masks (0.897 versus 0.837) is significant, showing the effect is a property of the geometry signal, diluted by imperfect segmentation. If this holds, automated staging of head-and-neck cancer can become more accurate, more interpretable, and less dependent on scanner-sensitive radiomics, with the main bottleneck being segmentation quality rather than feature design.

What carries the argument

The load-bearing mechanism is the geometry block: seven numbers computed from the connected-component-filtered predicted lymph-node mask—component count, largest-component volume and axis-aligned extent, dominance fraction, total nodal burden, and log-volume transforms—paired with a logistic regression using strong sparsity regularization. These features encode the same size and number axes that define the 7th-edition N category, whereas the radiomics baseline encodes intensity and texture; L1 selection keeps only total burden, largest-node extent, and dominance, so the decision is reducible to a clinician-inspectable rule. A secondary mechanism is the concordance-tracking loss, a smooth pai

What would settle it

Take a large, independent cohort whose lymph nodes are annotated individually, with no merged conglomerate masses, compute the seven geometry features and the 30 radiomics features from those ground-truth masks, and run the same strongly regularized N-stage classifier for each feature set. If the paired difference in balanced accuracy between geometry and radiomics has a confidence interval that includes zero, or reverses when components are split into individual nodes, the claim that size and number carry the nodal-staging signal is wrong. A second targeted check: add a laterality or neck-lev

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that the size and number axes of the predicted nodal segmentation carry more of the N-staging signal than a generic intensity/texture radiomics vector does, and that this can be exploited with seven compact geometry features. Replacing the 30-dimensional radiomics block in the N-stage classifier raises balanced accuracy on held-out patients from 0.691 to 0.720, with a paired bootstrap interval that includes zero; the authors therefore call it a trend rather than a proven improvement. The same replacement evaluated on ground-truth masks produces 0.897 versus 0.837, with a confidence interval that excludes zero, which they read as evidence that the ge

Load-bearing premise

The result rests on the premise that the connected components of the predicted lymph-node mask correspond to the individual nodes a radiologist would count and measure; if segmentation merges several nodes into one mass or invents spurious nodes, the geometry features may look good or bad for reasons unrelated to true size and number.

Editorial extensions

If this is right

  • Nodal staging can be served by a hand-sized set of interpretable descriptors—node count, largest-node size, total burden—instead of a 30-dimensional radiomics vector, with no accuracy loss and a possible gain.
  • Because the ground-truth-mask ceiling (0.897) far exceeds the deployed value (0.720), the next large accuracy jump for automated N-staging should come from segmentation quality, not from richer features.
  • If the size/number axes carry most of the nodal signal, research effort on N-staging should invest in reliable detection and disambiguation of individual nodes, including laterality and neck level, rather than intensity/texture descriptors.
  • A survival loss whose training value is an interpretable approximation of the concordance index makes early stopping and model selection transparent, which is useful in small-data clinical settings.
  • The full pipeline—segmentation, staging, and survival—can run in one container within a modest GPU memory and time budget, so the geometry-staging result is not just offline evidence but a deployable clinical workflow.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The two-by-two design (feature type versus mask source) generalizes as a diagnostic protocol: comparing proposed features on ground-truth versus predicted masks separates feature validity from segmentation error, a practice other image-based staging tasks could borrow.
  • Laterality-aware features—splitting nodal components by left/right or by a neck-level atlas—are the natural next test; the paper's own ceiling analysis implies most of the residual N-stage gap is the missing laterality axis and merged masses.
  • The concordance-tracking loss idea is not limited to survival; any smooth, differentiable surrogate whose value monotonically tracks a discrete evaluation metric could offer the same interpretable-training-signal benefit for ranking, detection, or calibration tasks.
  • A monotone relationship between predicted-mask geometry and true stage could be inverted into segmentation quality control: a predicted node count or largest-node size far outside the clinical range flags an over- or under-segmentation before a human reviews the case.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. HERMES is a containerized pipeline for the HECKTOR 2026 challenge, jointly performing GTVp/GTVn segmentation from PET/CT, radiological T/N staging, and recurrence-free survival prediction. Segmentation uses a 10-fold STU-Net Small ensemble with connected-component post-processing; the predicted masks drive downstream staging and survival modules. The paper's principal scientific claim is that compact, staging-aligned geometry features extracted from the predicted masks (nodal count, largest-node extent, burden) outperform a 30-dimensional radiomics descriptor for N-staging at lower dimensionality. On internal out-of-fold cross-validation, replacing radiomics with geometry raises N-stage balanced accuracy from 0.691 to 0.720, with a paired 95% CI of [-0.008, +0.067]; on ground-truth masks the gap is 0.897 vs 0.837 with a CI excluding zero. The manuscript also introduces a concordance-tracking survival loss, whose value tracks the C-index during training, and reports validation leaderboard scores (Mean Dice 0.641, T bal. acc. 0.580, N bal. acc. 0.642, RFS C-index 0.679). The paper is transparent about the statistical uncertainty of its main comparison and does not tune on the public validation set.

Significance. If the central claim holds, it is a clinically meaningful and arguably transferable result: simple, interpretable mask-derived features aligned with the size/number axes of staging definitions can outperform generic radiomics at lower dimensionality, and the full pipeline is deployable on a T4 GPU. The paper's strengths include explicit reporting of bootstrap confidence intervals, a clear distinction between trend and significance, a base/enhanced ablation design, release of code, and the use of the validation set only for final assessment. However, the main geometry-versus-radiomics comparison is evaluated on the same out-of-fold predictions that were used to select features, thresholds, and hyperparameters; the reported gains, including the significant ground-truth-mask gap, are therefore subject to selection-on-evaluation bias and do not yet constitute a truly independent out-of-sample test. The deployable CI includes zero, and the paper's own caveats already limit the strength of the claim.

major comments (2)
  1. [§2.4, §3, Table 3] The central N-staging comparison (0.691 vs 0.720, and the ground-truth-mask rows 0.837 vs 0.897) is evaluated on pooled out-of-fold predictions from the 782-patient training cohort, while the geometry feature set, the connected-component thresholds (GTVn ≥ 500 mm3, top-8, SUV peak ≥ 2.5), and the L1 regularization C=0.03 were all fixed on these same OOF predictions (as stated in §2.1, §2.2, §2.4). This is not a nested validation design, so the reported differences may be optimistically biased. The deployable paired 95% CI [-0.008, +0.067] already includes zero. To support the claim that geometry 'outperforms' radiomics, the authors should provide a nested cross-validation in which feature/threshold/hyperparameter selection is performed inside each training fold, or a held-out subset of the training cohort that does not participate in any selection. Without such an independent estimate, t
  2. [§3, Table 3] The ground-truth-mask analysis is presented as the most telling evidence ('geometry beats radiomics as features significantly'), but it is a post-hoc mechanism probe on the same cohort, using the same selected features and thresholds at each feature set's own best regularization. It does not provide independent confirmation of the deployed configuration because it shares the selection pathway with the OOF evaluation. The paper should either explicitly demote this analysis to an exploratory mechanism check or validate it on a separate dataset; as written, its evidential weight for the deployable claim is overstated.
minor comments (6)
  1. [Abstract] The standalone abstract (first page) states that the N-stage gain is 'our largest single design gain' without the caveat 'paired interval includes zero' that appears in the full-text abstract. Align the two abstracts so the statistical uncertainty is reported consistently.
  2. [§3] The sentence '0.95 of the paired bootstrap difference lies above zero' is inconsistent with the reported 95% CI [-0.008, +0.067], which contains zero. Clarify the bootstrap procedure: if reporting a percentile interval, the lower bound should correspond to the 2.5th percentile, not the 5th. This affects readability of the key result.
  3. [§2.2] The staging feature description does not specify whether the SUV gate (peak ≥ 2.5) is applied before or after the connected-component size filter, and how these interact for the geometry features. Please make the exact processing order explicit.
  4. [§2, Table 1] The notation '963/1123' for patch sizes is ambiguous; it likely means 96³ and 112³. Use proper superscripts or spell out '96x96x96' and '112x112x112'.
  5. [§2.4] The term 'regularization-oriented protocol' is used repeatedly but never operationally defined. Specify the decision criteria (e.g., when a more regularized model is preferred over a marginally better point estimate) so the protocol is reproducible.
  6. [§3, Fig. 3] Figure 3b shows error bars/regions but the caption does not state what they represent (presumably 95% CIs). Add explicit legend/caption text explaining whether the displayed intervals are unpaired or paired.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central staging comparison is against external challenge labels, and the self-cited concordance loss is fully defined and non-load-bearing.

full rationale

The derivation chain is segmentation -> predicted masks -> geometry features -> N-stage classifier -> comparison with external challenge-provided TN labels and a radiomics baseline. The geometry features (nodal count, largest-node extent, burden) intentionally mirror the size/number axes of AJCC/UICC 7th-edition N staging, but the stage labels are not computed from these features; they are organizer-provided radiological stages, and the masks are imperfect predicted outputs. The reported N-stage gain of +0.030 (95% CI [-0.008, +0.067]) and the ground-truth-mask gain of +0.060 (95% CI [+0.028, +0.093]) are empirical comparisons, not identities following from feature construction. The concordance-tracking loss is self-cited (ref [14] includes two present authors), but Eq. (1) defines the loss in full, and the paper explicitly reports that its accuracy advantage over Cox loss is within noise across seeds; it is retained for an interpretable training signal, not as a fitted target. Hyperparameters such as connected-component thresholds and C=0.03 were fixed on OOF predictions, which is a selection-on-evaluation risk (not nested), but this is a statistical optimism concern, not construction-equivalent circularity: the OOF predictions are genuine held-out predictions and the labels are external. No equation reduces an output to a fitted parameter, no load-bearing claim rests solely on a self-citation, and no known result is merely renamed.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central N-stage argument uses no invented entities. It relies on challenge labels, reference masks, and several post-processing thresholds fitted on OOF predictions. The survival loss is a self-citation from prior work, which adds a mild circularity burden but does not affect the N-stage comparison.

free parameters (5)
  • GTVp connected-component size threshold = 1000 mm3
    Fixed by an OOF grid sweep; removes small primary components.
  • GTVn connected-component size threshold = 500 mm3
    Fixed by an OOF grid sweep; defines the nodal component size floor.
  • Nodal SUV gate = 2.5 SUV peak
    Peak SUV threshold for GTVn components; fixed on OOF.
  • N-stage L1 regularization C = 0.03
    Chosen for stability on OOF; controls the sparsity of the logistic regression.
  • Concordance loss temperature tau = 0.1
    Sigmoid temperature in the pairwise survival loss; chosen by hand.
assumptions (5)
  • domain assumption Challenge-provided radiological TN labels under AJCC/UICC 7th edition are internally consistent and correct.
    Section 2 Data; the entire staging evaluation rests on these labels.
  • domain assumption Connected-component and SUV-gate post-processing on predicted masks yields components that correspond to individual nodal masses.
    Section 2.1; the paper notes conglomerate masses merge and the monotone trend is empirical.
  • domain assumption Ground-truth masks in the challenge are accurate enough to serve as a mechanism ceiling.
    Table 3 rows; used to separate feature quality from mask quality.
  • domain assumption The concordance-tracking surrogate loss from prior work [14] approximates 1 minus C-index well enough to guide early stopping.
    Section 2.3; adopted from the authors' own prior paper.
  • domain assumption The eight-center OOF split approximates generalization to unseen centers.
    Section 2.4; the hidden test includes previously unseen centers.

how reviews work

0 comments
Cite this review

Pith. "Pith review of HERMES: A Hybrid Ensemble for Head-and-Neck Tumor Segmentation, TN Staging, and Recurrence-Free Survival on PET/CT." pith.science (2026). https://pith.science/paper/KVZP4MMW

@misc{pith2026260726498,
  author       = {Pith},
  title        = {Pith review of: HERMES: A Hybrid Ensemble for Head-and-Neck Tumor Segmentation, TN Staging, and Recurrence-Free Survival on PET/CT},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KVZP4MMW}},
  note         = {Machine review of arXiv:2607.26498}
}
read the original abstract

We present HERMES (Hybrid Ensemble for Radiotherapy-target segmentation, Malignancy staging, and Event-free Survival), a single containerized algorithm for the three HECKTOR 2026 subtasks: segmentation of the primary tumor (GTVp) and pathological lymph nodes (GTVn), radiological T/N staging, and recurrence-free survival (RFS), computed from a paired FDG-PET/CT scan and an electronic health record. A 10-fold ensemble of STU-Net Small networks produces the segmentation; the predicted mask then drives two downstream tasks. Rather than pass a generic radiomics vector to the staging models, we derive from the predicted masks a compact set of geometry features aligned with the size and number axes of AJCC/UICC 7th-edition radiological N/T staging. On internal cross-validation these features raise N-stage balanced accuracy from 0.691 to 0.720 (+0.030), our largest single design gain, at lower feature dimensionality. For prognosis we combine complementary deep and clinical risk experts in an equal-weight ensemble, and train one deep expert with a concordance-tracking survival loss of our own, whose value approximates the concordance index during training. Every component was selected on honest out-of-fold predictions under a regularization-oriented protocol, with no tuning on the public validation set, and deployed as two decorrelated submissions. On the HECKTOR 2026 validation leaderboard, HERMES achieved a weighted score of 0.6454 (Mean Dice 0.641, T balanced accuracy 0.580, N balanced accuracy 0.642, RFS C-index 0.679) and qualified for the testing phase. Team: AMC_HNC.

Figures

Figures reproduced from arXiv: 2607.26498 by the authors.

Figure 1
Figure 1. HERMES pipeline. FDG-PET/CT and EHR feed a 10-fold STU-Net segmen￾tation; the predicted mask then drives TN staging (deep, radiomics, and geometry experts) and RFS (a deep survival ensemble with a concordance-loss expert plus a clinical Cox model). 2.1 Segmentation We use STU-Net Small (about 14 M parameters) [6], a scalable residual U-Net reported to match or exceed nnU-Net on large-scale benchmarks, which we adopt… view at source ↗
Figure 2
Figure 2. Predicted (filled: red GTVp, cyan GTVn) versus ground-truth (contours: yellow GTVp, green GTVn) segmentation on the maximum-tumor axial slice for one repre￾sentative patient from each of the eight training centers (deployed-model out-of-fold predictions). that it removes low-uptake nodal false positives at no overlap cost. Predictions are visually consistent with the reference across all eight centers, including the… view at source ↗
Figure 3
Figure 3. (a) Predicted-mask nodal geometry (largest-node extent, node count) is mono￾tone in ground-truth N-stage. (b) N-stage balanced accuracy (out-of-fold, patient￾level bootstrap 95% CIs): clinical plus geometry (0.720) versus clinical plus radiomics (0.691). ceiling. This is our largest design effect. For T-staging, adding primary-tumor geometry to the deep and radiomics fusion moves balanced accuracy from 0.444 to 0.45… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: The concordance loss value is an interpretable, metric-aligned training signal (representative fold, identical architecture and data). (a) Its training value stays on the scale of 1 − C-index and falls with the validation concordance throughout training. (b) Across all…

Review history (2 revisions) →

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

22 extracted references · 19 canonical work pages

  1. [1]

    arXiv:2509.00367 (2025)

    Saeed, N., et al.: A Multimodal and Multi-centric Head and Neck Cancer Dataset for Segmentation, Diagnosis and Outcome Prediction. arXiv:2509.00367 (2025)

    arXiv 2025

  2. [2]

    HEad and neCK TumOR (HECKTOR) 2025: Benchmark of Segmentation, Diagnosis, and Prognosis in Multimodal PET/CT

    Saeed, N., et al.: HEad and neCK TumOR (HECKTOR) 2025: Benchmark of Seg- mentation, Diagnosis, and Prognosis in Multimodal PET/CT. arXiv:2606.20143 (2026)

    work page Pith review arXiv 2025

  3. [3]

    Medical Image Analysis 77, 102336 (2022)

    Oreiller, V., et al.: Head and neck tumor segmentation in PET/CT: The HECK- TOR challenge. Medical Image Analysis 77, 102336 (2022)

    work page 2022

  4. [4]

    In: MICCAI, LNCS 9351, pp

    Ronneberger, O., Fischer, P., Brox, T.: U-Net: Convolutional Networks for Biomed- ical Image Segmentation. In: MICCAI, LNCS 9351, pp. 234–241 (2015)

    work page 2015

  5. [5]

    Nature Methods 18, 203–211 (2021)

    Isensee, F., et al.: nnU-Net: a self-configuring method for deep-learning-based biomedical image segmentation. Nature Methods 18, 203–211 (2021)

    work page 2021

  6. [6]

    arXiv:2304.06716 (2023)

    Huang, Z., et al.: STU-Net: Scalable and Transferable Medical Image Segmenta- tion Models Empowered by Large-Scale Supervised Pre-training. arXiv:2304.06716 (2023)

    arXiv 2023

  7. [7]

    In: BrainLes, LNCS 12962, pp

    Hatamizadeh, A., et al.: Swin UNETR: Swin Transformers for Semantic Segmen- tation of Brain Tumors in MRI Images. In: BrainLes, LNCS 12962, pp. 272–284 (2022)

    work page 2022

  8. [8]

    Cancer Research 77(21), e104–e107 (2017)

    van Griethuysen, J.J.M., et al.: Computational Radiomics System to Decode the Radiographic Phenotype. Cancer Research 77(21), e104–e107 (2017)

    work page 2017

Show all 22 references
  1. [9]

    Scientific Reports 7, 10117 (2017)

    Vallières, M., et al.: Radiomics strategies for risk assessment of tumour failure in head-and-neck cancer. Scientific Reports 7, 10117 (2017)

    2017

  2. [10]

    In: 3D Head and Neck Tumor Segmentation in PET/CT Challenge (HECKTOR), LNCS, vol

    Wang, K., et al.: Recurrence-free survival prediction under the guidance of auto- matic gross tumor volume segmentation for head and neck cancers. In: 3D Head and Neck Tumor Segmentation in PET/CT Challenge (HECKTOR), LNCS, vol. 13209, pp. 144–153. Springer (2022)

    2022

  3. [11]

    Journal of the Royal Statistical Society B 34(2), 187–220 (1972)

    Cox, D.R.: Regression Models and Life-Tables. Journal of the Royal Statistical Society B 34(2), 187–220 (1972)

    1972

  4. [12]

    BMC Medical Research Methodology 18, 24 (2018)

    Katzman, J.L., et al.: DeepSurv: personalized treatment recommender system using a Cox proportional hazards deep neural network. BMC Medical Research Methodology 18, 24 (2018)

    2018

  5. [13]

    In: AAAI (2018)

    Lee, C., Zame, W.R., Yoon, J., van der Schaar, M.: DeepHit: A Deep Learning Approach to Survival Analysis with Competing Risks. In: AAAI (2018)

    2018

  6. [14]

    arXiv:2607.16802 (2026).https://github.com/ Meixu-Chen/sigmoid-concordance-loss

    Chen, M., Wang, K., Wang, J.: Value-Monotonicity Matters: A Concordance Loss for Deep Survival Prediction. arXiv:2607.16802 (2026).https://github.com/ Meixu-Chen/sigmoid-concordance-loss

    2026 arXiv

  7. [15]

    Wiley-Blackwell (2009)

    Sobin, L.H., Gospodarowicz, M.K., Wittekind, C.: TNM Classification of Malig- nant Tumours, 7th edn. Wiley-Blackwell (2009)

    2009

  8. [16]

    JAMA 247(18), 2543– 2546 (1982)

    Harrell, F.E., et al.: Evaluating the yield of medical tests. JAMA 247(18), 2543– 2546 (1982)

    1982

  9. [17]

    Statistics in Medicine 30(10), 1105–1117 (2011)

    Uno, H., Cai, T., Pencina, M.J., D’Agostino, R.B., Wei, L.J.: On the C-statistics for evaluating overall adequacy of risk prediction procedures with censored survival data. Statistics in Medicine 30(10), 1105–1117 (2011)

    2011

  10. [18]

    arXiv:2211.02701 (2022)

    Cardoso, M.J., et al.: MONAI: An open-source framework for deep learning in healthcare. arXiv:2211.02701 (2022)

    2022 arXiv

  11. [19]

    In: CVPR (2016)

    He, K., et al.: Deep Residual Learning for Image Recognition. In: CVPR (2016)

    2016

  12. [20]

    Radiology 295(2), 328–338 (2020)

    Zwanenburg, A., et al.: The Image Biomarker Standardization Initiative: Stan- dardized Quantitative Radiomics for High-Throughput Image-based Phenotyping. Radiology 295(2), 328–338 (2020)

    2020

  13. [21]

    Biostatistics 8(1), 118–127 (2007)

    Johnson, W.E., Li, C., Rabinovic, A.: Adjusting batch effects in microarray ex- pression data using empirical Bayes methods. Biostatistics 8(1), 118–127 (2007)

    2007

  14. [22]

    The Lancet Oncology 17(4), 440–451 (2016)

    O’Sullivan, B., Huang, S.H., et al.: Development and validation of a staging sys- tem for HPV-related oropharyngeal cancer by the International Collaboration on Oropharyngeal cancer Network for Staging (ICON-S): a multicentre cohort study. The Lancet Oncology 17(4), 440–451 (2016)

    2016

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.