{"id":"f7545d4e-56b8-41fb-80db-69a54f466086","arxiv_id":"2412.04094","paper_version":3,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"An ensemble of nnU-Net, MedNeXt and SwinUNETR with radiomic-subtype-tuned post-processing achieves whole-tumor lesion-wise Dice of 0.926, 0.801 and 0.688 on the BraTS 2024 PED, MEN-RT and MET test sets.","lead":"The paper reports a BraTS 2024 entry that ensembles three standard MRI segmentation models and then applies radiomics-based tumor subtyping to tune post-processing thresholds per subgroup. Its test-set Dice numbers are solid, but the benefit of the subtyping step itself is not isolated in the reported experiments.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper's claimed novelty is the adaptive radiomic-subtype post-processing, yet validation tables show essentially no improvement over the plain ensemble (PED WT 0.931→0.931, MEN-RT GTV 0.794→0.794, MET WT 0.717→0.719), and no ablation against a global-threshold or no-post-processing baseline is…","rationale":"The reader's formal weakest_assumption concerns transfer of k-means subtype structure to the test distribution, which is a real risk, but the results tables expose an even more basic problem: the adaptive post-processing appears to have little or no effect on the validation set before any distribution-shift question is considered. If the post-processing step changes outcomes only at the third decimal or not at all, then even perfect cluster transfer would not support the paper's 'enhances precision and generalizability' claim. The reader's rationale does note the absence of an ablation against a global-threshold baseline and the tiny deltas, so there is substantial agreement, but the stated weakest_assumption is narrower than mine. I do not see a reason to reject the paper: the ensemble itself is a legitimate BraTS submission, the official test scores are trustworthy, code and a web demo are provided, and the missing evidence is one controlled comparison. The conditional verdict is appropriate: the paper should be credited for its benchmark contribution, but the central method claim should not be accepted without an ablation. My recommendation is therefore to leave the reader's CONDITIONAL verdict unchanged.","tokens_in":8717,"tokens_out":2527,"duration_ms":28152,"concrete_test":"Run the released HOPE-Segmenter-Kids pipeline on the BraTS 2024 validation sets in three configurations: (i) ensemble prediction only, (ii) ensemble plus a single global small-component-removal threshold, and (iii) ensemble plus the proposed k-means subtype-adaptive post-processing. Compare lesion-wise Dice and HD95 on the official challenge platform. If configuration (iii) does not statistically significantly outperform (ii) on WT Dice for PED, MEN-RT, and MET, the abstract's claim that the radiomic subtyping approach 'enhances precision and generalizability' should be removed or softened.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The benchmark scores themselves are credible: they come from the official BraTS platform, the authors state they never accessed validation ground truth or test labels, and the code is public. The load-bearing weakness is attribution of the central claim. The abstract and conclusions assert that the proposed adaptive pre- and post-processing 'enhances the precision and generalizability of segmentation models,' but the only within-paper evidence for the post-processing step are the 'Post-processing' rows in Tables 1–3. In Table 1 (PED validation), post-processing changes ensemble lesion-wise WT Dice by 0.931→0.931, ET by 0.68→0.665 (a decrease), and only ED improves meaningfully (0.956→0.967). In Table 2 (MEN-RT), both ensemble and post-processing give GTV Dice 0.794, i.e., no effect. In Table 3 (MET), post-processing changes ET/TC/WT from 0.739/0.756/0.717 to 0.739/0.755/0.719, again negligible. Because the k-means subtype clustering is used both for stratified fold splitting and for deriving per-subtype post-processing thresholds (Sections 3.1 and 3.4), the reader cannot tell whether the adaptive radiomic component contributes anything beyond a standard ensemble. A second, related gap is that the paper provides no cluster-stability evidence; if the k-means structure does not transfer from training radiomics (computed on ground-truth WT) to test-time predictions (computed on predicted WT), the per-subtype thresholds could even be misapplied. But the more immediate issue is that no configuration without the adaptive post-processing is evaluated, so the claimed enhancement is currently an assertion rather than a demonstrated result.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports a segmentation pipeline developed for the BraTS 2024 PED, MEN-RT, and MET challenges. The method combines three architectures (nnU-Net, MedNeXt, SwinUNETR) with weighted probability ensembling, and uses MRI radiomic features extracted with PyRadiomics from the largest whole-tumor connected component to cluster tumor subtypes via PCA and k-means. These subtypes are used both for stratified five-fold splitting and for per-subtype adaptive post-processing that removes small disconnected components and relabels regions. The authors present official validation and test lesion-wise Dice and HD95 metrics, with mean test whole-tumor Dice of 0.926 (PED), 0.801 (MEN-RT), and 0.688 (MET), and state that the adaptive pre- and post-processing enhances precision and generalizability.","tokens_in":9142,"tokens_out":5145,"duration_ms":49324,"significance":"If the radiomic-subtyping contribution were supported, the paper would be a useful demonstration of subtype-adaptive ensembling across three tumor domains. The benchmark results are credible: they come from the official BraTS evaluation platform, the authors state they had no access to validation ground truth or test labels, and the source code and a web application are publicly released. The main weakness is attribution: the validation tables do not show that the adaptive post-processing improves over the plain ensemble, and no ablation or stability analysis isolates the radiomic component. The paper is therefore better characterized as a strong challenge report than as evidence for the claimed methodological novelty.","major_comments":[{"comment":"The central claim that the adaptive radiomic pre/post-processing 'enhances the precision and generalizability of segmentation models' is not supported by the validation results. In Table 1, post-processing changes PED lesion-wise Dice from 0.680 to 0.665 for ET (a decrease), leaves WT unchanged at 0.931, and improves only ED from 0.956 to 0.967. In Table 2, MEN-RT GTV Dice is 0.794 before and after post-processing. In Table 3, MET values are essentially unchanged (WT 0.717 to 0.719, TC 0.756 to 0.755). Because no ablation against a global-threshold or no-post-processing baseline is reported, these tables cannot support the abstract's attribution. Please provide such an ablation with confidence intervals or significance testing, or revise the claims accordingly.","section":"Abstract; Section 5; Tables 1-3"},{"comment":"The k-means subtype model is trained on radiomic features extracted from the ground-truth whole-tumor masks of the training set, but at inference the subtype assignment uses predicted whole-tumor masks from the ensemble. The paper gives no evidence that the cluster structure transfers from ground-truth to predicted masks or across acquisition sites or protocols, and it reports no cluster-stability metrics. Since the adaptive post-processing thresholds are optimized per cluster on training cross-validation, unstable cluster assignments could make the adaptive step harm rather than help. Please report cluster sizes, within-cluster threshold variability, stability across folds, or a comparison with cluster-agnostic thresholds.","section":"Sections 3.1 and 3.4"},{"comment":"The Discussion states that weighted ensembles outperform simple averaged ensembles, but the paper does not show any comparison between the weighted ensemble and an unweighted average, nor does it quantify uncertainty in the fitted weights. For MET the weight for SwinUNETR is set to zero based on validation, and for PED the weights are nearly equal (0.33/0.34/0.33); the claimed advantage of weighting is therefore not demonstrated by the reported tables. Either include the comparison or soften the statement.","section":"Equation (1); Section 5"}],"minor_comments":[{"comment":"\"constrast-enhanced\" should be \"contrast-enhanced\".","section":"Section 2, PED paragraph"},{"comment":"The table headers contain formatting artifacts such as \"V alidation\" and \"T able\"; these should be fixed.","section":"Tables 1 and 2"},{"comment":"The ensemble equation uses \"Yensemble\" with inconsistent spacing; use \"Y_ensemble\" and consistent math formatting.","section":"Section 3.3"},{"comment":"The model name is written inconsistently as \"nnUNet\" and \"nnU-Net\"; please standardize.","section":"Throughout"},{"comment":"The figure legend mentions \"SNFH\" and labels not discussed in the MET evaluation table or text; clarify the relationship between these visual labels and the reported ET/TC/WT metrics.","section":"Figure 4"},{"comment":"There are several typographical spacing issues (e.g., \"T o\", \"V alidation\"); a careful proofreading pass is recommended.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"This is a competently executed challenge submission, and the official test numbers are valuable, but the paper's novelty claim is currently overstated relative to its own tables. The authors' previous related works (refs. 7 and 14) should be more clearly differentiated from the present contribution. In my view the paper is suitable for publication after a revision that adds the missing ablations and recalibrates the abstract and conclusion."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Take on arXiv:2412.04094: this is an honest challenge-report paper with real held-out numbers and public code. The test-set Dice scores (0.926 PED WT, 0.801 MEN-RT GTV, 0.688 MET WT) come from the official BraTS platform, and the authors explicitly state they had no access to validation ground truth or test labels, so the benchmark figures deserve credence.\n\nWhat's actually new: the three base models (nnU-Net, MedNeXt, SwinUNETR) are published and ensembling is standard BraTS practice. The one novel ingredient is the k-means radiomic subtyping used to set per-cluster post-processing thresholds, extending the authors' ISBI 2024 work (ref 14) to PED, MEN-RT, and MET. That specific combination is not in the cited literature. Code and a web application are released, which is reproducible.\n\nThe soft spot is the attribution of the central claim. The abstract and conclusion say the adaptive pre/post-processing 'enhances precision and generalizability,' but the validation tables show essentially no change from post-processing: PED WT 0.931 to 0.931, ET actually drops from 0.68 to 0.665, MEN-RT GTV stays at 0.794, MET WT moves 0.717 to 0.719. There is no ablation against a global threshold or a no-post-processing configuration, so the reader cannot tell whether the radiomic subtyping contributes anything beyond a standard weighted ensemble. The paper also does not test cluster stability: k-means is fit on ground-truth WT radiomics and applied to predicted WT at inference, and that transfer is assumed rather than checked.\n\nThese are correctable omissions, not analytical errors. The circularity burden is low because ensemble weights and thresholds are tuned on training cross-validation and the test split is independent. The citation pattern is normal; self-citations (refs 7 and 14) describe prior methods on which this work builds.\n\nBottom line: worth a serious referee as a BraTS challenge paper if the bar for acceptance is a reproducible, competitive pipeline with clearly labeled claims. But the authors should add a controlled ablation before the adaptive post-processing claim can stand. If I were reviewing, I'd ask for that as a major revision, not reject.","headline":"Credible BraTS 2024 benchmark results, but the adaptive radiomic post-processing that the paper headlines is not shown to do anything: the tables show no meaningful delta and there is no ablation.","tokens_in":9699,"tokens_out":2587,"would_cite":false,"duration_ms":24915,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A weighted ensemble of three deep learning models, with post-processing tuned per radiomic tumor subtype, reports whole-tumor Dice of 0.926 (PED), 0.801 (MEN-RT), and 0.688 (MET) on BraTS 2024 final test sets.","keywords":["brain tumor segmentation","MRI","deep learning","pediatric brain tumors","meningiomas","metastases","radiomics","model ensemble"],"falsifier":"On an external multi-institutional MRI dataset, compare the full pipeline against the same ensemble with a single global post-processing threshold; the central claim fails if the per-subtype version does not improve mean lesion-wise Dice, or if k-means clusters on held-out data do not reproduce the training cluster structure under bootstrap resampling.","tokens_in":8429,"feed_emoji":"🧠","tokens_out":7356,"duration_ms":61932,"temperature":0.7,"pith_summary":"The paper sets out to establish that brain tumor segmentation improves when a weighted ensemble of complementary deep learning models is combined with post-processing that adapts to the tumor subtype. It derives subtypes from MRI radiomic features—shape and intensity statistics of the largest whole-tumor component—reduced by PCA and clustered with k-means, then learns per-cluster threshold rules that remove small false-positive components and relabel tumor regions. On the BraTS 2024 final test sets, the pipeline reports mean lesion-wise whole-tumor Dice scores of 0.926 for pediatric brain tumors, 0.801 for meningioma radiotherapy volumes, and 0.688 for brain metastases. If the claim holds, the method is a competitive, code-released recipe for multi-tumor segmentation and a template for using imaging phenotypes to tailor segmentation post-processing.","feed_headline":"Radiomic subtyping lifts brain tumor segmentation to 0.926 Dice","feed_subtitle":"A three-model ensemble with per-subtype post-processing generalizes across pediatric tumors, meningioma, and metastases.","key_machinery":"The load-bearing mechanism is adaptive post-processing conditioned on radiomic tumor subtypes. For each task, 14 shape and 93 intensity features per MRI sequence are extracted from the largest connected component of the whole-tumor region, reduced by PCA to 9, 3, and 2 features for PED, MEN-RT, and MET, and grouped into clusters by k-means with the cluster count chosen by silhouette analysis. The ensemble prediction—a weighted average of probability maps from nnU-Net, MedNeXt, and SwinUNETR, with weights set from five-fold cross-validation—is then refined by per-cluster threshold searches that remove small disconnected components and relabel regions whose size ratio to the whole tumor falls below a threshold. For MET, SwinUNETR receives zero weight because of its markedly lower validation performance.","core_discovery":"The central claim is that a weighted ensemble of nnU-Net, MedNeXt, and SwinUNETR—with weights estimated from five-fold cross-validation—combined with MRI radiomic tumor subtyping and adaptive post-processing, enhances the precision and generalizability of brain tumor segmentation across heterogeneous tumor types. The authors report that the weighted ensemble outperforms simple probability averaging, and that per-subtype post-processing reduces false positives better than a one-size-fits-all rule. They demonstrate the pipeline on three BraTS 2024 challenges, reporting final test-set mean lesion-wise Dice scores of 0.926 for pediatric whole tumor, 0.801 for meningioma gross tumor volume, and 0.688 for metastatic whole tumor, with code and a web application released for independent use.","pith_inferences":["The k-means clusters likely correspond to interpretable imaging phenotypes such as lesion size, enhancement pattern, or edema burden, but the paper does not characterize them; naming these clusters could add clinical insight beyond segmentation accuracy.","The adaptive thresholds are learned from cross-validated training predictions, so on a new scanner or site the thresholds may need recalibration; an external multi-site study would show whether the per-subtype gains persist in practice.","The same subtype-then-post-process design could be tested on other lesion segmentation problems, such as stroke or lung nodules, where heterogeneity limits one-size-fits-all post-processing."],"forward_implications":["If correct, weighted ensembling of architecturally diverse models reliably improves segmentation over any single model, even when one architecture underperforms on a task.","The radiomic-subtyping plus per-cluster threshold recipe offers a transferable way to handle tumor heterogeneity without hand-crafting rules for each dataset.","The reported test-set Dice scores provide a competitive baseline for BraTS 2024 and a practical starting point for volumetric tumor measurement in clinical workflows.","Because the code and a demonstration web application are released, the method can be independently reproduced and tested on external clinical datasets, which the authors explicitly call for."],"supporting_citations":[{"why":"Supplies the PyRadiomics package used to extract the 14 shape and 93 intensity radiomic features per MRI sequence.","marker":"[1]"},{"why":"Provides the model-ensemble strategy for aggregating probability outputs, extended here with weighted averaging.","marker":"[7]"},{"why":"Provides the nnU-Net self-configuring segmentation framework that one of the three ensemble models is based on.","marker":"[12]"},{"why":"Supplies the radiomic-feature implementation used for tumor subtyping, cited alongside PyRadiomics.","marker":"[13]"},{"why":"The adaptive post-processing strategy this paper follows, with per-cluster threshold search on cross-validated predictions.","marker":"[14]"},{"why":"Introduces MedNeXt, the hybrid CNN-attention architecture used as one of the ensemble members.","marker":"[23]"},{"why":"Introduces Swin UNETR, the transformer-based architecture used as one of the ensemble members.","marker":"[10]"}],"fun_headline_variants":["Radiomic subtyping with ensemble nets 0.926 Dice on pediatric tumors","Three-model MRI ensemble scores 0.926 Dice on pediatric brain tumors","Ensemble and subtyping hit 0.926 lesion Dice for pediatric tumors","Per-subtype post-processing and ensemble: 0.926 Dice on pediatric tumors"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The k-means tumor-subtype clusters are learned from radiomic features of the training set, and the per-cluster post-processing thresholds are applied to test predictions, assuming that both the cluster structure and the optimal thresholds transfer from training and validation to the unseen test distribution.","fun_headline_variants_meta":{"raw":{"variants":["Radiomic subtyping with ensemble nets 0.926 Dice on pediatric tumors","Three-model MRI ensemble scores 0.926 Dice on pediatric brain tumors","Ensemble and subtyping hit 0.926 lesion Dice for pediatric tumors","Per-subtype post-processing and ensemble: 0.926 Dice on pediatric tumors"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001372,"raw_usage":{"total_tokens":5597,"prompt_tokens":1015,"completion_tokens":4582,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":631,"completion_tokens_details":{"reasoning_tokens":4496}},"tokens_in":631,"tokens_out":4582,"duration_ms":30485,"temperature":1.0,"reasoning_tokens":4496,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T21:44:59.765501+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On an external multi-institutional MRI dataset, compare the full pipeline against the same ensemble with a single global post-processing threshold; the central claim fails if the per-subtype version does not improve mean lesion-wise Dice, or if k-means clusters on held-out data do not reproduce the training cluster structure under bootstrap resampling.","supporting_citations":[{"cited_title":"Cancer Research77(21), e104–e107 (2017)","cited_arxiv_id":null,"evidence_quote":"Supplies the PyRadiomics package used to extract the 14 shape and 93 intensity radiomic features per MRI sequence."},{"cited_title":"In: Crimi, A., Bakas, S","cited_arxiv_id":null,"evidence_quote":"Introduces Swin UNETR, the transformer-based architecture used as one of the ensemble members."}],"review_version":1}