{"id":"ee1274fe-c92f-4a26-9892-8be6b1aafb75","arxiv_id":"2607.20028","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"low","formal_verification":"none","parameter_count":3,"one_line_summary":"For knee MRI meniscus segmentation, intensity normalisation choice causes ~1% external DSC differences while dataset shift causes ~10%, so normalisation is a minor lever for generalisability.","lead":"This paper compares seven intensity normalisation methods for knee MRI meniscus segmentation and finds small but significant performance differences on external test data, with Z-score, Nyúl histogram matching, and CLAHE most robust. The choice of normalisation matters far less than the domain shift between datasets.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No raw/normalisation-free arm: 'intensity normalisation had a measurable effect' is asserted without a control; all conclusions compare seven transformations, not the effect of normalisation vs none.","rationale":"The reader's weakest_assumption focuses on GMM component selection across DESS/qDESS. That is a legitimate methodological concern, but it is not the most load-bearing issue for the paper's central claim: GMM is one of the lower-performing methods externally, so an unstable component choice could move GMM by a few tenths of a DSC point without changing the main ranking of Nyúl, Z-score, and CLAHE or the domain-shift conclusion. The missing no-normalisation control is more central because the abstract and conclusion make a claim about 'intensity normalisation' as a category, while the experiment only provides evidence about the relative ordering of seven chosen transforms. Adding a raw/default arm would directly test whether the entire family of methods helps, hurts, or is neutral relative to doing nothing. The paper deserves credit for its public code, use of two public datasets, five-fold repeated measurements, and honest LMM-based statistical analysis; the domain-shift effect is well supported. The concern is therefore a condition for strengthening the claim, not a reason to reject the work. I agree partially with the reader because the no-normalisation baseline was already listed in the rationale, but it was not the reader's selected weakest assumption.","tokens_in":11001,"tokens_out":4290,"duration_ms":47619,"concrete_test":"Add an eighth arm using the same 5-fold nnU-Net pipeline with no explicit intensity normalisation method applied—raw DESS/qDESS inputs, still RSS-combined and cropped as in §2.2, but relying only on nnU-Net's default preprocessing rather than any of the seven schemes. Compute DSC, HD95, CCC, and MAE on both internal and external test sets, and include the arm as a reference level in the LMM of Eq. (1). If the raw arm performs within the 95% CIs of the seven methods or exceeds several, the claim that intensity normalisation has a measurable effect must be weakened to 'differences among normalisation methods are measurable but small.' If raw is clearly worse, the benchmark should report this to quantify the true effect size of normalisation relative to domain shift.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that normalisation choice has a small but measurable effect on external generalisability (marginal R² = 0.005, ~0.1–0.6% DSC differences), far smaller than the ~10% domain-shift drop. But every arm of the experiment applies one of seven preprocessing transformations. There is no arm that forgoes the method-specific normalisation, i.e., no baseline using raw (or nnU-Net-default-preprocessed) intensities. Consequently, the LMM in Eq. (1) tests differences among the seven normalised variants, not whether normalisation itself changes performance. The phrase 'intensity normalisation had a measurable effect' is therefore not directly supported: the same experimental design with a different arbitrary set of transforms (e.g., gamma correction or bias-field correction) could show similar between-arm differences even if all seven methods were, say, neutral or harmful relative to no explicit normalisation. The internal and external comparisons cannot distinguish 'normalisation matters' from 'among the transformations we picked, some differ.' This is not an internal inconsistency, but it is a missing control that conditions the headline claim. The GMM component-selection concern is real but less load-bearing, because GMM is in the lower-performing group externally and changing its selection rule is unlikely to alter the top-three ranking of Nyúl, Z-score, and CLAHE.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports a systematic comparison of seven intensity normalisation methods (Z-score, min-max, robust min-max, histogram equalisation, CLAHE, Nyúl histogram matching, and a GMM-based method) for 3D U-Net meniscus segmentation in knee MRI. All models are trained on the IWOAI 2019 DESS dataset with identical 5-fold splits and evaluated on an internal test split and on the external SKM-TEA qDESS dataset. The main empirical findings are that internal performance differs little across methods, while externally small but statistically significant differences emerge: Nyúl, Z-score, and CLAHE are most robust, with differences of roughly 0.1–0.6% DSC among methods and a marginal R² of 0.005, compared with an overall ~10 percentage-point drop from internal to external data. The authors conclude that intensity normalisation has a measurable but limited effect on generalisability relative to domain shift.","tokens_in":11325,"tokens_out":9787,"duration_ms":99177,"significance":"If the results hold, the paper is a useful empirical benchmark for an underexplored area: intensity normalisation for musculoskeletal MRI segmentation under domain shift. It has several concrete strengths: use of public datasets, identical cross-validation splits across all methods, a strong nnU-Net baseline, appropriate linear mixed-effects models with fold and test-image random intercepts, Tukey-adjusted pairwise comparisons, and public code. The headline quantitative result — normalisation method explains 0.5% of variance while domain shift costs ~10 DSC points — is a clear and actionable message for deployment decisions. The main weakness is that all arms receive a normalisation transform, so the design supports ranking the seven methods but not an absolute statement about the effect of normalisation versus none; this condition on the headline claim needs to be addressed.","major_comments":[{"comment":"The claimed effect of 'intensity normalisation' is not directly supported because there is no raw/no-normalisation control arm. Every arm applies one of seven transforms, so Eq. (1) and the marginal R² = 0.005 measure differences among those transforms, not the effect of normalisation versus none. The abstract's sentence 'intensity normalisation had a measurable effect on model generalisability' is therefore stronger than the design permits. Please add a no-normalisation arm (or at least an nnU-Net-default-preprocessing arm) or, if that is infeasible, restate the causal claims as being about choice among the seven tested methods.","section":"Abstract; §3.2, Eq. (1), Table 2"},{"comment":"The text states that experiments were planned using default nnU-Net settings. nnU-Net applies its own intensity normalisation in its preprocessing pipeline; the manuscript does not state whether this internal normalisation was disabled or overridden. If it remained active, linear methods such as Z-score and min-max would be conflated by a subsequent z-score transform. The numerical differences in Table 1 suggest the internal normalisation was adapted, but this must be stated explicitly for the benchmark to be interpretable and reproducible. Please specify the exact nnU-Net data-preprocessing configuration used.","section":"§2.4"}],"minor_comments":[{"comment":"The GMM normalisation is under-specified: no equation is given for how the selected component mean is used to standardise the image, no argument is given that the 'second largest mean' component corresponds to a stable tissue class across DESS and qDESS, and no sensitivity analysis is reported for the number of components or component-selection rule. Because GMM changes rank between internal and external data and the Discussion interprets this, the implementation and its stability should be documented or the interpretation limited.","section":"§2.3"},{"comment":"The caption refers to 'the five methods' but seven methods are compared; the figure actually shows seven estimated marginal means.","section":"Fig. 2 caption"},{"comment":"The unadjusted 95% CI for Nyúl is [0.05, 0.24] while the Tukey-adjusted pairwise p-value vs. Z-score is 0.053. These are not formally inconsistent because the CI is unadjusted, but the text should say so explicitly to avoid confusion.","section":"§3.2, Table 2"},{"comment":"Minor typographical issues: 'has been been shown' (§1.1), 'worse that all methods' (§3.1), and '1.5×IQR' should read '1.5·IQR' or '1.5 IQR'.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"This is a solid empirical benchmark whose central message is plausible. The missing no-normalisation control and the lack of an explicit statement about nnU-Net's internal normalisation are the main technical issues; both are fixable either by adding an arm (the code is public) or by carefully narrowing the claims. The GMM concern is real but does not threaten the main ranking."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis is a well-run benchmark: seven intensity normalisation methods for 3D knee MRI meniscus segmentation, tested on internal IWOAI 2019 data and external SKM-TEA data. The work is new in a modest way—previous normalisation comparisons were mostly brain or breast MRI, or internal-only—and it gets the experimental design right: identical 5-fold splits across methods, nnU-Net baseline, LMMs with fold and test-image random intercepts, Tukey-adjusted pairwise comparisons, public code.\n\nThe findings are clear. Internally, there is almost nothing between the methods. Externally, Nyúl, Z-score, and CLAHE beat the others by roughly 0.1–0.6% DSC, and the model explains only 0.5% of total variance. The headline that domain shift (about a 10% DSC drop) dwarfs normalisation choice is well supported.\n\nThe soft spots are real but not fatal. First, there is no no-normalisation arm. Every arm applies one of the seven transformations, so the abstract's phrase 'intensity normalisation had a measurable effect' is not directly tested. What is tested is whether the choice among these seven matters. That is a legitimate framing, but the wording oversells. Adding a raw/default-preprocessing control, or rewording the claim, would fix it. Second, the GMM implementation picks the component with the second largest mean with no justification that this matches the same tissue across DESS and qDESS; a sensitivity analysis would be cheap. It probably would not change the top three, since GMM is in the lower group externally. Third, the external qDESS echoes are combined with root-sum-of-squares uniformly; the authors don't check for interaction with the normalisation methods. Minor.\n\nThis is a solid, honest paper. It deserves a serious referee, not a desk reject. The revision path is clear.","headline":"A solid, well-run benchmark of seven normalisation methods for knee MRI segmentation; the main caveat is the missing no-normalisation control, which makes the 'intensity normalisation matters' framing slightly oversold.","tokens_in":11768,"tokens_out":3021,"would_cite":true,"duration_ms":27561,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"For knee MRI meniscus segmentation, the choice of intensity normalisation has a statistically significant but tiny effect on cross-domain performance; the dominant factor is the domain shift itself.","keywords":["intensity normalisation","MRI","knee meniscus segmentation","domain shift","generalisability","benchmark","U-Net","cross-dataset validation"],"falsifier":"Retrain the same 3D U-Net on IWOAI 2019 with GMM normalisation using each of the four fitted components (or a tissue-validated peak selection) and evaluate on SKM-TEA; if the ranking of GMM moves from the bottom group into the top group, the paper's method-specific conclusion about GMM would not hold. More broadly, if the same seven-method benchmark applied to an external dataset with milder protocol differences showed normalisation explaining more than a few percent of Dice variance, the claim that domain shift dwarfs normalisation would need qualification.","tokens_in":1495,"feed_emoji":"🩻","tokens_out":1898,"duration_ms":59498,"temperature":0.7,"pith_summary":"The paper systematically compares seven intensity normalisation methods for 3D meniscus segmentation in knee MRI, asking whether preprocessing alone can make a deep learning model more robust to unseen scanners and protocols. Trained on the IWOAI 2019 dataset and evaluated on the SKM-TEA external dataset, the methods perform nearly identically on internal test data, but on external data Nyul histogram matching, Z-score, and CLAHE are slightly - though significantly - better than min-max scaling, robust min-max, histogram equalisation, and a Gaussian mixture model method, with differences of roughly 0.1-0.6% Dice. However, every method drops about 10 points in Dice between internal and external data, and normalisation method explains only 0.5% of the total variance in Dice scores. The paper's central claim is that intensity normalisation has a measurable but limited impact on generalisability relative to the magnitude of domain shift.","feed_headline":"Normalisation barely dents MRI domain-shift drop","feed_subtitle":"Seven intensity methods differ by about 0.5% Dice externally; the cross-dataset gap is about 10%.","key_machinery":"The evaluative engine is a controlled benchmark: seven normalisation methods - Z-score, min-max scaling, robust min-max, histogram equalisation, CLAHE, Nyul histogram matching, and a four-component Gaussian mixture model variant inspired by white-stripe normalisation - are applied identically to training and test images before training a 3D U-Net with five-fold cross-validation using shared splits. The statistical load-bearer is a linear mixed-effects model of the form Metric ~ Method + (1|Fold) + (1|TestImage), which separates the tiny fixed effect of normalisation from the large random effect of test image and the negligible fold effect. This design allows small cross-method differences to","core_discovery":"On its own terms, the paper's central finding is that a systematic comparison of seven standard intensity normalisation schemes for knee MRI segmentation finds statistically significant differences only on external data, where Nyul histogram matching, Z-score, and CLAHE group ahead of the other four methods. Yet these differences are an order of magnitude smaller than the gap between internal and external datasets: estimated external Dice ranges from about 78.3% for histogram equalisation to 79.1% for Nyul, while all methods lose roughly 10 points relative to the about 89% internal baseline. A linear mixed-effects model attributes nearly all Dice variance to differences between test images (","pith_inferences":["Because the two datasets differ in many simultaneous ways (sequence variant, resolution, population, annotators), the estimated 0.5% variance share for normalisation may be specific to this domain pair; on a pair with more similar protocols, normalisation's relative contribution could be larger.","The GMM method's weak external performance could be an artifact of its component-selection heuristic rather than a failure of tissue-peak normalisation in general; a sensitivity analysis over which Gaussian component is used is a direct next experiment.","A testable extension is to combine the top normalisation methods with intensity augmentation or a harmonisation step to see whether their small gains are additive, redundant, or even interfering.","Z-score's reported about 2% systematic under-estimation of meniscal volume, concentrated at boundaries, could matter for downstream morphometric measurements even when Dice is unaffected - worth direct measurement."],"forward_implications":["For practitioners deploying knee MRI segmentation models to unseen scanners or protocols, choosing Nyul, Z-score, or CLAHE is a cheap and safe default, but expecting normalisation to close the external-data gap is unrealistic.","The roughly 10-point Dice drop between datasets means domain shift - protocol differences, patient population, annotation style - dominates performance; effort should shift toward harmonisation, domain adaptation, or targeted fine-tuning.","Nyul's better alignment of intensity distributions across datasets did not translate into a meaningful segmentation gain over Z-score, suggesting histogram matching is not a substitute for true domain adaptation.","The significant but tiny method effect (marginal R-squared 0.005) implies that studies evaluating normalisation on internal data alone cannot detect differences; external validation is necessary.","The absence of any HD95 differences across methods indicates that segmentation shape is insensitive to normalisation choice, even when volume-based scores differ slightly."],"fun_headline_variants":["Normalisation tweaks MRI segmentation but domain shift dominates","Intensity normalisation: minor gain, major domain-shift loss","Seven normalisers, one verdict: dataset gap dwarfs method gaps","Normalisation methods differ little next to knee MRI domain gap","Nyul, Z-score, CLAHE lead but domain shift dwarfs normalisation"],"cache_read_input_tokens":13056,"weakest_assumption_plain":"The GMM normalisation assumes that the fitted component with the second-largest mean represents the same tissue type in both the DESS and qDESS protocols, and the paper offers no evidence or sensitivity analysis for that mapping.","fun_headline_variants_meta":{"raw":{"variants":["Normalisation tweaks MRI segmentation but domain shift dominates","Intensity normalisation: minor gain, major domain-shift loss","Seven normalisers, one verdict: dataset gap dwarfs method gaps","Normalisation methods differ little next to knee MRI domain gap","Nyul, Z-score, CLAHE lead but domain shift dwarfs normalisation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000635,"raw_usage":{"total_tokens":2760,"prompt_tokens":732,"completion_tokens":2028,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":476,"completion_tokens_details":{"reasoning_tokens":1939}},"tokens_in":476,"tokens_out":2028,"duration_ms":13831,"temperature":1.0,"reasoning_tokens":1939,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T10:58:48.137494+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain the same 3D U-Net on IWOAI 2019 with GMM normalisation using each of the four fitted components (or a tissue-validated peak selection) and evaluate on SKM-TEA; if the ranking of GMM moves from the bottom group into the top group, the paper's method-specific conclusion about GMM would not hold. More broadly, if the same seven-method benchmark applied to an external dataset with milder protocol differences showed normalisation explaining more than a few percent of Dice variance, the claim that domain shift dwarfs normalisation would need qualification.","supporting_citations":[],"review_version":1}