{"id":"3d7b0744-05a9-4d9d-a9c6-5c33f1205ab5","arxiv_id":"2504.19203","paper_version":7,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Using instance normalization, GIN intensity augmentation, and supervised contrastive loss improves total knee replacement prediction from FS-IW-TSE MRI when tested on DESS MRI.","lead":"This paper tests three changes to an MRI-based knee-replacement prediction model: instance normalization, intensity-shifting augmentation, and a contrastive loss. The combined changes lift accuracy on a different MRI sequence from near-random to 70 percent, a step toward models that survive imaging-protocol changes.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Entropy-based target model selection may inflate reported gains; retest without target validation before accepting single-source DG claim.","rationale":"After reading the paper, the most load-bearing assumption is the target-validation entropy model-selection rule. The paper is transparent about using this rule, but its validity is not demonstrated for this task, and it changes the problem setting. If the entropy criterion fails, the central claim about generalization of the full pipeline (IN+GIN+CL) loses support, because the incremental gains of GIN+CL over IN alone are small (target accuracy 70.04 vs 68.74, F1 67.30 vs 66.11) and might be due to selection. I agree with the reader that this is the weakest point. The fold ambiguity and lack of released code are real reproducibility issues but do not directly threaten the internal validity of the reported effect; they only make it hard to verify. I therefore recommend keeping the CONDITIONAL verdict (UNCHANGED), pending the proposed reanalysis.","tokens_in":13652,"tokens_out":6785,"duration_ms":72085,"concrete_test":"Retrain the GIN+ContrastiveLoss+IN model under the same seven-fold splits but select the best epoch using only source validation accuracy (or fixed/last-epoch weights), with no target validation entropy. If the target test accuracy drops by more than ~3 points (e.g., below ~67), the entropy-based selection is a major contributor to the reported improvement, and the method should be reframed as unsupervised domain adaptation rather than single-source domain generalization.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section IV describes model selection using a source validation accuracy threshold and target validation entropy. The threshold is 'initially set to 0.75; if this threshold was not met, it was progressively lowered by 0.05,' so it is always eventually met; the operative criterion is therefore 'minimize entropy on the target validation set.' This is only valid if the low-density separation assumption holds for the target domain (TKR vs no-TKR in DESS MRI). The paper cites prior work for the idea, but does not empirically verify it here. If the assumption is violated, the chosen epoch may be confident but wrong, and the reported target-test gains (accuracy 52.87→70.04, F1 18.98→67.30) could reflect target-domain model selection rather than intrinsic generalization of the proposed training modifications. Additionally, the setup uses unlabeled target validation data during model selection, which is an unsupervised-domain-adaptation/transductive setting, not single-source domain generalization as claimed in the abstract. Since the evaluation is on the target test set after selection on target validation, the reported numbers are conditional on having access to target-distribution data, weakening the headline 'generalization' claim. The ablation tables (Table V–VII) also suggest some hyperparameter choices were selected on test performance, which further inflates optimistic estimates, though this is secondary to the validation-entropy issue.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript studies single-source domain generalization for MRI-based total knee replacement (TKR) prediction using the OAI dataset, with FS-IW-TSE as the source domain and DESS as the target domain. The proposed modifications are replacing batch normalization with instance normalization, adding GIN-based intensity augmentation with four augmented views per input, and adding a supervised contrastive loss to the classification loss. On the authors' seven-fold protocol, the full model improves target accuracy from 52.87 ± 3.17 to 70.04 ± 2.49, F1 from 18.98 ± 16.89 to 67.30 ± 3.57, and ROC AUC from 59.33 ± 6.20 to 78.12 ± 1.97, with smaller source-domain improvements. The paper includes ablations on contrastive-loss margins, metric-learning losses, and the GIN interpolation coefficient, and compares against RAC, MixStyle, CutOut, RSC, and AdvBias.","tokens_in":13899,"tokens_out":7633,"duration_ms":71481,"significance":"If the reported target-domain gains survive a target-blind evaluation protocol, the result is practically valuable: modest changes to normalization, augmentation, and the training objective substantially improve MRI-sequence transfer for a clinically relevant endpoint. The comparison includes several established domain-generalization methods, the target-domain improvements are consistent across folds, and the target accuracy, F1, and AUC improvements are statistically strong under paired tests. The main limitation is that model selection uses unlabeled target-domain validation entropy, which makes the current evaluation transductive rather than strictly single-source domain generalization; resolving this issue is essential before the headline claim can be accepted.","major_comments":[{"comment":"The model-selection protocol described in Section IV uses unlabeled target-domain validation entropy as the operative criterion. Because the source-accuracy threshold is 'initially set to 0.75; if this threshold was not met, it was progressively lowered by 0.05,' the threshold is always satisfied by construction, so the only constraining criterion is minimum entropy on the target validation set. This means the reported target test numbers in Table III are conditional on access to target-distribution data, which is an unsupervised domain adaptation/transductive setup rather than the single-source domain generalization claimed in the abstract. If the low-density separation assumption does not hold for this task, the selected epoch can be confidently wrong, and the improvement from 52.87% to 70.04% target accuracy may reflect target-informed selection rather than intrinsic generalization of IN+GIN+contrastive loss. Please re-evaluate with model selection based only on source validation accuracy (or a fixed epoch) and report whether the target gains persist; additionally, provide evidence for the low-density separation assumption, for example by correlating target validation entropy with target test accuracy across epochs.","section":"IV (Model selection)"},{"comment":"The abstract states that 'The GIN method with contrastive loss performed better than all evaluated single-source domain generalization methods when using 3D instance normalization.' Table III directly contradicts this for accuracy: AdvBias+IN achieves 70.26 ± 3.38 target accuracy while GIN+ContrastiveLoss+IN achieves 70.04 ± 2.49. The claimed superiority is therefore metric-dependent and should be qualified (e.g., 'higher F1 and AUC, comparable accuracy') or supported by a formal comparison with confidence intervals or paired tests.","section":"Abstract and Table III"},{"comment":"The text reports one-sided paired t-tests and states that 'the performance improvements are statistically significant' in both domains. However, the source-domain ROC AUC p-value is 8.37 × 10⁻², which is not significant at α = 0.05. The abstract's claim of 'statistically significant improvement in classification metrics across both domains' is thus too strong; either report significance per metric accurately or adjust the claim.","section":"V (Statistical significance)"},{"comment":"The evaluation protocol is not described precisely enough to be reproduced. The paper states that the seven-fold splits follow [9], where 'each fold consisted of six training splits, six validation splits, and one test set,' but it then designates 'the first validation split' as the target validation set and subdivides 'the first training split' into source training and source validation sets. It is unclear how the remaining five training and validation splits are used, how the 100-sample source validation set is drawn, and whether the same protocol is applied to all compared methods. Please provide a complete algorithmic description of the splits or release the exact fold indices, since the cross-validation results are the only evidence for the central claim.","section":"II (Dataset) and IV (Method)"},{"comment":"The hyperparameters of the final model (m_pos = 0, m_neg = 1, ContrastiveLoss, α ~ U(0,1)) appear to have been selected after inspecting target-domain results in Tables V–VII. If any of these comparisons used the target test set, the reported test metrics and p-values are optimistically biased because the test set has effectively been used for model selection. Please clarify whether the ablation comparisons were made on the target validation set or the target test set, and if target test information was used, apply a nested validation scheme or report the selection procedure explicitly.","section":"V (Ablations, Tables V–VII)"}],"minor_comments":[{"comment":"The text states that the L2 distance between normalized feature vectors lies in the range [0,1]; for unit-normalized vectors, the L2 distance lies in [0,2]. This should be corrected, although it does not change the main results.","section":"IV (Equation 2)"},{"comment":"There are several typographical issues: 'ContrasitveLoss' in Section IV, 'ConstrastiveLoss' in Table V, 'differential blocks' should be 'residual blocks' in Section V, and the title contains a spurious space in 'T otal.'","section":"Throughout"},{"comment":"The phrase 'no samples from the same domain in more than one domain' is confusing; please rephrase to state clearly that the source and target sets contain disjoint subjects or disjoint imaging sessions as intended.","section":"II (Dataset)"},{"comment":"The abbreviation RAC is used in Table III but defined only later in Section V; please define it in the table caption or at first use.","section":"V (Table III)"},{"comment":"The description of positive pairs is slightly ambiguous: 'its four augmented versions and the four augmented versions of all other samples in the mini-batch that share the same label' could be read as including the anchor's own augmented versions twice if the anchor is also 'another sample.' Please clarify the pairing rule.","section":"IV (Loss function)"}],"recommendation":"major_revision","confidential_remarks":"The main uncertainty is the target-validation-entropy selection protocol. I believe this is fixable by re-running the evaluation target-blind or by providing explicit evidence for the low-density separation assumption, so I recommend major revision rather than rejection. The absence of code and precise split indices is a reproducibility concern that should be addressed in the revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The punchline: this is a practical, well-executed empirical study showing that swapping batch norm for instance norm, adding GIN augmentation, and adding a supervised contrastive loss lifts TKR prediction from FS-IW-TSE to DESS MRI within the OAI cohort. The target-domain gains are large and statistically significant for accuracy and F1. That result is new for this task, even though every ingredient is published. The paper also does a fair comparison against several single-source DG methods and includes careful ablations over margins, loss functions, and the interpolation coefficient. Credit where due: the experimental work is solid and the paper is honest about its own limitations.\n\nThe soft spots are real and mostly center on the evaluation protocol. Section IV says the best model is selected using target-domain validation entropy, with a source-accuracy threshold that is always eventually lowered. That means unlabeled target data is used during model selection. This is a transductive or unsupervised-domain-adaptation setup, not classic single-source domain generalization as the abstract claims. If the low-density separation assumption does not hold for DESS TKR prediction, the reported target gains may be inflated. I think the stress-test note is correct on this point. A clean retest would select the model using only source validation or a fixed epoch budget, and then report target test numbers.\n\nSecond, the fold structure is described too tersely. Reusing the splits from [9] is fine, but the description of six training splits, six validation splits, and a first-validation target set is not reproducible without [9]'s files, and no code or fold files are released. That is a minor-to-moderate reproducibility issue, not fatal.\n\nThird, a small overclaim in the abstract: it says statistically significant improvements across both domains, but the source-domain AUC p-value is 0.0837, which is not significant at the usual threshold. The target-domain p-values are tiny, so the main claim still stands, but the wording overshoots.\n\nFinally, the hyperparameter choices for the proposed method come from the same test folds via ablations (Tables V–VII). The authors acknowledge limited exploration and this inflates optimism somewhat, but the main IN-vs-BN effect is large and consistent across folds, so I do not think this is a load-bearing flaw.\n\nBottom line: the paper deserves a serious referee. The right revision would clarify the transductive nature of the model selection, add a source-only selection baseline, and release the fold structure or code. Who benefits? Researchers working on MRI domain shift for OA prediction, and anyone building practical pipelines that can afford unlabeled target validation data.\n\nRecommendation: send it to peer review with a request for a source-only selection ablation and a corrected abstract. The core result is likely reproducible and useful, but the framing as single-source DG needs to be fixed.","headline":"A useful empirical combination of IN + GIN + contrastive loss for TKR prediction, but the target model selection uses unlabeled target data, which weakens the single-source DG claim.","tokens_in":14422,"tokens_out":1210,"would_cite":false,"duration_ms":14517,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A knee-replacement MRI classifier trained on one scan protocol can generalize to another protocol without target labels by swapping batch normalization for instance normalization, adding intensity-preserving augmentation, and aligning…","keywords":["knee osteoarthritis","total knee replacement prediction","MRI","domain generalization","instance normalization","contrastive learning","image augmentation","medical image analysis"],"falsifier":"Train the same three-component model on the same source and target MRI pair but select checkpoints by best source validation accuracy or by a fixed late epoch instead of by target-validation entropy, then measure target test accuracy; if it drops back toward the 50-60% range, the entropy criterion, not the representation learning, is carrying most of the reported gain. A complementary check is to keep the entropy rule but compare it against a random choice among checkpoints that pass the source accuracy threshold; if random selection performs about as well, the low-density separation assumption is not doing load-bearing work.","tokens_in":13446,"feed_emoji":"🦵","tokens_out":12556,"duration_ms":107751,"temperature":0.7,"pith_summary":"The paper claims that a deep-learning model predicting total knee replacement from knee MRI can be made to generalize across MRI protocols without any labeled target data. In the authors' setup, the baseline classifier, trained on fat-suppressed intermediate-weighted turbo spin-echo images and tested on dual-echo steady-state images, collapses to near-chance performance in the target domain (accuracy about 53%, F1 about 19%, ROC AUC about 59%). Replacing batch normalization with instance normalization, generating augmented views with global intensity non-linear augmentation, and adding a supervised contrastive loss raises target accuracy to about 70%, F1 to 67%, and ROC AUC to 78%, while also improving source-domain metrics. The paper reports that these gains are statistically significant and that the combination outperforms the evaluated single-source domain generalization methods under the same training setup.","feed_headline":"Instance norm lifts knee-MRI model 53% to 70% cross-sequence","feed_subtitle":"Batch normalization collapsed on unseen MRI; instance norm, GIN augmentation, and contrastive loss restored it.","key_machinery":"The load-bearing mechanism is the interaction of three training-time components. Instance normalization normalizes each 3D volume by its own statistics, removing the batch-dependent intensity statistics that differ between MRI protocols. GIN augmentation passes each volume through a shallow random-weight convolutional network with tanh activations and interpolates the result with the original volume using a mixing coefficient drawn from $U(0,1)$, creating intensity and texture variation that preserves anatomy. A supervised contrastive loss, $L = [d_p - m_{pos}]_+ + [m_{neg} - d_n]_+$, is applied to L2-normalized 256-dimensional feature vectors with margins $m_{pos}=0$ and $m_{neg}=1$, using four augmented views per sample plus same-label views from the mini-batch as positives and different-label views as negatives. The fourth mechanism is model selection: checkpoints must pass a source validation accuracy threshold (starting at 0.75 and lowered in steps of 0.05) and minimize prediction entropy on the unlabeled target validation set, an application of the low-density separation assumption.","core_discovery":"The central claim is that three changes to a standard 3D convolutional classifier and its training loop make an MRI-based total knee replacement predictor robust to a change in imaging sequence: instance normalization in place of batch normalization, global intensity non-linear (GIN) augmentation that interpolates each input volume with the output of a fixed random-weight nonlinear network, and a supervised contrastive loss that pulls together augmented views and same-label samples while pushing apart different-label samples. In the target domain, the combined model reaches 70.04 ± 2.49 accuracy, 67.30 ± 3.57 F1, and 78.12 ± 1.97 ROC AUC, compared with 52.87 ± 3.17, 18.98 ± 16.89, and 59.33 ± 6.20 for the baseline. The source domain also improves on all three metrics. Ablation studies attribute the effect to the combination: without instance normalization, GIN plus contrastive loss reaches only 57.90 target accuracy, while the full model reaches 70.04. The authors also find that the contrastive loss with margins (0,1) outperforms InfoNCE, NCA, and triplet losses in this small-batch 3D setting.","pith_inferences":["The same recipe should transfer to other MRI tasks that suffer from protocol shift, such as cartilage lesion segmentation or meniscal tear classification, since the contrastive loss only needs image-level labels.","The largest reported gain comes from fixing the baseline's target-domain collapse to the majority class; on a protocol shift that does not produce severe class imbalance, the method's advantage over a well-calibrated baseline would likely be smaller, and this is testable with the same ablation.","The model-selection rule uses unlabeled target-validation entropy, but the paper does not report what happens with simpler rules such as last-epoch selection or best source-validation accuracy; comparing those rules would separate the contribution of representation learning from the contribution of checkpoint selection."],"forward_implications":["A TKR classifier trained on one MRI sequence can be applied to another sequence without collecting new labels, because none of the three components uses target labels during training.","Target-domain F1 rises from roughly 19 to 67, meaning the model stops collapsing to the no-TKR class when the imaging protocol changes.","The combined method is reported to outperform the evaluated single-source domain generalization alternatives when instance normalization is used, so the components do not simply trade source performance for target performance.","Ablations show that instance normalization is the largest single contributor: removing it drops target accuracy from 70.04 to 57.90 even with GIN and contrastive loss still active.","The interpolation coefficient in GIN is a real tuning knob: $\\alpha=1$ (extreme augmentation) gives the worst source accuracy, while $U(0,1)$ mixing gives the best target accuracy."],"supporting_citations":[{"why":"Supplies the matched case-control MRI cohort, the seven-fold split structure, and the baseline architecture that the experiments reuse and modify.","marker":"[9]"},{"why":"Provides the adversarial bias-field augmentation method that serves as a comparison baseline the proposed approach must beat.","marker":"[15]"},{"why":"Provides MixStyle, a feature-statistics mixing baseline the proposed approach is compared against.","marker":"[16]"},{"why":"Supports the claim that batch normalization degrades under domain shift, motivating the switch to instance normalization.","marker":"[18]"},{"why":"Introduces the global intensity non-linear augmentation whose random-weight network and interpolation scheme the paper adopts.","marker":"[20]"},{"why":"Provides the supervised contrastive loss implementation used to align same-label augmented views.","marker":"[23]"},{"why":"Earlier work combining GIN with a contrastive loss for segmentation; the paper adapts the idea to classification with a different loss formulation.","marker":"[24]"},{"why":"States the low-density separation assumption that justifies using unlabeled target-validation entropy for model selection.","marker":"[25]"}],"fun_headline_variants":["Instance norm, GIN aug, contrastive loss boost knee-MRI cross-sequence accuracy","Three tweaks lift knee-MRI model from 53% to 70% on a new MRI sequence","Knee-MRI AI: instance norm, GIN, contrastive loss fix cross-sequence generalization","Beat batch norm: instance norm, GIN aug, contrastive loss for knee-MRI"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a model's average uncertainty on unlabeled target validation images is a reliable guide to its true target accuracy, and that the reused fold splits from the earlier study are faithful; if either assumption fails, the reported target gains could be inflated.","fun_headline_variants_meta":{"raw":{"variants":["Instance norm, GIN aug, contrastive loss boost knee-MRI cross-sequence accuracy","Three tweaks lift knee-MRI model from 53% to 70% on a new MRI sequence","Knee-MRI AI: instance norm, GIN, contrastive loss fix cross-sequence generalization","Beat batch norm: instance norm, GIN aug, contrastive loss for knee-MRI"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000285,"raw_usage":{"total_tokens":1731,"prompt_tokens":1047,"completion_tokens":684,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":663,"completion_tokens_details":{"reasoning_tokens":586}},"tokens_in":663,"tokens_out":684,"duration_ms":6066,"temperature":1.0,"reasoning_tokens":586,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:58:42.765897+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same three-component model on the same source and target MRI pair but select checkpoints by best source validation accuracy or by a fixed late epoch instead of by target-validation entropy, then measure target test accuracy; if it drops back toward the 50-60% range, the entropy criterion, not the representation learning, is carrying most of the reported gain. A complementary check is to keep the entropy rule but compare it against a random choice among checkpoints that pass the source accuracy threshold; if random selection performs about as well, the low-density separation assumption is not doing load-bearing work.","supporting_citations":[{"cited_title":"Prediction of total knee replacement using deep learning analysis of knee MRI,","cited_arxiv_id":null,"evidence_quote":"Supplies the matched case-control MRI cohort, the seven-fold split structure, and the baseline architecture that the experiments reuse and modify."},{"cited_title":"MixStyle Neural Networks for Domain Generalization and Adaptation","cited_arxiv_id":"2107.02053","evidence_quote":"Provides the adversarial bias-field augmentation method that serves as a comparison baseline the proposed approach must beat."},{"cited_title":"Realistic Adversarial Data Augmentation for MR Image Segmentation","cited_arxiv_id":"2006.13322","evidence_quote":"Provides MixStyle, a feature-statistics mixing baseline the proposed approach is compared against."},{"cited_title":"Learning to optimize domain specific normalization for domain generalization,","cited_arxiv_id":null,"evidence_quote":"Supports the claim that batch normalization degrades under domain shift, motivating the switch to instance normalization."},{"cited_title":"Adversarial Consistency for Single Domain Generalization in Medical Image Segmentation,","cited_arxiv_id":null,"evidence_quote":"Earlier work combining GIN with a contrastive loss for segmentation; the paper adapts the idea to classification with a different loss formulation."},{"cited_title":"Mixed Samples as Probes for Unsupervised Model Selection in Domain Adaptation,","cited_arxiv_id":null,"evidence_quote":"States the low-density separation assumption that justifies using unlabeled target-validation entropy for model selection."}],"review_version":1}