{"id":"dc3469d0-a4bb-4cea-b453-00b78f89f397","arxiv_id":"2411.12201","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"ISRL jointly learns image and deformation-shape features under an invariant risk minimization penalty, improving classification accuracy on 2D and 3D medical images under distribution shift.","lead":"This paper combines invariant risk minimization with learned deformation-based shape features to classify images more reliably when training and test data come from different environments. It reports accuracy gains on synthetic 2D drawings and on real brain and cardiac MRI datasets.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Real-data experiments test within the same environments, not on held-out environments, leaving the central cross-environment robustness claim unverified on brain and cardiac MRI.","rationale":"The reader's weakest_assumption centered on whether the environments contain the active spurious correlations and whether IRM can isolate an invariant predictor. I agree this is important, but the more immediate gap is that the experimental protocol does not test on held-out environments at all for the real datasets. The reader flagged the ADNI split inconsistency, which I also see, but the deeper problem is that even a corrected split would not test cross-environment generalization unless a held-out environment is used. The paper's introduction and abstract emphasize robustness to shifted environments, yet the 3D brain and cardiac evaluations use random splits within the same environment pools, so the central claim is unverified on those benchmarks. This is more load-bearing than missing error bars or undisclosed hyperparameters because it determines whether the reported improvements support the paper's stated contribution. I therefore recommend conditional acceptance, with mandatory held-out environment experiments and a clarified, consistent data split description.","tokens_in":14933,"tokens_out":9315,"duration_ms":98039,"concrete_test":"Re-run the 3D brain experiment with a leave-one-age-group-out protocol: train on two age environments (e.g., 50s-60s and 70s) and test on the remaining one (80s-90s), cycling through all three folds, using the identical backbones and hyperparameter selection. Do the same for cardiac MRI by training on the non-CHF environment and testing on the CHF environment, and vice versa. Report ISRL alongside ERM, IRM, CORAL, DANN, IWDAN, and CLOvE under these held-out environment conditions. If ISRL does not maintain its advantage over IRM/domain-generalization baselines, the central claim of robustness to shifted environments on real data is unsupported.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that ISRL learns representations that generalize across environments, with 'consistently higher classification accuracy' on shifted distributions. This claim is only directly tested on the 2D synthetic dataset, where a held-out test environment (pe=0.9) is used. For the 3D brain and cardiac experiments, the 'environments' (age groups, CHF status) are all present in the training split, and the reported test sets are random splits (65/25/10 and 70/15/15) of the same pools. Thus the model is never evaluated on an unseen age group or unseen CHF status. A random split within the same environments does not measure robustness to distribution shift; ERM-style methods can exploit environment-specific shortcuts that are still present in the test split. This is particularly consequential because the paper motivates the method with the failure of predictors under age and CHF shifts. The ADNI description is also internally inconsistent (690 images total but three environments of 150 each; a 65/25/10 split yields non-integer counts), which suggests the environment definition is not clean. As a result, the reported gains on brain and cardiac tasks (e.g., ISRL 84.44 vs IRM 72.67 on ResNet) may reflect superior feature extraction on the same distribution rather than successful invariant learning across environments. The paper's own Limitation section concedes that IRM can fail when training distributions do not cover test scenarios, but the current protocol does not even create such a coverage gap.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ISRL, a framework that combines deformation-based geometric shape features with image features and trains a classifier under an invariant risk minimization (IRM) penalty to improve robustness across environments. The method is evaluated on a synthetic 2D colored Quickdraw benchmark, 3D brain MRIs (ADNI) with age as environment, and 3D cardiac MRI videos with congestive heart failure as environment, comparing against ERM, IRM, and several domain generalization baselines across multiple backbones. The authors report consistent accuracy gains, especially on the synthetic benchmark, and provide code.","tokens_in":15189,"tokens_out":3507,"duration_ms":38803,"significance":"If the claims hold, the paper would make a useful contribution by extending invariant representation learning to an integrated image-plus-deformation shape space, an area that has not been explored. The synthetic benchmark with a held-out test environment is a strength, as is the public code release and the inclusion of diverse backbones and medical imaging tasks. However, the real-data experiments do not currently validate the central cross-environment generalization claim, since the test splits are random splits of the same environment pools rather than held-out environments. The reported numbers also lack any measure of variance, which weakens the claim of consistent improvement. The significance is therefore conditional on substantially strengthening the experimental protocol.","major_comments":[{"comment":"The central claim of cross-environment robustness is not tested on the real 3D datasets. For brain MRI, the three age environments are all included in the training split, and the 65/25/10 split appears to be a random split of the full 690-image pool; for cardiac MRI, the two CHF environments are both in training, with a random 70/15/15 split. In neither case is a model evaluated on an unseen age group or an unseen CHF status. A random split within the same environments does not measure robustness to distribution shift, because environment-specific shortcuts present in training also appear in the test split. To support the stated motivation, the authors should either evaluate on held-out environments (e.g., train on two age groups and test on the third, or train only on non-CHF subjects and test on CHF subjects) or explicitly report per-environment test accuracy and frame the real-data results as same-distribution feature learning rather than invariance. As written, the paper's own Limitation section concedes that IRM can fail when training distributions do not cover test scenarios, but the current protocol does not even expose the model to that scenario.","section":"§4.2, Tables 2 and 3; §4 dataset descriptions"},{"comment":"The synthetic test protocol is ambiguous. The text states that three environments are created with pe = 0.2, 0.1, 0.9 for two training and one testing environment, but then says the synthesized dataset is split 70% train, 15% validation, 15% test. If the 15% test split is a random subset of all three environments, then the test set includes examples from the training environments and is not a held-out environment. If the test set is restricted to the pe = 0.9 environment, the split description and the reported percentages are misleading. Please clarify how the split is applied, and ideally report results with the test set consisting solely of the pe = 0.9 environment, since that is the only direct evidence for cross-environment generalization in the paper.","section":"§4, 2D simulated data paragraph"},{"comment":"The ADNI environment definition is internally inconsistent. The text says there are three age environments (50s–60s, 70s, 80s–90s) and that each environment includes a total of 150 images, which would be 450 images in total, yet the dataset is stated to contain 690 images. The 65/25/10 split of 690 images yields non-integer counts (448.5, 172.5, 69), so it is unclear how the split is applied and whether the environments are balanced and disjoint. Please report the exact number of subjects per environment, the number per class per environment, and how the split is performed (per environment or globally).","section":"§4, 3D brain MRI paragraph"},{"comment":"No measure of variance is reported for any result. All accuracy, precision, and F1 values are point estimates from a single run or a single split. Several reported gains are modest (e.g., 88.76 vs. 86.51 for ViViT on cardiac MRI, and 76.67 vs. 74.00 for CNN on brain MRI), and without repeated runs it is impossible to determine whether the differences are statistically meaningful. The paper's phrasing 'consistently outperforms' requires supporting evidence such as means and standard deviations over multiple seeds, or a paired significance test. This is particularly important for the real-data results, where the differences are smaller than on the synthetic benchmark.","section":"§4.2, Tables 1–3 and Fig. 3"}],"minor_comments":[{"comment":"The figure contains a typo: 'Geomtric' should be 'Geometric'.","section":"Figure 1 caption"},{"comment":"The sentence 'Overall, our model consistently archives higher accuracy (> 10%) than the baselines' should read 'achieves'.","section":"Fig. 3 caption"},{"comment":"The notation in Eq. (5) is difficult to parse, with a scalar w = 1.0 described as a 'dummy classifier'. Please rewrite this equation more clearly, specifying exactly which variables are held fixed when the gradient is computed, since the penultimate sentence of §3.2 says 'Minimizing the loss above' but the formal statement of the optimization objective is ambiguous.","section":"§3.2, Eq. (5)"},{"comment":"The sentence 'we found 146 videos have CHF (E1, 48/98 as scar/non-scar) and 364 not (E2, 210/154 as scar/non-scar)' could be clarified by stating the total scar/non-scar counts (which appear to be 258 scar and 252 non-scar) so that the class balance across environments is immediately interpretable.","section":"§4, cardiac MRI paragraph"}],"recommendation":"major_revision","confidential_remarks":"The methodological novelty is fairly incremental: Eq. (5) is the standard IRMv1 penalty applied to a fused image and deformation feature representation, following Geo-SIC (Wang and Zhang, NeurIPS 2022) from the same group. The novelty claim is not the main concern; the main concern is that the experimental section does not yet substantiate the cross-environment generalization claim on real data. If the authors can add held-out-environment experiments or clearly re-scope the real-data claims, the paper could become publishable. The lack of error bars is also a serious reporting gap that should be addressed regardless of venue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a reasonable engineering paper that combines Geo-SIC's deformation shape encoder with the IRM penalty, and the joint-vs-disjoint training comparison (Fig. 4) is a genuine new empirical result. But the headline claim—consistent robustness to shifted environments—is only tested on the synthetic 2D data. The brain and cardiac MRI experiments split randomly within the same environment pools, so they never evaluate on a held-out age group or CHF status. That is a load-bearing gap, not a cosmetic one.\n\nWhat the paper does well: the ablation shows shape features add real value; the joint training gain (~2-3%) is consistent across backbones; the motivation (age and CHF confounds) is clinically relevant; and the code is public. The limitation section is also honest about IRM's assumptions.\n\nWhere it falls short: no error bars anywhere, so single-run numbers like 87.56 vs 71.11 need a multi-seed check. The ADNI description is internally inconsistent: three environments of 150 images each sums to 450, but the total is 690, and a 65/25/10 split of 690 is non-integer. The fusion weight β is mentioned but never given a value. And critically, the real-data evaluation protocol does not support the claim of cross-environment generalization. If the test set is a random split of the same age groups and CHF groups seen in training, the model can exploit environment-specific shortcuts that still exist in the test split. The reported gains over IRM and ERM may simply reflect better shape-aware feature extraction on the same distribution.\n\nNone of this invalidates the core approach. The synthetic experiment with a truly held-out test environment shows a large and consistent improvement, which is evidence that the method can work. But the medical imaging sections need a proper held-out environment evaluation before the central claim is credible.\n\nRecommendation: this deserves a serious referee. It is not a desk reject; the flaws are fixable with rigorous experiments. A good referee should ask for multi-seed runs, consistent environment definitions, a held-out age/CHF group, and clear hyperparameter reporting.","headline":"Sensible integration of shape features and IRM, but real-data evaluation doesn't test the cross-environment claim.","tokens_in":15762,"tokens_out":3718,"would_cite":false,"duration_ms":36923,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Joint shape-invariance learning outperforms image-only models.","keywords":["invariant risk minimization","shape representation learning","deformation-based shape analysis","image classification","domain generalization","spurious correlation","medical image classification","geodesic shooting"],"falsifier":"Train ISRL and image-only IRM on the same three datasets with deliberately misspecified environments, for example random age splits for ADNI and random CHF splits for the cardiac data. If ISRL's advantage over IRM persists under misspecified environments, then the invariance mechanism is not what produces the gains; if the advantage disappears, the result is confirmed to depend on the environments containing the true spurious correlations.","tokens_in":14706,"feed_emoji":"📐","tokens_out":8158,"duration_ms":75054,"temperature":0.7,"pith_summary":"This paper tries to establish that image classifiers become more accurate and more stable across changed environments when they learn invariant features in a joint space of raw images and deformation-based shape representations, rather than in image space alone. It proposes ISRL, which combines a template-registration network that encodes each image as an initial velocity field (the shape code) with an invariant risk minimization (IRM) penalty that forces the fused image+shape predictor to be simultaneously good in every training environment. The claim is supported by experiments on simulated 2D drawings with color as a spurious factor, on 3D brain MRIs where age is the confounder, and on cardiac MRI videos where heart-failure history is the confounder; in each setting ISRL reports higher accuracy and F1 than ERM, image-only IRM, and four domain-generalization baselines.","feed_headline":"Outperforms image-only models via joint shape-invariance learning","feed_subtitle":"Deformation-shape features plus IRM lift accuracy under color, age, and heart-failure confounds.","key_machinery":"The central object is the joint loss $\\ell_{\\mathrm{ISRL}} = \\ell_{\\mathrm{SRL}} + \\beta \\ell_{\\mathrm{IC}}$, where $\\ell_{\\mathrm{SRL}}$ trains an encoder-decoder to predict the initial velocity field $v_n(0)$ of a geodesic deformation, solved via the Euler–Poincaré differential equation, that maps a template to each input image, and $\\ell_{\\mathrm{IC}}$ is the invariant classification loss built from the IRM penalty $\\sum_{e} R_e(\\psi) + \\lambda \\|\\nabla_{w=1.0} R_e(w \\cdot \\psi)\\|_2^2$. The shape code $v$ and image code $I$ are concatenated in a latent space and fed to the classifier. The mechanism that carries the argument is the pressure the IRM gradient penalty puts on the fused representation to be optimal in every environment simultaneously, which is what supposedly removes spurious factors while preserving the shape features that genuinely determine the label.","core_discovery":"The paper's central claim is that shape features, when learned jointly with image features under an invariant risk minimization penalty, contain the invariant information needed to suppress spurious correlations that fool ordinary classifiers. The core discovery is the framework itself: a geometric shape learning network (an encoder-decoder that predicts the initial velocity field of a geodesic deformation aligning an image to a template) feeding into an invariant classification network, whose loss adds the environment-wise IRM gradient penalty to the shape reconstruction loss. With this coupling, the model consistently outperforms image-only invariant learning and empirical risk minimization on all tested backbones: on 2D Quickdraw with color confounds, ISRL reaches 87.56% accuracy under ResNet versus 71.11% for IRM; on 3D brain MRIs it reaches 84.44% versus 72.67%; on cardiac videos 88.76% versus 86.51% under ViViT. The paper argues these gains come from leveraging shape's invariance to intensity and texture changes while IRM discards environment-specific spurious shape changes.","pith_inferences":["Inference: If the reported gains hold under properly validated environments, the same joint shape-invariance recipe could be applied to other medical imaging tasks with known confounders, such as imaging-site batch effects, sex, or medication history, where shape codes are already available from registration pipelines.","Inference: The paper's environment definitions for the medical datasets are coarse and potentially miscalibrated (for example, each age environment is described as 150 images while the total is 690, and the 65/25/10 split is unexplained), so a careful reader should re-run with well-defined, non-overlapping environments before treating the medical numbers as decisive evidence.","Inference: Because the IRM penalty is sensitive to the chosen environments and penalty weight, ISRL's advantage could partly come from the extra shape reconstruction signal acting as a regularizer rather than from true invariance; an isolation test would decide between the causal and the regularization explanation."],"forward_implications":["ISRL reports gains across four backbones on 2D and 3D tasks, so the combination of shape features and IRM is not tied to a single network architecture.","On the 2D Quickdraw task, raising the label-flip probability from 0% to 50% causes the baselines to drop 30–55% in accuracy while ISRL drops only about 13–20%, suggesting the learned representation retains useful shape cues even when the spurious color-label link weakens.","The joint training scheme beats the disjoint two-step pipeline by 2–3%, so optimizing shape reconstruction and invariant classification together, rather than as a pre-processing stage, contributes to the reported performance.","The ablation shows that adding geometric shape features improves both ERM and IRM, and adding IRM improves both image-only and image+shape models; ISRL, which does both, is best, indicating the two ingredients are additive."],"supporting_citations":[{"why":"Supplies the invariant risk minimization penalty that ISRL adapts to the fused image and shape representation.","marker":"[2]"},{"why":"Provides the template-registration shape learning network, Geo-SIC, that produces initial velocity fields as shape codes.","marker":"[55]"},{"why":"Defines the LDDMM deformation energy and framework for deformation-based shape representations.","marker":"[5]"},{"why":"Gives the geodesic shooting derivation used to compute initial velocity fields from the EPDiff equation.","marker":"[52]"},{"why":"Supplies the ADNI brain MRI dataset and motivates the age-as-environment experiment for AD classification.","marker":"[23]"},{"why":"Supplies the cine cardiac MRI videos with scar labels and motivates the heart-failure-history environment.","marker":"[57]"},{"why":"Provides the 2D Quickdraw dataset used to construct the color-confounded simulated experiment.","marker":"[25]"}],"fun_headline_variants":["Shape-aware IRM beats image-only classifiers","Invariant shape features crush confound-driven bias","Joint shape+image invariant learning wins across domains","ISRL: learning shape invariances to outsmart spurious cues","Deformation-shape invariances lift classifier robustness"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the chosen environments—color on Quickdraw, age on brain MRIs, and congestive heart failure on cardiac MRIs—are exactly where the spurious correlations live, and that the training environments cover the test distribution, so the IRM penalty can identify a single predictor that is optimal across all of them.","fun_headline_variants_meta":{"raw":{"variants":["Shape-aware IRM beats image-only classifiers","Invariant shape features crush confound-driven bias","Joint shape+image invariant learning wins across domains","ISRL: learning shape invariances to outsmart spurious cues","Deformation-shape invariances lift classifier robustness"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000258,"raw_usage":{"total_tokens":1606,"prompt_tokens":994,"completion_tokens":612,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":610,"completion_tokens_details":{"reasoning_tokens":536}},"tokens_in":610,"tokens_out":612,"duration_ms":6427,"temperature":1.0,"reasoning_tokens":536,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T17:48:33.108550+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train ISRL and image-only IRM on the same three datasets with deliberately misspecified environments, for example random age splits for ADNI and random CHF splits for the cardiac data. If ISRL's advantage over IRM persists under misspecified environments, then the invariance mechanism is not what produces the gains; if the advantage disappears, the result is confirmed to depend on the environments containing the true spurious correlations.","supporting_citations":[{"cited_title":"Geo-sic: Learning de- formable geometric shapes in deep image classifiers","cited_arxiv_id":null,"evidence_quote":"Provides the template-registration shape learning network, Geo-SIC, that produces initial velocity fields as shape codes."},{"cited_title":"Computing large deformation metric map- pings via geodesic flows of diffeomorphisms","cited_arxiv_id":null,"evidence_quote":"Defines the LDDMM deformation energy and framework for deformation-based shape representations."},{"cited_title":"Diffeomorphic 3d image registration via geodesic shooting using an efficient adjoint calculation","cited_arxiv_id":null,"evidence_quote":"Gives the geodesic shooting derivation used to compute initial velocity fields from the EPDiff equation."},{"cited_title":"The alzheimer’s disease neuroimaging initiative (adni): Mri methods","cited_arxiv_id":null,"evidence_quote":"Supplies the ADNI brain MRI dataset and motivates the age-as-environment experiment for AD classification."},{"cited_title":"Ai based cmr assessment of biventricular function: clinical sig- nificance of intervendor variability and measurement errors","cited_arxiv_id":null,"evidence_quote":"Supplies the cine cardiac MRI videos with scar labels and motivates the heart-failure-history environment."},{"cited_title":"The quick, draw!-ai experi- ment","cited_arxiv_id":null,"evidence_quote":"Provides the 2D Quickdraw dataset used to construct the color-confounded simulated experiment."}],"review_version":1}