{"id":"ce8b0f48-4dd4-4694-9038-b1d36122ec6b","arxiv_id":"2505.11702","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A lightweight post-hoc adapter trained with an anchored least-squares loss can make a frozen pretrained vision encoder approximately rotation- and noise-invariant with little damage to its original feature space.","lead":"The paper adds a small adapter network after a frozen pretrained image model so the combined model becomes robust to rotated or noisy inputs without retraining the original model. It introduces two training losses and shows large accuracy gains on rotated and noisy images while keeping the original feature space almost unchanged.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Structure-preservation metrics may be computed on the adapter's training set; the claimed near-isometry of the latent space could reflect in-sample memorization of the anchor term rather than a property of the learned map on held-out data.","rationale":"The reader's weakest-assumption (collision rate / local entanglement) is a legitimate condition and the paper itself discloses the ResNet50 failure as a limitation, so it is partially internalized. The more consequential soft spot is the unstated evaluation domain for the structure-preservation metrics. The paper's central novelty is achieving invariance 'without corrupting the existing capabilities' of the pretrained encoder, and Section 4.1.2 is the only quantitative evidence for that. Because the MaWa anchor term directly optimizes identity on training points, reporting R2/NRMSD on the same points would be circular in a mild sense: it would confirm the loss was optimized, not that the learned adapter is nearly isometric on the data distribution. The one-hidden-layer MLP is heavily overparameterized relative to STL10's 5,000 training images, so memorization is plausible. Classification accuracy on original test data staying near 98% is reassuring for class structure but does not imply pairwise distance preservation. A held-out recomputation is cheap and decisive. This does not overturn the invariance results, so the CONDITIONAL verdict remains appropriate; it adds a specific condition that the authors should verify.","tokens_in":37460,"tokens_out":8135,"duration_ms":82607,"concrete_test":"Retrain the MaWa rotation adapter on a random 80% split of the STL10 training features, then compute the Section 4.1.2 metrics (R2, NRMSD, L1/L2) on the held-out 20% and on the test set using the authors' evaluation code. If held-out R2 falls below roughly 0.8 or NRMSD rises above roughly 0.05, the near-isometry claim in Tables 2-3 is an in-sample artifact; if metrics match the reported values, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The MaWa loss (Eq. 21) contains an explicit anchoring term that drives E_theta(F(x_i)) to F(x_i) on the training batch. The structure-preservation evaluation (Section 4.1.2, Eqs. 34-38) reports R2, NRMSD, and Lipschitz constants on pairs from supp(mu_X) but never states whether that support is the training set, the test set, or a held-out split. With d=384, the one-hidden-layer adapter has roughly 3.1M parameters, while STL10 provides only 5,000 clean training images (plus s=3 augmentations), so the identity map on training points can be memorized. If Tables 2-3 are computed on the training distribution, R2=0.96 and NRMSD=0.02 are largely a restatement of the first term of the loss, not evidence that the original latent space is preserved on new inputs. The preserved clean-test classification accuracy (98.22 vs 98.58) suggests the adapter does not destroy class structure, but pairwise distance isometry is a much stricter claim and is not established by the paper as written.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper formalizes the problem of post-training augmentation invariance, in which a lightweight adapter E_theta is appended to a frozen pretrained encoder F so that the composite E_theta∘F becomes approximately invariant to specified augmentations while preserving the metric structure of F's latent space. Two training objectives are proposed: Markov-Wasserstein (MaWa) minimization, an anchored MSE loss that pulls augmented features toward their clean counterparts, and Wasserstein correlation (WaCo) maximization, a transport-based dependence objective. The experiments on STL10 and TinyImageNet with DINOv2, SwAV, R-DINO, CLIP, and a supervised ResNet50 show large gains in rotation and noise robustness (for example, DINOv2 rotation accuracy rises from 71.22% to 94.74% with a nonlinear classifier), whereas SimCLR and HSIC objectives degrade the latent space. The paper also introduces collision-rate diagnostics, reports an honest failure case on supervised ResNet50, and makes code available.","tokens_in":37687,"tokens_out":8152,"duration_ms":81247,"significance":"The empirical findings are substantial and potentially useful: the proposed method is simple, keeps the backbone frozen, and works across several modern self-supervised feature spaces, which is a practically attractive combination. The formal definitions of augmented encoders and (t,µ_X,F,V)-invariance provide a reusable vocabulary, and the derivation of the MaWa loss from the Markov-Wasserstein metric is a nice conceptual bridge. The honest treatment of the ResNet50 failure via collision rates, the systematic comparison against SimCLR and HSIC, and the public code are strengths. The main unresolved issue is that the signature claim of near-isometric structure preservation is not yet verified on held-out data; as written, the reported R² and NRMSD values may partly restate the anchor term in the training loss when computed on training points. If the structure-preservation claim is confirmed on a proper held-out split, this would be a solid contribution to the adapter and invariance literature.","major_comments":[{"comment":"The structure-preservation statistics are computed on pairs from supp(µ_X), but the paper never states whether supp(µ_X) is the training split, the test split, or a pooled set. This matters because the MaWa loss (Eq. 21) explicitly anchors E_θ(F(x_i)) to F(x_i) on the training batch; with roughly 3.1M parameters and only 5,000 clean training images, the adapter can memorize the identity on those points. If Tables 2-3 are computed on the training distribution, the reported R²=0.96 and NRMSD=0.02 are largely a restatement of the first term of the loss rather than evidence that the learned map is nearly isometric on new inputs. Please state the data split explicitly and recompute the structure metrics on held-out clean data (e.g., the STL10 test set) to substantiate the abstract's claim that E_theta acts 'nearly isometrically' on the latent distribution.","section":"4.1.2 (Eqs. 34-39, Tables 2-3)"},{"comment":"The evaluation protocol does not specify which data split is used for training the adapter, for training the classifier C, and for computing the augmented accuracy. If the classifier is trained and evaluated on the same augmented data, or if the adapter is trained and evaluated on the same clean data, the reported accuracy gains could partly reflect memorization rather than true invariance. Please describe the exact protocol (for example, adapters and classifiers trained on the STL10 train split and augmented accuracy computed on the test split) and confirm that the EC baseline follows the same split so that the comparisons in Tables 1, 6, and the appendix are interpretable as generalization results.","section":"4.1.1 (Invariance Evaluation)"}],"minor_comments":[{"comment":"All tables report point estimates without seeds, confidence intervals, or error bars; given the stochasticity of MLP training, please add at least a few seeds (with means and standard deviations) for the main classification and structure-preservation tables, or explicitly state that each result is from a single run.","section":"Section 4.2 and all tables"},{"comment":"There are stray closing parentheses in the augmented term, which currently reads '∥(Eθ◦F)(t(ak_i,xi))−F(xi)∥2^2' with an extra ')' before the minus sign; please correct the notation.","section":"Eq. (21) and Eq. (25)"},{"comment":"The paper acknowledges that the experimental WaCo results use the sliced variant SWaCo and that the theoretical connection between SWC and WC maximization is left open; this caveat should be stated more prominently in the contributions section so readers do not infer that the exact WC objective is what was trained.","section":"Footnote 1 and Remark 3.6"},{"comment":"The claim that the aligned collision rate 'predicts the feasibility of MaWa minimization' is based on five models; this is an interesting observation, but it should be presented as suggestive rather than as a validated predictor, and the small sample size should be acknowledged.","section":"Section 4.2.4 (Collision Rate Analysis)"},{"comment":"The MNIST dimensionality-reduction experiment is evaluated only through t-SNE visualizations; please add a quantitative metric (e.g., classification accuracy on the reduced space or a distance-preservation statistic) to support the claim that the WaCo loss performs dimensionality reduction while preserving structure.","section":"Section 4.2.5 (WaCo for Dimensionality Reduction)"}],"recommendation":"major_revision","confidential_remarks":"The paper fits the scope of the journal and the core idea is interesting, but the central near-isometry claim is not yet supported by a clearly specified evaluation split. The stress-test concern is valid and should be addressed before publication. The self-citation (Eikenberry 2023) is used in the appendix for standard measure-theoretic support and appears appropriate. I see no circularity in the main evaluation, since classification accuracy is measured on augmented inputs by classifiers trained on clean data."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a useful empirical paper, not a deep theoretical one. It defines post-training augmentation invariance cleanly, and shows that a one-hidden-layer adapter trained with an anchored MSE (MaWa) on frozen self-supervised features recovers large chunks of accuracy loss on rotations and noise. The results across DINOv2, SwAV, R-DINO, CLIP, and TinyImageNet are consistent, and the ResNet50 failure is honestly tied to collision rates. I believe the main finding.\n\nWhat's genuinely new: the augmented-encoder formalism, the collision-rate proxy, and the WaCo application. But the paper's central trick is simple: anchor the adapter to the identity on clean features and pull augmented features to the clean anchor. That works, and the framing is useful.\n\nSoft spots, in decreasing order:\n\n1. Structure preservation is likely measured in-sample. The scatter plots and R2/NRMSD are computed on pairs in supp(mu_X), which in practice is the training set. With 5k clean STL10 images and a 3.1M-parameter MLP, the anchor term in Eq. 21 can memorize the identity on training points, so R2=0.96 doesn't establish isometric behavior on new data. The clean held-out classification accuracy staying at 98.2 is reassuring, but that's a much weaker claim than pairwise isometry. The paper should state what split was used, and ideally compute isometry on held-out features.\n\n2. No error bars or seeds anywhere. All tables are point estimates. For an empirical paper with 100-epoch training, this is fixable but should be requested.\n\n3. Eq. 24 has an apparent typo: the second term in the tensor product sum should involve the augmented argument t(ak,x), not the clean x. The subsequent derivation is correct, but as printed it's confusing.\n\n4. The WaCo theory is explicitly deferred; Remark 3.6 and the footnote admit the sliced-Wasserstein proxy lacks a rigorous justification. That's honest, but it means WaCo is currently an empirical trick.\n\nNone of these are fatal to the main accuracy claim. The core invariance result is credible because the loss directly optimizes for it and the classifier is trained on clean data and evaluated on augmented data, which is not circular. The paper deserves a serious referee. I'd send it with clear requests: report structure-preservation on held-out features, add seeds/error bars, fix Eq 24, and either tighten the isometry claim or soften it.","headline":"A solid empirical paper with a genuinely useful framing: a simple anchored-MSE adapter recovers large invariance gaps on frozen features, but the near-isometry claim as written is likely in-sample and should be verified on held-out data.","tokens_in":38214,"tokens_out":3227,"would_cite":true,"duration_ms":34104,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","62H20","49Q22"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a frozen pretrained encoder can be made approximately invariant to augmentations by appending a one-hidden-layer MLP adapter trained with either of two optimal-transport losses, while leaving the original latent…","keywords":["post-training augmentation invariance","augmented encoders","adapter networks","optimal transport","Wasserstein correlation","Markov-Wasserstein kernels","frozen pretrained models","structure preservation"],"falsifier":"Take a self-supervised encoder with near-zero aligned collision rate for rotations, train a MaWa adapter, and check both rotated classification accuracy and the $R^2$ of the original pairwise-distance scatter; if either fails to reproduce the paper's reported regime (accuracy near clean accuracy and $R^2$ close to 1), the central claim is contradicted. A sharper disconfirming case would be a low-collision encoder where MaWa nevertheless collapses or corrupts the clean latent space.","tokens_in":37214,"feed_emoji":"🔄","tokens_out":7601,"duration_ms":70864,"temperature":0.7,"pith_summary":"The paper sets out to prove that a pretrained encoder can be given new invariance properties after training, without changing the encoder's weights or damaging its existing feature space. The proposed mechanism is a lightweight one-hidden-layer MLP adapter appended to the frozen latent space, trained with one of two optimal-transport losses: Markov-Wasserstein (MaWa) minimization, an anchored mean-squared error, or Wasserstein correlation (WaCo) maximization. On STL10 with DINOv2 features, the MaWa-trained adapter lifts classification accuracy on arbitrarily rotated images from 71.22% to 94.74%, and on heavily noised images from 58% to 86%, while preserving pairwise distances in the original latent space almost exactly. The paper also defines augmented encoders and a formal notion of post-training augmentation invariance, and it argues that contrastive and Hilbert-Schmidt independence losses fail at this task because they corrupt the original latent geometry.","feed_headline":"One-layer adapter makes frozen features rotation-invariant","feed_subtitle":"Rotated-image accuracy rises from 71% to 95% with the encoder untouched and the latent space nearly isometric.","key_machinery":"The load-bearing object is the augmented encoder, a Markov-Wasserstein kernel $E_T(x)$ that returns the distribution of encoded versions of $x$ under a family of augmentations; it turns augmentation into a probabilistic encoding process. The main training mechanisms are two losses: MaWa, an anchored mean-squared error that pulls encoded augmented views toward their frozen clean embeddings while anchoring the identity on non-augmented inputs (derived as minimization of a Markov-Wasserstein metric), and WaCo, which maximizes the Wasserstein correlation of the joint distribution induced by the augmented encoder, forcing augmented views of an input to be encoded close together without needing an identity anchor. A supporting diagnostic is the aligned collision rate, which measures whether augmented features remain separable by class after a best rigid alignment; it predicts whether MaWa will succeed or face competing constraints.","core_discovery":"The central discovery is that augmentation invariance can be cleanly separated from representation learning: instead of retraining or fine-tuning the encoder, one reshapes its latent space with a small adapter, provided the augmentation-induced displacement is globally coherent. The paper formalizes this through augmented encoders, probabilistic encoders that map each input to the distribution of its augmented and encoded views, and through the definition of $(t,\\mu_X,F,V)$-invariance, which requires the composite $E_\\theta \\circ F$ to ignore collision-free augmentations while agreeing with $F$ up to an admissible structure-preserving map $V$. On feature spaces in which augmented views land near their clean counterparts after a global rigid alignment (low aligned collision rate), the MaWa objective achieves near-isometric behavior, with $R^2=0.96$ and NRMSD $=0.02$ for rotations, and large gains in augmented classification accuracy; the WaCo objective is competitive and can additionally reduce dimensionality. The paper reports that where that coherent-displacement condition fails, as on supervised ResNet50 features, MaWa collapses, and it flags this as a structural limitation in its Section 5.1. The paper further shows that SimCLR and HSIC objectives, two natural candidate losses, produce poor invariance and badly corrupt the original latent point cloud.","pith_inferences":["The aligned collision rate could serve as a cheap pre-training diagnostic: for a new encoder and augmentation, compute it before training, and choose MaWa only when the aligned rate is near zero.","Because the MaWa collapse on supervised ResNet50 happens without any architectural change, the failure mode is likely inherent to anchored objectives; unanchored or class-conditional realignment may be needed for feature spaces where augmentations locally entangle classes.","A natural stress test is an augmentation that preserves each input's class on average but permutes local neighborhoods (e.g., rotating a subset of classes while leaving others fixed); the framework as stated would predict higher aligned collision rates and graceful degradation from MaWa to WaCo.","The near-isometry finding suggests that transport-based dependence measures track geometric structure that contrastive or kernel independence losses discard, which may point to principled structure-preserving objectives for other post-training modifications beyond invariance."],"forward_implications":["Pretrained encoders can be adapted to new augmentations using only unlabeled samples from the original distribution, with no fine-tuning and no access to the training pipeline.","Because the adapter is near-isometric on the clean distribution, downstream tasks built on the original features should behave essentially unchanged, while augmented inputs become usable.","The distinction between MaWa and WaCo gives a practical choice: use MaWa for maximal invariance and structure preservation, and WaCo when dimensionality reduction is also wanted or when collision rates are high.","Augmentations with near-zero aligned collision rates are essentially free wins for the adapter, whereas augmentations with high residual collision rates require the unanchored WaCo loss or a different architectural strategy.","Treating adversarial perturbations as augmentations may yield adversarially robust latent spaces without adversarial training, a direction the paper identifies."],"supporting_citations":[{"why":"Documents that fine-tuning can distort pretrained features, motivating the frozen-encoder adapter approach.","marker":"Kumar et al. (2022)"},{"why":"Supplies the DINO self-supervised objective behind the primary DINOv2 features.","marker":"Caron et al. (2021)"},{"why":"Supplies the DINOv2 model whose frozen features are the primary testbed.","marker":"Oquab et al. (2023)"},{"why":"Provides the SimCLR contrastive loss used as the representative alternative that corrupts the latent space.","marker":"Chen et al. (2020)"},{"why":"Defines HSIC, used as the alternative independence-maximization loss.","marker":"Gretton et al. (2005)"},{"why":"Shows HSIC maximization can drive representation learning, justifying its use as a candidate; the paper then shows it fails post-training.","marker":"Li et al. (2021)"},{"why":"Gives the transport-based correlation framework underlying the WaCo loss.","marker":"Wiesel (2022)"},{"why":"Provides results on transport dependency measures that motivate WaCo's invariance mechanism.","marker":"Nies et al. (2021)"},{"why":"Provides the differentiable augmentation operations used in all experiments.","marker":"Riba et al. (2020)"}],"fun_headline_variants":["Frozen features gain rotation invariance via small adapter","Adapter reshapes latent space; no fine-tuning needed","Lightweight MLP adds invariance to pretrained nets","Rotation-invariant features from one-layer adapter","Post-training invariance without touching weights"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a single global adapter can correct the augmentation-induced displacement of the frozen latent space: after a rigid alignment, augmented features of each class must still sit closer to their own class than to any other, so the MaWa objective faces no competing constraints; the paper's own tables show this condition fails for supervised ResNet50 features.","fun_headline_variants_meta":{"raw":{"variants":["Frozen features gain rotation invariance via small adapter","Adapter reshapes latent space; no fine-tuning needed","Lightweight MLP adds invariance to pretrained nets","Rotation-invariant features from one-layer adapter","Post-training invariance without touching weights"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000201,"raw_usage":{"total_tokens":1480,"prompt_tokens":1147,"completion_tokens":333,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":763,"completion_tokens_details":{"reasoning_tokens":264}},"tokens_in":763,"tokens_out":333,"duration_ms":3090,"temperature":1.0,"reasoning_tokens":264,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:49:36.632281+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a self-supervised encoder with near-zero aligned collision rate for rotations, train a MaWa adapter, and check both rotated classification accuracy and the $R^2$ of the original pairwise-distance scatter; if either fails to reproduce the paper's reported regime (accuracy near clean accuracy and $R^2$ close to 1), the central claim is contradicted. A sharper disconfirming case would be a low-collision encoder where MaWa nevertheless collapses or corrupts the clean latent space.","supporting_citations":[{"cited_title":"Measuring statistical dependence with hilbert-schmidt norms","cited_arxiv_id":null,"evidence_quote":"Defines HSIC, used as the alternative independence-maximization loss."},{"cited_title":"Kornia: an open source differentiable computer vision library for pytorch","cited_arxiv_id":null,"evidence_quote":"Provides the differentiable augmentation operations used in all experiments."}],"review_version":2}