{"id":"b4477e5d-eeec-4bb6-91ce-e2d0449d694f","arxiv_id":"2501.08712","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Transformation representations learned by contrasting image pairs can replace transformation labels in equivariant self-supervised learning and improve downstream transfer.","lead":"This paper introduces a self-supervised method that learns a continuous description of image changes from pairs of original and changed images, then uses that description to train features that keep track of those changes. The approach improves transfer classification and object detection over several contrastive baselines, and also works with complex changes such as AugMix.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The image-invariance of transformation representations is asserted but not guaranteed by the loss; the equivariant objective can be satisfied by a collapsed or image-dependent shortcut, so the central claim needs a direct check.","rationale":"The reader's verdict is conditional, and the weakest assumption identified is exactly the image-invariance and transferability of y_x^t. My stress-test agrees with this identification. The paper's own Table 7 provides indirect evidence that the loss decomposition does not uniquely enforce image-invariance: removing L_trans still yields high transformation classification accuracy, suggesting that f_T can learn a useful but possibly image-dependent mapping without the STL objective. The paper claims in the abstract that the method 'ensures transformation representation is image-invariant', but the loss only encourages invariance on the training distribution with a particular batch construction, and there is no theoretical argument or direct evaluation that this holds for unseen images. The reported tables do not include a direct measurement of image-invariance of y_x^t on held-out images; the transformation prediction in Table 5 and the equivariance metrics in Table 6 are indirect and can be satisfied by an image-dependent f_T if the evaluator uses the same image for deriving and applying the transformation. The concrete test I propose would settle this by directly measuring the invariance property on unseen images and by testing transfer of the transformation vector to a new image. A negative result would not necessarily invalidate all reported gains, but it would weaken the central claim that label-free transformation representations serve the role of labels in equivariant learning; the gains could instead come from the auxiliary losses regularizing f. Therefore the verdict stays CONDITIONAL: the method is plausible and the experiments are suggestive, but the central mechanism needs a direct invariance check before the claim can be accepted unconditionally.","tokens_in":17463,"tokens_out":2108,"duration_ms":20195,"concrete_test":"Directly test image-invariance of transformation representations on held-out images: train STL as described, then freeze f and f_T, and for a fixed transformation t (e.g., crop with fixed parameters and color jitter with fixed parameters), compute y_x^t and y_{x'}^t for many unseen test images. Measure the mean pairwise distance within the same t versus the distance to y_x^{t'} for a different t'. If the same-transformation distances are not significantly smaller than different-transformation distances (e.g., using the same ranking metrics as Table 6, or a simple classification accuracy on test images), the image-invariance hypothesis fails. Also recompute Table 6 after removing the transformation-prediction evaluation protocol and instead evaluating transfer of a fixed y from one image to a new image's equivariant transformation via Eq.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that a label-free transformation representation y_x^t = f_T(f(x), f(t(x))) encodes t independently of x, and that Eq. (9) plus the equivariant objective in Eq. (7) actually learns this. The loss in Eq. (9) aligns y_x^t with y_{x'}^t only for same-transformation pairs drawn in the aligned batch, using an InfoNCE denominator built from other transformation representations in the batch. Nothing in this loss prevents a trivial solution where f collapses or where f_T ignores the pair and outputs a constant or a function of the image identity, especially since f is shared across all three losses and the equivariant loss (12) can be driven down by a constant phi. The paper's own ablation (Table 7) shows STL without L_trans still reaches transformation classification of 69.57% and with L_trans alone (without L_equi) classification of 93.92%, which suggests f_T can be image-dependent yet still pass the stated image-invariance hypothesis. The claim that the transformation representation is image-invariant is therefore not established by the reported experiments; the reported gains could arise from the equivariant branch acting as an auxiliary regularization rather than from a genuinely image-invariant transformation encoding. This is a load-bearing concern because if y_x^t is not image-invariant across unseen images, Eq. (6) transfers a transformation from x' to x incorrectly, and the equivariant alignment in Eq. (12) no longer matches true transformations, so the mechanism proposed in the paper is not what produces the reported gains.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Self-supervised Transformation Learning (STL), a method for learning equivariant representations without transformation labels. STL uses an auxiliary encoder f_T that maps a pair of representations (f(x), f(t(x))) to a transformation representation y_x^t. The training objective combines three InfoNCE losses: an invariant loss (Eq. 11) aligning f(x) and f(t(x)); an equivariant loss (Eq. 12) aligning phi(y_{x'}^t, f(x)) with f(t(x)), using a transformation representation from a different image x' to avoid trivial solutions; and a transformation alignment loss (Eq. 13) that pulls y_x^t and y_{x'}^t together for the same transformation t on different images. The paper reports linear transfer results on 11 downstream datasets, object detection on VOC, transformation prediction quality, and ablations, and claims improvements over existing equivariant SSL methods in 7 of 11 classification tasks.","tokens_in":17810,"tokens_out":6541,"duration_ms":59797,"significance":"The core idea is timely and potentially useful: replacing discrete transformation labels with learned continuous transformation representations could handle composite transformations such as AugMix and capture interdependencies among simple transformations, which label-based equivariant methods cannot. The paper provides a clean formulation, an aligned-batch implementation that keeps computational overhead at about 1.11x, compatibility with several base SSL models, and a set of ablations that show each loss term contributes. The code is released. The main reservations are that the key image-invariance property (Eq. 8) is not directly tested and that the ImageNet100 benchmark tables lack variance information; both concerns are addressable with additional experiments.","major_comments":[{"comment":"The main ImageNet100 results are reported as single accuracy numbers without standard deviations or seed counts, while the STL10 results in Table 11 include mean±std over three seeds. The headline claim of outperforming existing methods in 7 of 11 datasets is sensitive to small differences: for example, in Table 2 STL improves over AugSelf on CIFAR100 by 0.75 points (66.84 vs 66.09), while it is lower on MIT67 (56.64 vs 57.16) and SUN397 (44.69 vs 45.24), and the mean improvement over SimCLR with AugMix is only 0.09 points (64.18 vs 64.09). Without variance information, the 7-of-11 claim cannot be distinguished from run-to-run noise. The authors should report mean±std over at least three seeds for the ImageNet100 experiments, or explicitly state the number of seeds, and discuss whether the reported differences are significant.","section":"§4.1, Tables 2–4"},{"comment":"The image-invariance of the transformation representation is the load-bearing assumption of the method, but it is not directly verified. The loss in Eq. (13) aligns y_x^t and y_{x'}^t only for same-transformation pairs sampled in the aligned batch; because f_T also sees f(x), nothing in the InfoNCE objective prevents y_x^t from encoding image identity or collapsing to a constant. The claim in the abstract that the method 'ensures transformation representation is image-invariant' is therefore not established by the reported experiments. The ablation in Table 7 shows that adding Ltrans improves transformation classification from 69.57 to 93.67, but this demonstrates the importance of the loss term, not that the learned representation satisfies Eq. (8) on unseen images. I ask the authors to add a direct invariance test, such as the variance of y_x^t across many images x for a fixed t, or the accuracy of the equivariant transfer Eq. (6) when y is drawn from held-out images, compared with an image-specific or random baseline. If Eq. (8) fails on unseen images, the equivariant objective in Eq. (12) no longer aligns with true transformations, and the reported gains could reflect an auxiliary regularization effect rather than the proposed mechanism.","section":"§3.2, Eqs. (8), (12), (13); Table 7"}],"minor_comments":[{"comment":"ImageNet100 is listed as having 1,000 classes; the ImageNet100 split used in this paper (from [42]) contains 100 classes. This is a typo but should be corrected.","section":"Appendix B, Table 10"},{"comment":"The sentence 'we usetrainval set for training' is missing a space, and 'pre-taining' in Appendix B should be 'pretraining'.","section":"Appendix C"},{"comment":"The InfoNCE denominator is written as a sum over y_i ≠ y, but the negative set should be explicitly the other batch elements; the shorthand used in Eqs. (11)–(13) is not defined until later. Please clarify the notation.","section":"§3.3, Eq. (10)"},{"comment":"For stochastic transformations such as random crop and color jitter, it is not specified how 'identical transformations' are synchronized across paired images (e.g., shared crop coordinates and jitter parameters). This detail matters for the correctness of Ltrans and Lequi and should be stated explicitly, ideally with a reference to the code.","section":"§3.3, Figure 3"},{"comment":"The ablation study reports only single runs for the STL10-pretrained model; while Table 11 gives mean±std for the full STL, the ablations would be more convincing with repeated seeds.","section":"§4.3, Table 7"}],"recommendation":"major_revision","confidential_remarks":"The paper is close to being acceptable, but the two major comments are essential. Given that the authors already report mean±std for STL10, extending this practice to ImageNet100 and adding a direct invariance metric should be feasible. I would not reject; a major revision is appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nHere's my take on arXiv:2501.08712. STL replaces transformation labels with a contrastively learned transformation representation derived from image pairs. That is a real conceptual step beyond E-SSL, AugSelf, EquiMod, and SIE, which all rely on parameterized labels. It also lets them handle complex transformations like AugMix, which prior label-based equivariant methods cannot. The three-loss decomposition is clean, the ablation in Table 7 shows each loss contributes independently, and the extensions to BYOL, SimSiam, and Barlow Twins are a useful plus. Code is released.\n\nThe soft spots are real but mostly about overclaiming rather than the core mechanism. The paper states the transformation representation is image-invariant, but the loss in Eq. (9) only aligns same-transformation pairs from different images; it does not directly penalize image-specific content in y. The paper never directly measures invariance, e.g., by holding t fixed across many x and checking the variance of y. The ablation in Table 7 (w/o L_trans still reaches 69.57% transformation classification) suggests the equivariant branch alone can extract transformation information, so the story may be closer to 'auxiliary regularization' than to a certified image-invariant encoding. That is a gap worth probing, not a demonstrated failure, because the equivariant loss still forces phi to work on held-out images via L_equi.\n\nThe bigger quantitative concern is that the headline ImageNet100 results in Tables 2-4 have no error bars or seed counts, so the '7 of 11' claim may be within run-to-run noise. The STL10 results in Table 11 are averaged over three seeds and look robust, so this is addressable. The 'label-free' framing is also slightly qualified: the aligned transformed batch construction needs to know transformation identities to pair images, though the paper is transparent about that. Citation pattern is sound; all relevant baselines are covered.\n\nOverall, this is a solid, well-motivated paper with a genuinely new idea. It deserves a serious peer review. I would send it to a referee and ask for direct image-invariance checks and error bars on the main tables.","headline":"STL is a genuinely label-free step for equivariant SSL that deserves review, but the image-invariance claim is under-tested and the main results lack error bars.","tokens_in":18331,"tokens_out":3812,"would_cite":true,"duration_ms":40072,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper proposes replacing transformation labels with learned, image-invariant transformation representations, and reports gains on 7 of 11 classification benchmarks and on object detection.","keywords":["self-supervised learning","equivariant representations","transformation representation","contrastive learning","representation learning","image transformations","transfer learning"],"falsifier":"Train STL with transformations seen during pretraining, then probe held-out images and transformations: if the transformation representation $y_x^t$ changes substantially when the same $t$ is applied to different images, so that intra-transformation variance across images exceeds inter-transformation separation, the image-invariance premise fails. A direct test is to measure whether $\\phi(y_{x'}^t, f(x))$ aligns with $f(t(x))$ when $x$ and $x'$ are drawn from a distribution very different from the pretraining data; if alignment error grows sharply, the equivariant branch is not actually transferring transformations.","tokens_in":17283,"feed_emoji":"🔄","tokens_out":7815,"duration_ms":62676,"temperature":0.7,"pith_summary":"Equivariant representation learning usually needs explicit transformation labels, which forces each transformation to be treated independently and rules out complex, composable transformations like AugMix. This paper proposes Self-supervised Transformation Learning (STL), which replaces the label with a transformation representation computed from the pair of an original image and its transformed version. STL trains that representation to be image-invariant by contrastively aligning the same transformation applied to different images, then uses it to parameterize the equivariant transformation in representation space. The authors show this improves transfer performance over label-based equivariant baselines on 7 of 11 classification benchmarks and on object detection, and that it can use complex transformations that existing equivariant baselines cannot use. The practical payoff is that equivariant learning no longer depends on knowing the transformation group in advance.","feed_headline":"No-label equivariant learning beats labeled baselines on 7 of 11 tasks","feed_subtitle":"A contrastive alignment makes transformation representations image-invariant, unlocking complex augmentations like AugMix.","key_machinery":"The central object is the transformation representation $y_x^t = f_T(f(x), f(t(x)))$, produced by an auxiliary encoder $f_T$ from the representation pair of an image and its transformed version. Its defining property is image-invariance: $y_x^t = y_{x'}^t$ for the same $t$, enforced by the self-supervised transformation loss $L_{\\mathrm{trans}}$ using InfoNCE. The second mechanism is the equivariant transformation $\\phi(y_{x'}^t, f(x))$, a hypernetwork-parameterized map that applies a transformation code obtained from another image $x'$ to the representation of $x$, so that the equivariant branch cannot cheat by copying the input pair. Together they replace discrete transformation labels with continuous codes that can represent composed transformations such as AugMix.","core_discovery":"The central claim is that a learned vector $y_x^t = f_T(f(x), f(t(x)))$ can serve as a continuous, label-free code for the transformation $t$ itself, provided it is trained to be invariant to the choice of image $x$. STL enforces this with a contrastive loss that aligns $y_x^t$ and $y_{x'}^t$ for identical transformations applied to different images, so the representation encodes the transformation rather than the image content. The equivariant branch then applies $\\phi(y_{x'}^t, f(x))$ to a different image $x$, avoiding the trivial solution where the equivariant map would just copy the representation it saw, and aligns the result with $f(t(x))$. On ImageNet100-pretrained ResNet-50, STL reaches 64.18% mean accuracy over 11 downstream classification tasks and 51.95 AP on VOC07+12 detection, outperforming label-based equivariant baselines while keeping batch construction at the same complexity as SimCLR.","pith_inferences":["The image-invariance hypothesis suggests a continuous transformation algebra: if $y_t$ encodes $t$ independently of $x$, operations like composition and interpolation of transformations could be performed in representation space and probed with arithmetic tests, which the paper does not run.","The paper's limitation statement implies that multi-image transformations such as mixup are out of reach; a natural extension would be an encoder over sets of representations that produces a transformation code from more than two images.","The same contrastive-alignment idea could transfer to other modalities where transformations are hard to label, such as audio pitch shifting or time stretching, though the paper only evaluates images.","Because transformation representations are continuous, they could be used as goals in reinforcement learning or as supervisory signals for controllable generation, a use the paper does not explore."],"forward_implications":["Equivariant learning can be applied to transformations that have no discrete label space, such as AugMix, because the transformation is represented by a learned vector rather than a category or parameter vector.","Transformation representations organize by type and intensity in the learned space, so similar color transformations sit close together and intensity levels order continuously, giving a structured transformation geometry.","The same batch configuration as standard contrastive learning suffices, with roughly 10% higher per-iteration cost, so the method can be dropped into existing pipelines without enlarged batches.","STL improves transfer performance across the base models SimCLR, BYOL, SimSiam, and Barlow Twins, meaning the transformation-learning module is a portable addition rather than a redesign.","Object detection also benefits, with STL reaching 51.95 AP_all and 52.70 with AugMix, so the equivariant signal helps downstream tasks that demand precise localization."],"supporting_citations":[{"why":"Supplies the InfoNCE loss and the base invariant learning framework that STL extends.","marker":"[4]"},{"why":"EquiMod is the explicit label-conditioned equivariant baseline whose label dependence STL removes.","marker":"[9]"},{"why":"SIE provides the hypernetwork-based equivariant transformation architecture that STL reuses with transformation representations instead of labels.","marker":"[13]"},{"why":"AugSelf is the implicit equivariant baseline that predicts transformation parameters from representation pairs.","marker":"[28]"},{"why":"AugMix is the complex composed transformation that STL handles but label-based equivariant methods cannot.","marker":"[23]"},{"why":"Supplies the detection evaluation protocol and serves as one of the base invariant models.","marker":"[48]"},{"why":"VOC07+12 is the object detection benchmark used to measure transfer performance.","marker":"[11]"}],"fun_headline_variants":["Label-free transformation codes beat labeled equivariant models","STL learns equivariance without transformation labels, wins 7/11","Self-supervised transformation codes unlock AugMix and top benchmarks","Equivariant learning without labels: STL surpasses on detection and more","Transformation representations replace labels for better equivariance"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The approach rests on the assumption that a transformation representation computed from one image pair actually encodes the transformation independently of the image content, and that contrastively aligning same-transformation pairs across different images is enough to make that true in practice.","fun_headline_variants_meta":{"raw":{"variants":["Label-free transformation codes beat labeled equivariant models","STL learns equivariance without transformation labels, wins 7/11","Self-supervised transformation codes unlock AugMix and top benchmarks","Equivariant learning without labels: STL surpasses on detection and more","Transformation representations replace labels for better equivariance"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00033,"raw_usage":{"total_tokens":1849,"prompt_tokens":968,"completion_tokens":881,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":584,"completion_tokens_details":{"reasoning_tokens":797}},"tokens_in":584,"tokens_out":881,"duration_ms":8249,"temperature":1.0,"reasoning_tokens":797,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:19:51.545563+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train STL with transformations seen during pretraining, then probe held-out images and transformations: if the transformation representation $y_x^t$ changes substantially when the same $t$ is applied to different images, so that intra-transformation variance across images exceeds inter-transformation separation, the image-invariance premise fails. A direct test is to measure whether $\\phi(y_{x'}^t, f(x))$ aligns with $f(t(x))$ when $x$ and $x'$ are drawn from a distribution very different from the pretraining data; if alignment error grows sharply, the equivariant branch is not actually transferring transformations.","supporting_citations":[{"cited_title":"EquiMod: An Equivariance Module to Improve Self-Supervised Learning","cited_arxiv_id":"2211.01244","evidence_quote":"EquiMod is the explicit label-conditioned equivariant baseline whose label dependence STL removes."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"AugSelf is the implicit equivariant baseline that predicts transformation parameters from representation pairs."},{"cited_title":"Zbontar, L","cited_arxiv_id":null,"evidence_quote":"Supplies the detection evaluation protocol and serves as one of the base invariant models."},{"cited_title":"Everingham, L","cited_arxiv_id":null,"evidence_quote":"VOC07+12 is the object detection benchmark used to measure transfer performance."}],"review_version":1}