{"id":"84998aaf-95f3-4ffe-877d-afa731846ef3","arxiv_id":"2607.04044","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"SiamJEPA, a masked-image JEPA variant with Siamese student encoders and an EMA teacher, improves ImageNet linear probing accuracy over a JEPA-like baseline and beats MAE at 400 epochs.","lead":"This paper tests whether using two masked views through a shared Siamese encoder improves a JEPA-style self-supervised vision model on ImageNet linear probing, reporting faster convergence and higher accuracy than a weak JEPA-like baseline. It could matter to anyone designing self-supervised representation learners because it suggests the Siamese pair acts as a regularizer rather than just an architectural detail.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper varies λKL inside a two-branch Siamese architecture but never trains a true single-encoder JEPA; the central comparison to 'single-encoder JEPA variants' is therefore unsupported.","rationale":"Read in good faith: the paper's main empirical contribution is a systematic ablation of λKL, free-bits, weight decay, masking, pooling, and EMA in its two-branch model. Those tables are informative even if the headline overreaches. The single-seed results, hyperparameters selected on the same benchmark, and the MAE pooling mismatch are secondary because they affect the MAE comparison more than the core 'Siamese vs single encoder' question. The decisive issue is that the paper compares λKL≈0 within the Siamese model to λKL=0.01 within the same Siamese model and calls the former 'JEPA-like.' That is not a single-encoder baseline. The posterior q in §3.2 conditions on both H_cls^(1) and H_cls^(2), so the two branches interact in the predictor even when λKL=0. The claim that λKL=0 recovers a single-encoder JEPA is an asserted limit, not a derivation or experiment. Since the abstract and conclusion specifically attribute the benefit to the architectural inductive bias of Siamese student encoders, this missing condition is load-bearing. The concrete test above would settle it. I agree with the reader's identification, and the verdict remains CONDITIONAL pending that test.","tokens_in":65739,"tokens_out":5773,"duration_ms":66525,"concrete_test":"Use the same MAE-based codebase and training budget (ViT-B, batch 8192, LR 1.5e-4, EMA schedule, 400 epochs, ImageNet-1K) to train a genuine single-encoder JEPA: one student encoder f encodes a context mask M1; a predictor g maps h(H_cls) plus a latent variable to teacher target latents for the masked set \\bar M = M \\ M1, with loss (1/|\\bar M|) || Mask(Y_patch - \\hat Y_patch, \\bar M) ||^2 and no second branch or posterior coupling. Evaluate with the same mean-pooled 10th-layer linear probe at epochs 101 and 400. If this single-encoder baseline reaches ≥63.4% / ≥70.2% (SiamJEPA λKL=0.01), the claimed single-encoder advantage is not established; if it is clearly below, the claim gains direct support. Also compare against SiamJEPA λKL=0.00001 to separate architecture from KL strength.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that Siamese student encoders are an important inductive bias and outperform single-encoder JEPA. The evidence is Table 2/Table 3: SiamJEPA with λKL=0.01 reaches 70.15% vs 68.91% for SiamJEPA with λKL=0.00001, which is labeled 'JEPA-like.' But that baseline is not a single-encoder JEPA. In §3.2–3.3, the model always has two shared student encoders f applied to disjoint masks, and the predictor g uses posterior q(Z^(1) | h(H_cls^(1)), H_cls^(2)) (and the symmetric term), so both branches enter the prediction path even when λKL≈0. Varying λKL changes a KL-regularization weight, not the number of student encoders. The assertion in §3.3 that λKL=0 makes the model 'similar to that of single encoder JEPA' is therefore not demonstrated; the posterior coupling prevents that limit. No actual single-encoder JEPA (e.g., an I-JEPA-style model in the same MAE codebase) is trained anywhere in the paper. Thus the abstract's and conclusion's claims about the architectural benefit of Siamese student encoders rest on an unverified equivalence. The ablation itself is internally consistent and may support a more modest claim about KL regularization strength within the two-branch model, but it does not support the headline comparison.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SiamJEPA, a JEPA-style self-supervised model with two Siamese student encoders processing disjoint masked views, an EMA teacher encoder, a KL consistency term between the two student branches, and a latent prediction objective with a probabilistic predictor. The method is evaluated by ImageNet-1K linear probing with ViT-Base, using the MAE codebase as a common framework. The central empirical claim is that Siamese student encoders act as a regularizer: increasing the KL weight from λKL=1e-5 to 0.01 improves linear probing accuracy and accelerates early learning, and the model reaches about 70.15% at 400 epochs versus 68.91% for the JEPA-like baseline. The paper also compares against MAE, CAE, and I-JEPA, and reports ablations of mask type, weight decay, free-bit threshold, learning rate, pooling strategy, EMA schedule, and predictor depth.","tokens_in":66084,"tokens_out":5529,"duration_ms":63563,"significance":"If the claims are supported, the paper would provide a useful empirical study of an architectural inductive bias in latent predictive models, with the attractive practical finding that a small KL-consistency term improves representation quality and convergence speed within a two-branch architecture. The work is reproducible in spirit: the code URL is given, the implementation is based on the public MAE codebase, and the ablations are extensive, including per-epoch linear probing curves. However, the headline architectural claim — that Siamese student encoders outperform ``single-encoder JEPA variants'' — is not tested, because no single-encoder JEPA is trained. The internal comparison of λKL values is consistent and supports a more modest claim about KL regularization strength, but the paper's abstract and conclusion assert a broader result.","major_comments":[{"comment":"The central comparison to ``single-encoder JEPA'' is not made. The baseline labeled JEPA-like is SiamJEPA with λKL=0.00001, which still contains two Siamese student encoders and, crucially, the posterior q(Z(1)|h(H_cls(1)), H_cls(2)) defined in §3.2 conditions on both branches and feeds both into the predictor g. Even as λKL→0, the model does not reduce to a single-encoder JEPA; the posterior coupling remains. The sentence in §3.3 that λKL=0 makes the model ``similar to that of single encoder JEPA'' is therefore asserted, not demonstrated. Since no I-JEPA-style single-encoder model is trained in the same codebase, the abstract's ``consistently outperforms comparable single-encoder JEPA variants'' and the conclusion's ``corresponding JEPA-like baseline'' overstate the evidence. The paper should either train a genuine single-encoder baseline (e.g., removing the second branch and conditioni","section":"§3.3, Table 2"},{"comment":"The configuration of the headline SiamJEPA result is internally inconsistent. §4.1 states that all SiamJEPA models are evaluated with mean pooling of the 10th layer, but Table 2's note says the JEPA-like baseline uses mean pooling of the 10th layer while SiamJEPA uses mean pooling of the final (12th) layer. More importantly, Table 2 reports 70.7% for SiamJEPA with λKL=0.01 and weight decay 0.05 at epoch 400, but Table 3 (λKL=0.010, weight decay 0.05, 10th layer) reports 69.33%, and Table 7 (weight decay 0.1) reports 70.15% for the 10th layer and 70.71% for the 12th layer. The value 70.7 does not match any clearly specified row. The authors must harmonize the evaluation protocol and report the exact configuration (layer, pooling, weight decay) for every number in Table 2.","section":"Table 2 vs Tables 3 and 7, §4.1"},{"comment":"The training-efficiency comparison with MAE conflates epochs with compute. Table 1 shows MAE uses an effective batch size of 4096 while SiamJEPA uses 8192, so at 400 epochs SiamJEPA processes twice as many images. The paper's statement that SiamJEPA outperforms MAE ``using less than one-quarter of the training epochs'' ignores this difference. If the claim is about limited training budgets, the comparison should report wall-clock time or FLOPs, or match the total number of samples processed. As written, the efficiency claim is not fully supported.","section":"§4.2, Table 1"},{"comment":"The strongest comparison — 70.15% for λKL=0.01 versus 68.91% for λKL=0.00001 — also changes weight decay: the λKL=0.01 run uses weight decay 0.1 while the weak-KL baseline uses weight decay 0.05. Table 3 does include a λKL=0.00001, weight-decay-0.1 run, but it collapses to 62.54%, so the interaction is real. However, the paper should make this interaction explicit in the text and in Table 2, since the headline gap is not purely attributable to λKL. A clean same-weight-decay comparison at 0.05 gives 69.33% versus 68.91%, a much smaller difference, which should be reported in the main comparison so readers can see the effect size.","section":"Table 3, §4.3"}],"minor_comments":[{"comment":"The text refers to ``Figure 5 shows the ablation study of masking strategies,'' but the actual results are in Table 5. The reference should be corrected.","section":"§4.5"},{"comment":"The text says ``SiamJEPA model with λKL = 10−4 as a JEPA-like method,'' but the tables use λKL = 10−5 (0.00001). This is a factor-of-ten discrepancy and should be fixed.","section":"§3.3"},{"comment":"There is a duplicated ``where where'' in the decomposition of the encoder outputs. Minor typo.","section":"§3.1"},{"comment":"The predictor-depth discussion is honestly hedged, but the conclusion in §5 still says the Siamese architecture ``serves as an effective regularizer.'' The open questions in §4.9 and §6 (small scale, sensitivity to implementation details, predictor-depth uncertainty) should be reflected more prominently in the abstract, which currently states the architectural claim without qualification.","section":"§4.9"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is transparent that SiamJEPA is essentially masked PhiNetv2 and that the KL/stop-gradient recipe is from the authors' prior work. The novelty is mostly empirical, and the missing single-encoder baseline is a load-bearing gap for the title and abstract claims. I believe the paper can be made acceptable by training a true single-encoder JEPA baseline in the same codebase, harmonizing the inconsistent tables, and reframing the claims to what the experiments actually show. The internal λKL ablation is the strongest evidence and should be foregrounded."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The empirical core is believable: in this two-branch Siamese setup, bumping the KL regularization weight from 0.00001 to 0.01 or 0.03 improves ImageNet linear probing at every epoch, and block masking beats random masking. Those ablations are internally consistent and the paper is honest about the corrected pooling detail and the hyperparameter sensitivity. The source code is public. That part earns credit.\n\nThe problem is the framing. The paper claims Siamese student encoders outperform single-encoder JEPA variants, but no single-encoder JEPA is ever trained. The baseline is the same two-branch SiamJEPA with λKL≈0. That is not a single-encoder model: both branches still feed the posterior, and the predictor is coupled to both views. Setting λKL=0 does not remove the second encoder from the prediction path, so the abstract's and conclusion's architectural claims rest on an asserted equivalence that is not demonstrated. The stress-test note gets this right. The paper itself even says in §3.3 that λKL=0 makes the model \"similar to that of single encoder JEPA,\" but similarity is not the controlled comparison needed.\n\nOther soft spots are proportional and manageable: single-seed results without error bars (though the trend across λKL values is consistent), hyperparameters selected on the same benchmark used for the final comparison, and the MAE comparison using different pooling (CLS vs mean) — a real but minor mismatch, since the gain over MAE is not the paper's main point. The architecture is largely a masked-input PhiNetv2 with KL/EMA machinery from the authors' prior work, so novelty is modest, but the specific ablation question is reasonably new and the answer is plausibly useful to people building JEPA variants for video or world models.\n\nThe paper deserves peer review but needs major revision before acceptance: train an actual single-encoder baseline in the same codebase, or rewrite the claims to be about KL regularization strength within a two-branch architecture. The current evidence supports the modest claim, not the strong one. I would read a revised version; I would not cite this version.","headline":"Useful ablation inside a two-branch model, but the headline claim about single-encoder JEPA is not supported by the experiments.","tokens_in":66577,"tokens_out":1456,"would_cite":false,"duration_ms":20767,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Giving a JEPA model two student encoders and aligning the two views improves its representations and speeds early learning.","keywords":["self-supervised learning","JEPA","Siamese student encoders","masked latent prediction","EMA teacher","KL regularization","ImageNet linear probing","vision transformer"],"falsifier":"Train a genuine single-encoder JEPA with the same codebase, data, batch size, and 400-epoch budget, and evaluate it with the same linear probe. If it reaches or outperforms SiamJEPA's 70.15% without the second student encoder, then the Siamese encoders are not the source of the gain; conversely, reproducing 70.15% only when the KL term is active would support the paper's regularizer claim.","tokens_in":65586,"feed_emoji":"🧠","tokens_out":7366,"duration_ms":80283,"temperature":0.7,"pith_summary":"The paper asks what two student encoders contribute to a joint-embedding predictive architecture (JEPA), and answers that they act as a regularizer for the latent-prediction objective. SiamJEPA feeds two disjointly masked views of an image to a weight-shared student pair, predicts the EMA teacher's masked-token latents, and adds a KL term that aligns probabilistic latents inferred from the two views. The paper reports that raising this KL term improves ImageNet linear probing from 68.91% to 70.15% at 400 epochs, and accelerates early training: at epoch 101 the regularized model reaches 63.44% versus 60.88%. If right, the result matters because Siamese consistency becomes a tunable inductive bias that other JEPA-style models could adopt without changing the prediction task.","feed_headline":"Twin student encoders lift JEPA accuracy by 1.2 points","feed_subtitle":"A KL consistency term between the two masked views speeds early learning and reaches 70.15 percent at 400 epochs.","key_machinery":"The load-bearing object is the Siamese student pair with its KL-regularized probabilistic predictor. Two masked views produce two latent representations; a posterior distribution is built from both views, a prior from only one, and the KL loss (with stop-gradient on the prior) makes each single view predictive of the information shared across views. λKL continuously interpolates between the Siamese configuration and a JEPA-like configuration, and the EMA teacher provides the latent targets for the normalized-MSE masked-prediction term (Sim-2). Disjoint masking prevents shortcut solutions by ensuring the two views are complementary.","core_discovery":"SiamJEPA is a masked latent-prediction architecture in which two student encoders with shared weights process independently and disjointly masked views of the same image. An EMA teacher produces targets for masked-token prediction, while a KL term (Sim-1) forces the probabilistic latent variable inferred from both views to match the prior inferred from each single view, with a stop-gradient on the prior. The central claim is that this KL consistency term is a regularizer, not a nuisance: with λKL=0.00001 the model behaves like a JEPA-style baseline, and raising λKL to 0.01 improves ImageNet linear probing from 68.91% to 70.15% after 400 epochs and accelerates early-stage learning (63.44% vs","pith_inferences":["The paper never trains a true single-encoder JEPA in its own codebase; a direct run of such a model would settle whether the reported gain comes from having two encoders or from the KL regularizer working within a two-encoder setup.","If the KL term is mainly a regularizer, simpler alignment objectives, such as cosine similarity between pooled view representations, might reproduce much of the gain at lower computational cost; this is a natural testable extension.","The largest gap appears early in training, so the benefit may be most valuable at short pretraining budgets or in continual-learning settings where rapid representation formation matters.","The paper itself flags that its findings may not transfer to larger backbones or deeper predictors; testing SiamJEPA at larger scale and with deeper predictors is the open check implied by its own discussion."],"forward_implications":["If the KL term is a genuine regularizer, turning it up should keep improving representation separability up to a point; the paper finds λKL values of 0.01–0.03 beat 0.00001 across all training lengths.","The early-training gap (63.44% vs 60.88% at epoch 101) means a Siamese-regularized JEPA can reach a given linear-probe accuracy in roughly half the epochs of its weakly regularized counterpart.","Larger weight decay becomes useful for SiamJEPA under long training, while the weakly regularized baseline plateaus or collapses, suggesting the Siamese regularizer interacts positively with other forms of regularization.","Because the KL term is decoupled from the masked-prediction loss, the recipe should transfer to other JEPA-style architectures and modalities if the paper's inductive-bias claim is correct.","Even simple random masking becomes competitive with block masking when the mask ratio is tuned, indicating that the Siamese consistency, not the masking sophistication, drives the gain."],"fun_headline_variants":["Siamese encoders regularize JEPA, lifting accuracy 1.2 points","Twin student encoders speed JEPA learning and boost linear probe","KL consistency term in JEPA acts as regularizer, gains 1.2%","SiamJEPA: two student encoders improve JEPA by 1.2 points","JEPA with Siamese encoders hits 70.15% linear probe accuracy"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The central comparison assumes SiamJEPA with λKL=0.00001 behaves as a JEPA-like baseline, yet the two Siamese branches and the KL term remain in the model; no true single-encoder JEPA is trained, so the paper's claim that Siamese encoders beat single-encoder JEPA depends on an equivalence asserted in §3.3 and Table 2 rather than demonstrated.","fun_headline_variants_meta":{"raw":{"variants":["Siamese encoders regularize JEPA, lifting accuracy 1.2 points","Twin student encoders speed JEPA learning and boost linear probe","KL consistency term in JEPA acts as regularizer, gains 1.2%","SiamJEPA: two student encoders improve JEPA by 1.2 points","JEPA with Siamese encoders hits 70.15% linear probe accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000318,"raw_usage":{"total_tokens":1691,"prompt_tokens":859,"completion_tokens":832,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":603,"completion_tokens_details":{"reasoning_tokens":726}},"tokens_in":603,"tokens_out":832,"duration_ms":9197,"temperature":1.0,"reasoning_tokens":726,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T08:40:30.173877+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a genuine single-encoder JEPA with the same codebase, data, batch size, and 400-epoch budget, and evaluate it with the same linear probe. If it reaches or outperforms SiamJEPA's 70.15% without the second student encoder, then the Siamese encoders are not the source of the gain; conversely, reproducing 70.15% only when the KL term is active would support the paper's regularizer claim.","supporting_citations":[],"review_version":2}