{"id":"8b4ec351-c1c6-4c85-9b0c-f426a17d055b","arxiv_id":"2411.13036","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"AltO trains homography networks on unlabeled multimodal image pairs by alternating a spatial Barlow Twins loss for geometry with a Barlow Twins loss for modality alignment.","lead":"This paper introduces AltO, a training method that aligns pairs of images from different sensors or styles without needing labeled ground-truth homographies. It alternates between teaching the network to match geometry and teaching it to ignore modality differences, and reports large gains over prior unsupervised methods on satellite, seasonal, and RGB-NIR image pairs.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim is broadly supported, but the GL-phase objective can be minimized by a degenerate homography that maps the moving image onto a constant-texture region when the fixed image contains low-texture areas; the paper does not test this failure mode, and the ablations do not rule it out.","rationale":"The paper makes a credible empirical contribution with clear ablations and large margins over baselines. The reader correctly identified the trivial-solution concern as the weakest assumption, and I agree with the conditional verdict. However, my analysis shifts the emphasis: the more specific and testable risk is not only the constant-encoder/identity-homography collapse, but the possibility that the GL loss itself rewards a degenerate non-identity alignment when the fixed image contains low-texture regions. This is a concrete failure mode that the current ablations do not rule out, and it is directly load-bearing for the claim that AltO trains homography estimators without ground truth. I would keep the verdict CONDITIONAL, with the condition being that the authors either (a) demonstrate robustness to degenerate initializations or provide a mechanism that prevents the collapse, or (b) add an experiment varying texture and initial displacement to bound the regimes where AltO succeeds. I do not see evidence of internal inconsistency: the EM analogy, the GBT loss, and the GAP design are clearly described, and the ablation tables support the importance of alternation and GAP. The absence of error bars and the test-set-based architecture selection are legitimate concerns but do not undermine the central claim as severely as the unverified collapse-resistance assumption.","tokens_in":13439,"tokens_out":1655,"duration_ms":15169,"concrete_test":"Run a targeted degenerate-warp experiment on Google Map and Deep NIR: initialize the registration network to predict a homography that maps the moving image onto a low-texture or border region of the fixed image (e.g., a large translation toward a uniform area), disable MARL, and train with only the GL loss for a few epochs. If Lg decreases and the predicted homography remains degenerate, the GL objective admits non-identity collapse, and the alternation's robustness claim is incomplete. If the network recovers the correct alignment even in this initialization, the concern is mitigated. Also report MACE with standard deviation over at least three random seeds.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that alternating GL/MARL phases train a registration network on multimodal pairs without ground truth. What must hold is that the GL objective (Eq. 5) is a faithful proxy for geometric alignment, not merely a scalar that can be reduced by a collapsed warp. Section 4.2 defines Lg by comparing spatial feature statistics of warped moving image eI_A and fixed image I_B, with redundancy reduction across channels. This objective is invariant to a homography that maps the moving image onto a region of the fixed image whose feature statistics match \"averaged\" content, and it is minimized by aligning statistical summaries rather than pixelwise structure. The only evidence against collapse is the alternating schedule and GAP ablations in Sections 6.1-6.2, both on Google Map. No experiment varies texture content, occlusion, or initial displacement magnitude. The trivial-solution concern the authors identify in Section 3.2 is the constant-encoder/identity-homography collapse; the complementary failure mode, in which the registration network actively maps onto a low-information region while the encoder statistics remain consistent, is not analyzed. The practical-negligibility claim in Section 5.4 for the supervised gap is also unsupported: no perceptual or downstream task evaluation is provided, and a 3x larger corner error can be material in stitching. However, this is secondary; the primary load-bearing concern is the unverified assumption that Lg cannot be minimized by a degenerate non-identity homography under GL-only optimization, which the alternation does not explicitly forbid.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes AltO, an unsupervised framework for homography estimation on multimodal image pairs (e.g., satellite/map, RGB/NIR). The training alternates two phases: a Geometry Learning (GL) phase that trains the registration network with a proposed Geometry Barlow Twins (GBT) loss on spatial feature maps, and a Modality-Agnostic Representation Learning (MARL) phase that trains an encoder/projector with a standard Barlow Twins loss. The framework is evaluated on Google Map, Google Earth, and Deep NIR datasets using four different registration backbones (DHN, RAFT, IHN-1, RHWF-1). The authors report that AltO outperforms existing unsupervised methods and approaches supervised performance, and they provide ablations on alternating optimization, global average pooling, loss-function combinations, and encoder/projector architecture.","tokens_in":13776,"tokens_out":5179,"duration_ms":53213,"significance":"If the central claim holds, AltO would be a notable step for unsupervised multimodal homography estimation: it decouples the geometry and modality gaps in a principled alternating scheme, works across several registration architectures, and comes with released source code. The MACE numbers in Table 1 are substantially better than the existing unsupervised baselines on all three benchmarks, and the consistency across four backbone architectures is a genuine strength. However, the empirical claims are currently supported by single-run results, and one load-bearing failure mode of the GBT loss is not tested, so the significance is conditional on those points being resolved.","major_comments":[{"comment":"The GBT loss Lg computes normalized cross-correlations over spatial positions but does not constrain the absolute feature magnitude or the information content of the region onto which the moving image is warped. If the fixed image contains low-texture areas, the loss can in principle be minimized by a homography that maps the moving image onto such a region, since the diagonal correlation term (1-C(n,ii))^2 can be driven to zero for low-variance features. The paper's trivial-solution discussion in §3.2 covers only the constant-encoder/identity-homography collapse, and the ablations in §6.1–6.2 do not vary texture content, occlusion, or initial displacement magnitude to rule out this complementary failure mode. Please add experiments (e.g., synthetic pairs with low-texture fixed images, or monitoring of the region where the warped image lands) that demonstrate the GL phase cannot collapse to low-information regions, or otherwise modify the loss to explicitly penalize such degenerate warps.","section":"§4.2, Eq. (5)"},{"comment":"All quantitative claims rest on single-run MACE values with no error bars, confidence intervals, or number of seeds. Given that the reported performance differences between AltO and the strongest unsupervised baselines are sometimes large but the variance is unknown, the claim that AltO 'outperforms' existing methods is not yet statistically supported. Please report mean and standard deviation over at least three seeds, and state the evaluation protocol used for the baseline numbers.","section":"Table 1 and §5.3"},{"comment":"The architecture split in Table 5 is selected using the Google Map test set, because no validation split is described; this test-set selection can inflate the reported numbers and weaken the claim that the chosen Encoder(1,2)/Projector(3) split is optimal. In addition, Table 2 omits the 'Alternating without MARL' condition, so the ablation cannot separate the contribution of alternating optimization from the contribution of the MARL phase. Please add the missing condition and perform the ablations on a held-out validation split.","section":"§6.4 and §6.1"}],"minor_comments":[{"comment":"The statement that the remaining gap to supervised methods is 'negligible in practical applications' is not supported by any perceptual or downstream-task evaluation; a roughly three-fold larger corner error can be material in stitching or fusion. Either soften the claim or add a quantitative downstream evaluation.","section":"§5.4"},{"comment":"The notation for the feature tensors is inconsistent: the text describes f_A as 'feature vectors' in R^{N×D×H×W}, and the definition 'subtracting the spatial mean from each unit' is ambiguous about which axis is centered. Please clarify by defining the normalization explicitly over h,w for each (n,i).","section":"§4.2, Eq. (5)"},{"comment":"The explanation that GAP prevents the trivial solution because 'local features are forced to be similar' is speculative; a more direct diagnostic, such as feature statistics before and after the projector with and without GAP, would make the claim more convincing.","section":"§6.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for a computer vision venue, and the code release plus compatibility across four registration architectures are strengths. The main technical risk is the degenerate-warp collapse of the GL loss, which I would like to see addressed with targeted experiments before acceptance. The statistical-reporting gap (no seeds, no error bars) and the test-set selection of the architecture in §6.4 also need to be fixed. I found no evidence of citation manipulation or self-citation driving the results."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: AltO is a real step forward for unsupervised homography on multimodal pairs. The Geometry Barlow Twins loss—treating spatial positions as the batch dimension—and the alternating GL/MARL schedule are genuinely new, and the empirical margins over prior unsupervised baselines are large and consistent across three datasets and four backbones. The ablations are sensible: alternating optimization and GAP both matter, and the loss-combination study is thorough. Code is public. That is a solid contribution.\n\nThe soft spots are mostly about evidence quality, not the core idea. Table 1 has no error bars or seed counts, so we can't tell if the margins are stable. Section 6.4 picks the encoder/projector split by testing on Google Map without any described validation split, which risks tuning to the test set; that's a real methodological leak. And the claim that the residual gap to supervised methods is 'negligible in practical applications' is asserted, not demonstrated—no stitching or downstream task is evaluated, and a 3x corner error could matter. The authors do acknowledge the gap and slower training in the limitation section, which helps.\n\nThe stress-test concern about the GL objective being gameable by a degenerate warp is worth taking seriously, but it didn't land as fatal on reading. GBT compares global spatial correlation per channel with mean normalization; a warp onto a constant-texture region would produce near-zero activations after normalization, which doesn't obviously reduce the loss. The more defensible version of the concern is that global spatial correlation is a weak proxy for geometric alignment, and the paper provides no failure-mode analysis for low-texture or repetitive scenes. Given the benchmarks show large, consistent improvements, I'd call that a discussion gap rather than a load-bearing flaw.\n\nBottom line: this paper deserves a careful referee. It's a genuine new method with reproducible code and strong empirical support, but the missing error bars, validation leakage, and the unsupported negligibility claim should be fixed in revision. I'd bring it to a reading group and would cite it in my own work.","headline":"A genuinely new unsupervised multimodal homography method with strong empirical results, but the global-statistic geometry loss and missing error bars keep it from being fully settled.","tokens_in":14304,"tokens_out":3573,"would_cite":true,"duration_ms":38156,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"AltO trains homography estimators on multimodal image pairs with no ground-truth homography, using alternating optimization to separate geometry and modality gaps.","keywords":["homography estimation","multimodal image alignment","unsupervised learning","alternating optimization","Barlow Twins","Geometry Barlow Twins","image registration","modality-agnostic representation"],"falsifier":"Train IHN-1 with AltO on a new multimodal pair dataset with a large modality gap using the same hyperparameters; if the mean corner error stays near the roughly 24-pixel identity-warp baseline, or the encoder's output becomes near-constant while the registration network predicts near-identity homographies, the alternating schedule failed to prevent collapse and the paper's central claim is undercut.","tokens_in":13269,"feed_emoji":"🗺️","tokens_out":7600,"duration_ms":66858,"temperature":0.7,"pith_summary":"AltO is an unsupervised learning framework that estimates the homography between two images—the planar projective transform aligning two views—when the images come from different modalities, such as map versus satellite or RGB versus near-infrared, and no ground-truth homography is available. The paper argues that existing unsupervised methods fail on such pairs because they try to close the geometry gap and the modality gap simultaneously, which collapses into a trivial solution where the encoder produces a constant output and the network predicts the identity warp. AltO avoids this by alternating two optimization phases, similar to Expectation-Maximization: a Geometry Learning phase trains the registration network with a spatial variant of the Barlow Twins loss, and a Modality-Agnostic Representation Learning phase trains the encoder and projector with the standard Barlow Twins loss. On Google Map, Google Earth, and Deep NIR, AltO beats other unsupervised methods and approaches supervised performance; for example, IHN-1 trained with AltO reaches a mean corner error of 3.06 on Google Map compared with 0.92 for supervised IHN-1. This matters because ground-truth homographies are hard to collect in real stitching, fusion, and remote-sensing settings.","feed_headline":"AltO trains image alignment on multimodal pairs without labels","feed_subtitle":"Its alternating geometry and modality phases beat other unsupervised methods and near supervised accuracy on three benchmarks.","key_machinery":"The machinery is the alternating two-phase optimization schedule together with the two Barlow Twins-derived losses. Geometry Barlow Twins extends the Barlow Twins objective to 2D feature maps by treating the spatial locations (height and width) as samples, so the cross-correlation between channels of the warped and fixed feature maps is pulled toward the identity matrix, aligning local geometric features without requiring exact pixel equality. The Modality-Agnostic Representation Learning phase applies the standard Barlow Twins loss to global-average-pooled embeddings, making the representation space shared across modalities. Alternation is the anti-collapse device: each phase trains only one part of the model while freezing the other, so the encoder and the registration network cannot jointly 'solve' the loss by outputting a constant and an identity warp. Global average pooling is also load-bearing; without it, the MARL phase forces fine local features to match before the registration network is trained, and the ablation in Section 6.2 shows this re-introduces the trivial solution.","core_discovery":"The paper's central claim is that the trivial-solution collapse that ruins naive unsupervised homography training on multimodal pairs can be avoided by an alternating schedule, so a registration network can be trained without any ground-truth homography even when the input pair spans different domains. In the Geometry Learning phase, the registration network is trained by warping the moving image and maximizing the similarity between its features and those of the fixed image using the proposed Geometry Barlow Twins loss, which reinterprets the spatial dimensions of feature maps as the batch dimension and drives the channel cross-correlation matrix toward the identity. In the Modality-Agnostic Representation Learning phase, with the registration network frozen, the shared encoder and projector are trained with the original Barlow Twins loss so the two modalities map into a common feature space. The two phases alternate within each mini-batch, and the paper reports that this, together with global average pooling in the projector, prevents the encoder from collapsing and the network from predicting the identity homography. The framework is architecture-agnostic: the same alternating procedure trains DHN, RAFT, IHN-1, and RHWF-1, and in each case it outperforms existing unsupervised methods by a wide margin while approaching supervised accuracy.","pith_inferences":["An implication the paper leaves implicit is that its alternating schedule is essentially a coordinate-descent heuristic with no formal convergence or anti-collapse guarantee; proving a condition under which alternation avoids the trivial solution would turn the empirical recipe into a general principle.","The GAP requirement suggests a resolution-versus-representation trade-off: a testable extension would be replacing GAP with positional pooling or a lightweight attention pooling to see whether multimodal alignment can be pushed to higher-resolution images without collapse.","Because AltO is label-free and architecture-agnostic, the same two-phase schedule may transfer to other geometric alignment tasks on multimodal data, such as optical flow or stereo disparity, although the paper only demonstrates homography.","The residual gap to supervised performance (for instance 3.06 versus 0.92 on Google Map) suggests the modality-agnostic representation still loses some spatial precision; a concrete next experiment is to couple AltO with a refinement stage that uses the original image resolution."],"forward_implications":["AltO can replace ground-truth homography labels during training for any compatible registration network, and the reported mean corner errors approach those of supervised training of the same network (IHN-1+AltO: 3.06 versus 0.92 on Google Map, 1.82 versus 1.60 on Google Earth, 3.11 versus 2.11 on Deep NIR).","Without the alternating schedule, training fails: Table 2 shows MACE values around 24 pixels (no better than an identity warp) for every tested backbone, while the alternating version drops to roughly 3-6 pixels.","Removing global average pooling from the projector also collapses training (MACE stays near 24 pixels), so the MARL phase must compare global rather than fine local features until the registration network is ready.","Because the framework is architecture-agnostic, stronger or future registration networks can be plugged in directly and inherit the same unsupervised training procedure.","The label-free regime extends the practical reach of homography estimation to multimodal stitching, multispectral fusion, and remote-sensing alignment, where collecting ground-truth warps is impractical."],"supporting_citations":[{"why":"Provides the Barlow Twins redundancy-reduction objective from which both the modality loss and the proposed Geometry Barlow Twins loss are derived.","marker":"[12]"},{"why":"Supplies the Google Map and Google Earth multimodal datasets used for training and evaluation.","marker":"[11]"},{"why":"Supplies the Deep NIR multimodal dataset used for training and evaluation.","marker":"[31]"},{"why":"Is the first unsupervised deep homography baseline whose pixel-space reconstruction fails on modality gaps and is the main comparison point.","marker":"[9]"},{"why":"Provides the perceptual-loss unsupervised baseline and the ResNet-34 encoder/projector split that AltO adopts.","marker":"[10]"},{"why":"Introduces the deep homography estimation task, the four-point offset parametrization, and the synthetic deformation used to construct training pairs, and serves as one registration backbone.","marker":"[8]"},{"why":"Provides the iterative homography estimation network IHN-1 used as a backbone and supervised reference.","marker":"[18]"},{"why":"Provides the recurrent homography network RHWF-1 used as a transformer-based backbone and supervised reference.","marker":"[19]"},{"why":"Provides the RAFT recurrent network used as another registration backbone, with DLT used to convert optical flow into homography.","marker":"[33]"},{"why":"Supplies the ResNet-34 architecture used for the encoder and projector in the experiments.","marker":"[21]"}],"fun_headline_variants":["AltO: no labels, no problem for multimodal homography","Trivial solution? Not with AltO's alternating phases","Unsupervised homography for cross-domain pairs: AltO wins","AltO beats unsupervised baselines for multimodal homography","No labels? No problem. AltO aligns multimodal pairs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the alternating GL/MARL schedule prevents the trivial-solution collapse in which the encoder outputs a constant and the registration network predicts the identity homography; the paper supports this only with the Google Map ablation in Section 6.1 and offers no formal guarantee.","fun_headline_variants_meta":{"raw":{"variants":["AltO: no labels, no problem for multimodal homography","Trivial solution? Not with AltO's alternating phases","Unsupervised homography for cross-domain pairs: AltO wins","AltO beats unsupervised baselines for multimodal homography","No labels? No problem. AltO aligns multimodal pairs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000542,"raw_usage":{"total_tokens":2639,"prompt_tokens":1028,"completion_tokens":1611,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":644,"completion_tokens_details":{"reasoning_tokens":1528}},"tokens_in":644,"tokens_out":1611,"duration_ms":12300,"temperature":1.0,"reasoning_tokens":1528,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T16:53:32.434278+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train IHN-1 with AltO on a new multimodal pair dataset with a large modality gap using the same hyperparameters; if the mean corner error stays near the roughly 24-pixel identity-warp baseline, or the encoder's output becomes near-constant while the registration network predicts near-identity homographies, the alternating schedule failed to prevent collapse and the paper's central claim is undercut.","supporting_citations":[{"cited_title":"Barlow twins: Self- supervised learning via redundancy reduction","cited_arxiv_id":null,"evidence_quote":"Provides the Barlow Twins redundancy-reduction objective from which both the modality loss and the proposed Geometry Barlow Twins loss are derived."},{"cited_title":"MacDonald","cited_arxiv_id":null,"evidence_quote":"Supplies the Deep NIR multimodal dataset used for training and evaluation."},{"cited_title":"Chen, Shreyas S","cited_arxiv_id":null,"evidence_quote":"Is the first unsupervised deep homography baseline whose pixel-space reconstruction fails on modality gaps and is the main comparison point."},{"cited_title":"RAFT: recurrent all-pairs field transforms for optical flow","cited_arxiv_id":null,"evidence_quote":"Provides the RAFT recurrent network used as another registration backbone, with DLT used to convert optical flow into homography."}],"review_version":1}