{"id":"4bc402d1-7760-421e-9a16-f7db5b87b728","arxiv_id":"2502.01035","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"CropTTA estimates data uncertainty in deep homography estimation from the standard deviation of displacement predictions across random thermal-image crops, improving failure detection in satellite-thermal UAV geo-localization.","lead":"This paper introduces CropTTA, a test-time method that crops thermal images into multiple views and measures how consistently a deep homography network's displacement predictions agree across those views, using the spread as a confidence signal for UAV satellite-thermal geo-localization.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table I and Fig. 4 show CropTTA's uncertainty is poorly calibrated at DC=128 (AUC 0.74, rejected samples do not reduce error), so the headline DC=512 success does not establish a general uncertainty estimate; the rejection threshold sc is also unreported.","rationale":"The reader's weakest assumption was that the std across crops (Eq. 6) may not correlate with alignment error for confidently-wrong networks. My concern identifies concrete empirical evidence of this failure in the paper's own Table I and Fig. 4 at DC=128, where the uncertainty ranking is poor and rejection worsens accuracy. This is closely related but more specific: it shows the assumed correlation breaks down in a particular search-radius regime, not merely as a hypothetical. The unreported threshold sc compounds the problem by making the headline success rate unverifiable. These points reinforce the CONDITIONAL verdict: the method appears promising at DC=512, but the general claim of effective data uncertainty measurement is not established, and the evaluation lacks key operating-point details. I did not find evidence of internal inconsistency in the mathematical derivations of Section III; the derivations are sound. The concern is about the empirical validity of the uncertainty proxy and the reproducibility of the reported operating points.","tokens_in":10817,"tokens_out":11131,"duration_ms":114456,"concrete_test":"Using the released code and Boson dataset, compute the Spearman rank correlation between U_TTA (Eq. 6) and MACE/CE separately for each DC in {128, 256, 512}. Then, with a single fixed threshold sc (reported in the paper), compute the mean error of rejected vs accepted samples at each DC. If at DC=128 the rank correlation is not significantly positive and the rejected samples have lower mean error than accepted ones, CropTTA is anti-calibrated in that regime. Report sc explicitly to make the operating point reproducible.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that CropTTA provides effective data uncertainty measurement for TG, demonstrated by a 7.25 m mean center error at 97.5% success rate for DC=512 in Table I. However, the same table shows that at DC=128, CropTTA with STHN increases MACE from 7.51 m (no UE) to 8.26 m and CE from 6.66 m to 7.75 m while rejecting only 1.5% of samples. For the mean of the accepted samples to rise from 7.51 to 8.26, the rejected samples must have had substantially below-average error, implying that the uncertainty score is inversely related to localization error in this regime. This is corroborated by Fig. 4, where the ROC AUC at DC=128 is only 0.74 versus 0.96 at DC=512. While the paper acknowledges that baselines outperform at DC=128 and calls this an 'inherent lower bound', it does not explain why rejecting the highest-uncertainty samples would worsen accuracy, which directly contradicts the assumption in Eq. (6) that crop-consistency std tracks alignment error. A confounding factor is that the CropTTA model is fine-tuned with the crop loss, so the baseline and the UE method use different weights; the paper never reports the fine-tuned model's no-rejection performance, preventing isolation of the rejection benefit. Additionally, the rejection threshold sc is never stated, so the reported success rates are operating points on an unverifiable curve. These issues mean the headline result is not a robust demonstration of a general data-uncertainty estimator.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes UASTHN, an uncertainty estimation framework for deep homography estimation (DHE) in UAV satellite-thermal geo-localization. The main component, CropTTA, measures data uncertainty as the standard deviation of recovered four-corner displacements across random crops of the thermal image at test time (Eq. (6)), and optionally combines this with deep ensembles (DE) for model uncertainty. A rejection threshold sc is used to discard high-uncertainty predictions, and success rates (SR) are reported as the proportion of non-rejected samples. Experiments on the Boson-nighttime dataset compare CropTTA against DE and direct modeling (DM) across three DHE backbones (DHN, IHN, STHN) at three search radii (DC = 128, 256, 512 m). The headline result is a mean center error of 7.25 m with a 97.5% success rate for STHN with CropTTA at DC = 512 m. The crop-recovery mathematics (Eqs. (1)-(5)) is sound, and the empirical comparison is extensive across backbones and uncertainty baselines. However, the rejection threshold sc is never stated, the DC = 128 m results appear to contradict the central assumption that crop-consistency standard deviation tracks alignment error, and the fine-tuning protocol confounds the effect of rejection with the effect of training with the crop loss.","tokens_in":11161,"tokens_out":4378,"duration_ms":40239,"significance":"If the claims hold, CropTTA would be a simple, backbone-agnostic, and efficient way to provide data-uncertainty estimates for thermal geo-localization, which is safety-relevant for UAV navigation in GPS-denied or low-light conditions. The idea of using crop-consistency as an uncertainty signal is a reasonable and novel contribution for this task, and the paper ships public code/models, includes extensive ablations, and evaluates with ROC curves, which is a standard and non-circular protocol. However, the current evidence is not fully convincing: the success-rate numbers are operating points on an unreported threshold curve, the DC = 128 m regime shows a direct contradiction with the assumed variance-error relationship, and the comparison to baselines is confounded by fine-tuning. These issues are load-bearing for the central claim that CropTTA provides effective and general data-uncertainty measurement, and they require additional analysis and reporting.","major_comments":[{"comment":"The rejection threshold sc is defined in Section III-B but its value is never reported anywhere in the paper, including in the experiments or the supplementary material. Since every Success Rate in Table I and the headline \"97.5% success rate\" are defined as the proportion of samples not rejected by this threshold, the reported SR values are unconstrained operating points. Without sc, the reader cannot verify the rejection behavior, reproduce the results, or compare the operating points across methods. Please report the exact sc used for each DC and backbone, or alternatively present threshold-free metrics (e.g., ROC AUC with a justified failure definition, or a sweep over sc) as the primary evaluation.","section":"Section III-B and Section V-B, Table I"},{"comment":"At DC = 128 m, CropTTA with STHN increases MACE from 7.51 m (no UE) to 8.26 m and CE from 6.66 m to 7.75 m while rejecting only 1.5% of samples (SR 98.5%). For the mean of the accepted samples to increase, the rejected samples must have had substantially below-average error, implying that the uncertainty score is inversely related to alignment error in this regime. This directly contradicts the assumption in Eq. (6) that crop-consistency standard deviation tracks alignment error. The paper's statement that \"baselines without UE outperform, indicating inherent lower bounds\" (Section V-B) does not explain this contradiction. Please provide an analysis of the DC = 128 m case, including the error distribution of rejected samples and a discussion of why rejection degrades accuracy, since this bears directly on the generality of CropTTA as a data-uncertainty estimator.","section":"Section V-B, Table I, Fig. 4"},{"comment":"The CropTTA models are first trained for 100k steps without CropTTA and then fine-tuned for an additional 200k steps with the crop loss LCropTTA (Eq. (8)), while the \"no UE\" baselines in Table I are presumably the unfine-tuned models. The comparison therefore conflates the effect of the rejection mechanism with the effect of fine-tuning with the crop loss. The paper never reports the fine-tuned model's no-rejection performance (i.e., using only the original displacement with no threshold applied). Please report MACE and CE for the fine-tuned model with no rejection, so that the incremental benefit of the rejection step can be isolated. Without this, the reported improvement at DC = 512 m cannot be attributed to the uncertainty estimation rather than to the fine-tuning protocol.","section":"Section IV (Implementation Details) and Table I"}],"minor_comments":[{"comment":"The abstract states \"geo-localization error of 7 m with a 97% success rate\" while Table I reports CE = 7.25 m and SR = 97.5% for the same configuration; please make the numbers consistent or clarify the rounding.","section":"Abstract and Section V-B"},{"comment":"The ROC analysis defines \"expected rejected predictions\" as predictions exceeding 25 m MACE, but this threshold is not justified. Please motivate this choice or show that the conclusions are robust to the threshold value.","section":"Fig. 4 caption"},{"comment":"Equation (9) defines U^DE_RS->RT, but the following sentence says \"where U TTA ∈ R^{2×4} denotes the model uncertainty\"; this should read U^DE.","section":"Section III-C, Eq. (9)"},{"comment":"The merge function f(·) is only described in Fig. 3f as choosing min, max, or add, and the default is stated to be max in the ablation text. Please state the default merge function explicitly in the main text near Eq. (11).","section":"Section III-C, Eq. (11) and Fig. 3f"},{"comment":"The number of ensemble models N_m used for the DE experiments is not stated, although it affects the DE uncertainty estimates and inference times; please report it in the implementation details.","section":"Section V-B, Table I"},{"comment":"The claim that this is the first solution to address uncertainty estimation for localization using cross-domain data is strong; please provide a more precise comparison with prior uncertainty estimation methods in cross-domain localization to support the novelty claim.","section":"Contributions, Section II"}],"recommendation":"major_revision","confidential_remarks":"The paper has a solid core idea and extensive experiments, but the missing rejection threshold is a reproducibility problem, and the DC = 128 m contradiction plus the fine-tuning confound are load-bearing for the main claim. These are fixable with additional analysis and reporting, so I recommend major revision rather than rejection. The authors should also ensure that the reported success rates are accompanied by the exact thresholds and by a no-rejection baseline for the fine-tuned model."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear [Colleague],\n\nYou should know this paper has a genuinely new idea and a real hole in its evidence. The new idea is CropTTA: crop the thermal image, recover full-image displacements with Eqs. (1)-(5), and use the standard deviation across crops as data uncertainty. The recovery math is sound, it slots into any homography network, and the code and models are public. The experiments are extensive: three DHE backbones (DHN, IHN, STHN), three uncertainty baselines (DE, DM, CropTTA), ROC curves, failure cases, and inference-time numbers. That is real work.\n\nThe hole is that the headline success rates are operating points on an unreported threshold. The rejection threshold s_c appears in Eq. (6) and the text but is never given a value anywhere. You cannot reproduce the 97.5% SR at DC=512 without it. That alone would force a conditional verdict.\n\nWorse, the DC=128 column undercuts the central assumption. With STHN, CropTTA rejects 1.5% of samples as high-uncertainty and the mean accepted MACE goes from 7.51 m to 8.26 m, CE from 6.66 m to 7.75 m. The rejected samples were better than average. The paper calls this an 'inherent lower bound' but never explains how that squares with Eq. (6), where std of recovered displacements is supposed to track alignment error. At DC=128 the uncertainty score is inversely related to error. That is not a minor edge case; it is the regime where high-frequency localization matters.\n\nThere is also a confound: CropTTA is fine-tuned with the crop loss, so the UE model and the baseline have different weights. The paper never reports the fine-tuned model's no-rejection performance, so the DC=512 improvement (from 12.70 m to 7.93 m MACE) could come partly from the crop-loss fine-tuning rather than from rejection. Single dataset, no error bars, and the overbroad 'first solution' claim in the contributions round it out.\n\nWho is this for? Someone building safety-critical TG or using test-time augmentations for regression uncertainty will want to know about CropTTA. It deserves a serious referee, but the referee should require three things: report the actual s_c value (or a curve over thresholds), add the fine-tuned no-rejection baseline, and either explain the DC=128 inversion or qualify the method as useful mainly for coarse localization.\n\nRecommendation: send it to peer review with a request for major revision.\n\nRegards.","headline":"A real crop-consensus uncertainty mechanism with sound recovery math, but unreported rejection thresholds, a confounded fine-tuning setup, and DC=128 results that invert the core assumption.","tokens_in":11682,"tokens_out":3466,"would_cite":true,"duration_ms":33672,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A UAV can tell when its thermal-to-satellite alignment is untrustworthy by checking whether random crops of the thermal image produce agreeing homography predictions.","keywords":["uncertainty estimation","deep homography estimation","thermal geo-localization","test-time augmentation","deep ensembles","UAV navigation","satellite-thermal alignment","failure detection"],"falsifier":"Take a thermal image whose true alignment error is known to be large and whose appearance is textureless or self-similar, such as a dark desert scene with repeated dunes, and run CropTTA. If it assigns a low uncertainty score while the center error exceeds the acceptance threshold, the consensus proxy has failed. A systematic search over the Boson-nighttime test set for low-scored, high-error samples would settle whether the claim generalizes; the paper's ROC analysis at DC=512 reports AUC=0.96, so the residual false-acceptance cases are the ones to inspect.","tokens_in":10592,"feed_emoji":"🛰️","tokens_out":5379,"duration_ms":50355,"temperature":0.7,"pith_summary":"This paper argues that a UAV can tell when its thermal-to-satellite image alignment is unreliable by making random crops of the thermal image at test time and checking whether the recovered alignments agree. If the four-corner displacements predicted from different crops disagree strongly, the system rejects the match instead of trusting it. The authors claim this crop-consensus uncertainty signal, combined with deep ensembles for model uncertainty, is the first uncertainty-estimation method for cross-domain localization. They report that with the STHN two-stage homography network it cuts mean center error to 7.25 m with a 97.5% success rate within a 512 m search radius, and that it detects failures caused by textureless thermal images, corrupted images, geometric noise, self-similar maps, outdated maps, and images that extend beyond the satellite map.","feed_headline":"Crop consensus flags bad satellite-thermal alignments at 97% success","feed_subtitle":"Spreading crops across a thermal image reveals when the alignment network disagrees, trimming mean center error to 7.25 m.","key_machinery":"The central object is the crop-consensus uncertainty score $U^{\\mathrm{TTA}}_{RS\\to RT}$: after generating $N_C-1$ cropped views of the thermal image, each with crop offset $o_c$, the homography network predicts displacements whose standard deviation across views, taken per corner, is the data-uncertainty measure. The recovery step matters because cropped views predict displacements relative to the crop; equations (1)-(5) compose the homographies via DLT to map each prediction back to the original thermal frame. The mechanism's premise is that all cropped views share the same underlying homography, so genuine agreement indicates confidence, while disagreement flags textureless, corrupted, self-similar, or out-of-range inputs. Deep ensembles supply a second channel, model uncertainty, merged with the crop score by max, min, or sum.","core_discovery":"The central discovery is that data uncertainty in deep homography estimation can be measured without retraining or auxiliary networks: crop the thermal image into several views with a known offset, run the existing homography network on each, transform each predicted displacement back to the original frame using a DLT-based composition of homographies, and take the standard deviation of the recovered displacements as the uncertainty score. A high score triggers rejection of the alignment. The paper shows this CropTTA mechanism outperforms or matches a direct modeling baseline and deep ensembles across three homography networks (DHN, IHN, and STHN), and that combining it with deep ensembles gives a fuller data-plus-model uncertainty picture. The reported headline result is a mean center error of 7.25 m and a success rate of 97.5% for STHN two-stage with CropTTA at a 512 m search radius, with the strongest gains at large search radii where alignment failures have a long-tailed error distribution.","pith_inferences":["The same crop-consensus principle should transfer to other cross-modal registration tasks, such as RGB-thermal or radar-satellite alignment, because it never depends on the modality-specific appearance; a testable extension would be to run it on a daytime or seasonal variant of the same data.","The rejection threshold $s_c$ is a free parameter; a natural extension is to set it adaptively from a validation set to meet a target success rate, rather than fixing it, as the paper's own future-work section hints.","The uncertainty score could feed a sensor-fusion layer that downweights or re-plans when thermal geo-localization disagrees with other navigation cues, rather than only discarding the sample; this is an implicit consequence of treating the score as a reliability measure."],"forward_implications":["A real-time UAV navigation system can run CropTTA at inference and reject low-confidence alignments instead of feeding drifted poses to the controller; with early stopping, STHN two-stage runs in about 78 ms on an RTX 2080 Ti.","At large search radii (DC=512 m), where failure errors are long-tailed, CropTTA gives the biggest accuracy and success-rate gains, suggesting uncertainty screening matters most for low-frequency localization.","Because CropTTA only wraps the existing homography network with crops, it can be added to any deep homography estimator without retraining an uncertainty module, though fine-tuning with a crop-augmented loss improves it.","Combining CropTTA with deep ensembles yields a comprehensive uncertainty estimate that captures both hard-to-align data and model disagreement, with max as the chosen merge function."],"supporting_citations":[{"why":"Supplies the STHN two-stage homography network that, with CropTTA, produces the headline 7.25 m / 97.5% result.","marker":"[14]"},{"why":"Provides the Boson-nighttime dataset and the thermal image synthesis method (TGM) used for training and evaluation.","marker":"[13]"},{"why":"DHN baseline that introduced four-corner displacement deep homography estimation.","marker":"[16]"},{"why":"IHN iterative homography baseline used in the comparisons.","marker":"[17]"},{"why":"Direct Modeling baseline for data uncertainty that CropTTA is compared against.","marker":"[37]"},{"why":"Deep Ensembles formulation adopted for model uncertainty estimation.","marker":"[30]"},{"why":"DLT transform used to recover full-frame displacements from cropped predictions.","marker":"[36]"}],"fun_headline_variants":["Crop splits reveal when thermal-satellite alignment fails","Uncertainty from crop consensus sharpens UAV geo-localization","Test-time crops expose alignment doubts for safer UAV navigation","No retraining needed: crop views gauge homography confidence","CropTTA: data uncertainty from homography disagreement"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that the spread of predicted corner displacements across random crops tracks the true alignment error; if the network is consistently and confidently wrong on every crop, the uncertainty score stays low and the failure is not flagged, and this proxy was validated only on the Boson-nighttime dataset.","fun_headline_variants_meta":{"raw":{"variants":["Crop splits reveal when thermal-satellite alignment fails","Uncertainty from crop consensus sharpens UAV geo-localization","Test-time crops expose alignment doubts for safer UAV navigation","No retraining needed: crop views gauge homography confidence","CropTTA: data uncertainty from homography disagreement"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00063,"raw_usage":{"total_tokens":2949,"prompt_tokens":1020,"completion_tokens":1929,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":636,"completion_tokens_details":{"reasoning_tokens":1848}},"tokens_in":636,"tokens_out":1929,"duration_ms":11971,"temperature":1.0,"reasoning_tokens":1848,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T16:49:32.665417+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a thermal image whose true alignment error is known to be large and whose appearance is textureless or self-similar, such as a dark desert scene with repeated dunes, and run CropTTA. If it assigns a low uncertainty score while the center error exceeds the acceptance threshold, the consensus proxy has failed. A systematic search over the Boson-nighttime test set for low-scored, high-error samples would settle whether the claim generalizes; the paper's ROC analysis at DC=512 reports AUC=0.96, so the residual false-acceptance cases are the ones to inspect.","supporting_citations":[{"cited_title":"Sthn: Deep homogra- phy estimation for uav thermal geo-localization with satellite imagery,","cited_arxiv_id":null,"evidence_quote":"Supplies the STHN two-stage homography network that, with CropTTA, produces the headline 7.25 m / 97.5% result."},{"cited_title":"Long-range uav thermal geo-localization with satellite imagery,","cited_arxiv_id":null,"evidence_quote":"Provides the Boson-nighttime dataset and the thermal image synthesis method (TGM) used for training and evaluation."},{"cited_title":"Iterative deep homog- raphy estimation,","cited_arxiv_id":null,"evidence_quote":"IHN iterative homography baseline used in the comparisons."},{"cited_title":"Simple and scalable predictive uncertainty estimation using deep ensembles,","cited_arxiv_id":null,"evidence_quote":"Deep Ensembles formulation adopted for model uncertainty estimation."},{"cited_title":"Direct linear transformation from comparator coordinates into object space coordinates in close- range photogrammetry,","cited_arxiv_id":null,"evidence_quote":"DLT transform used to recover full-frame displacements from cropped predictions."}],"review_version":1}