REVIEW 3 major objections 4 minor 30 references
OPTIMUS: Observing Persistent Transformations in Multi-temporal Unlabeled Satellite-data
T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read If a model can order images in time, the scene has changed; OPTIMUS turns this into a change detector.
desk verdict A genuinely fresh self-supervised change detection idea with a strong reported gain, but the small evaluation set and an untested train/inference query-position gap make the headline AUROC provisional. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the pivot score, defined for a time series of model confidences $S=\{s_1,\dots,s_n\}$ as $P=\max_i \left|\frac{1}{i}\sum_{j=1}^i s_j - \frac{1}{n-i}\sum_{j=i+1}^n s_j\right|$. It measures the largest discrepancy between the mean confidence before and after a split point, so a high value indicates an abrupt, persistent transition. The training task that produces these confidences is a Siamese network with a ResNet-50 backbone, trained with binary cross-entropy to predict whether a query image is temporally closer to anchor set $A_1$ or $A_2$; queries are sampled outside the anchor interval so that temporal and visual closeness agree. The pivot score carries the argument because it converts the model's ordering ability into a single change score.
What would settle it
Construct a labeled set of time series with known persistent changes, run OPTIMUS, and compute the pivot score twice: once using only model outputs for images that lie between the two anchors, and once using only out-of-interval images. If the in-interval pivot score separates changed from unchanged series no better than chance while the out-of-interval score does well, then the transfer assumption on which the method rests is false.
Extended reading notes
Core claim
The paper's central claim is that temporal-order recovery is a faithful proxy for persistent change: if a model can tell whether an image lies before or after a given pair of anchor images, then the underlying scene must contain a lasting transformation. The discovery is that this principle can be operationalized end-to-end without any labels. OPTIMUS samples queries that fall outside the anchor interval during training, which guarantees that visual similarity and temporal proximity agree, and then uses the pivot score of the model's outputs at test time to flag step-like transitions. The paper reports that this approach outperforms contrastive and supervised baselines by a wide margin, and that an iterative fine-tuning variant localizes changes to 128x128 patches with AUROC 94.15%.
Load-bearing premise
The model learns to order images that lie outside the anchor interval, but at inference its confidence is scored for every image in the series; the paper assumes this ordering ability transfers to in-interval images.
Editorial extensions
If this is right
- OPTIMUS can filter vast unlabeled satellite archives down to the small fraction of locations with persistent changes, making expert annotation for rare change types affordable.
- Because the method only needs unlabeled RGB time series, it can be applied to any geography with repeated satellite coverage, including non-urban regions that existing datasets ignore.
- The iterative localization variant shows the same principle works at patch level, extending change detection from series-level classification to spatial localization.
- The method is largely independent of the choice of backbone, indicating that the ordering objective, not the encoder, is what drives performance.
- Because the training signal is temporal order rather than pixel difference, OPTIMUS is insensitive to seasonal cycles such as foliage change and crop rotation.
Reading between the lines
- Editorial inference: the order-recovery principle is not specific to satellite imagery; it should transfer to any collection of unlabeled temporal images where persistent change is rare, such as wildlife camera traps or medical follow-up scans, provided the same out-of-interval query construction holds.
- Editorial inference: the pivot score's max-split form is best suited to abrupt single events; slow monotonic changes may be better captured by the Spearman coefficient or by a combined score, which the paper's own ablation suggests but does not develop.
- Editorial inference: a direct test of the paper's unstated transfer assumption would be to train on out-of-interval queries only and evaluate on in-interval queries; if performance collapses, a modified objective that exposes the model to in-interval queries during training would be needed.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces OPTIMUS, a self-supervised method for detecting persistent, non-seasonal changes in satellite image time series. The method trains a Siamese ResNet-50 classifier to predict whether a query image is temporally closer to an early or late anchor set, using only queries outside the anchor interval during training. At test time, it applies the classifier to every image in a series, computes a pivot score from the resulting confidence sequence, and thresholds that score to label the series as changed or unchanged. On an evaluation set of 300 globally sampled time series with human labels, OPTIMUS achieves AUROC 0.876, compared with 0.563 for the best baseline (CaCo pretrained on OSCD). The paper also proposes an iterative 128x128 variant for spatial localization and reports ablations over backbone, initialization, context size, and change measure.
Significance. If the results hold, OPTIMUS is a significant contribution: it is a fully self-supervised pipeline that does not require change labels, addresses rare and non-urban change categories, and releases a one-million-series training dataset and code. The core idea--that recoverable long-term ordering implies persistent change--is intuitive and falsifiable, and the evaluation is independent of the training objective (no direct fitting of human labels). The main strengths are the scale of the training data, the zero-shot evaluation against human annotations, and the use of threshold-independent AUROC as the primary metric. The weaknesses described below concern whether the reported AUROC is robust to the training/inference distribution shift and to evaluation-set selection; these need to be addressed before the quantitative claim can be accepted.
major comments (3)
- [Section 4 (Training Example Selection; Change Score Measures)] The binary classifier is trained only on triplets (A1, A2, Q) in which Q lies before A1 or after A2, explicitly avoiding queries between the anchors. At inference, however, the change score is computed as S = {b(I1, In, Ij) | j = 1..n}, so the model is applied to every image in the series, including images strictly between the anchors and the endpoints that coincide with an anchor image. The paper provides no evidence that the model's out-of-interval ordering ability transfers to in-interval queries, and no calibration or accuracy numbers for either query type are reported. If in-interval scores are influenced by seasonal appearance rather than by long-term ordering, the pivot score P can be elevated on unchanged series or distorted on changed series, and the reported AUROC of 0.876 would not reflect the stated principle. This concern is load-bearing because P is sensitive to any systematic trend in S. The authors should (i) report the classifier's accuracy on held-out in-interval and out-of-interval triplets, (ii) compare pivot scores with anchor endpoints excluded or with the endpoints handled by a held-out protocol, and (iii) ablate the contribution of the endpoint scores to P.
- [Section 5.1 versus Appendix B] The description of how baseline change scores are computed is internally contradictory. Section 5.1 states that for CaCo, 'we randomly sampled three image pairs from the first and last years, then computed the average distance between their representations,' and that SeCo is evaluated in the same way. Appendix B, however, states that 'in our evaluation of both SeCo and CaCo, change scores are calculated by dividing the detected change by the total number of pixels in the image.' These two protocols produce different scores, and the main quantitative comparison depends on which one was actually used. The authors should specify the exact protocol for each baseline, ideally with a shared evaluation harness, so that the improvement from 0.563 to 0.876 can be audited.
- [Section 5.1 and Appendix C] The evaluation set contains only 300 time series with no reported confidence intervals or error bars, and all model-selection decisions (backbone, initialization, context size, and the choice of pivot score over Spearman rank correlation) are made by comparing AUROC on the same evaluation set. Table 1 reports a single point estimate, and the ablations in Appendix C select the best configuration on the test set rather than on a separate validation split. This test-set selection can inflate the reported performance. The authors should report bootstrap or other confidence intervals for the AUROC and F1 values, and should either use a separate validation set for the ablations or explicitly disclose and analyze the selection bias.
minor comments (4)
- [Section 4, Change Score Measures (pivot score equation)] The pivot score formula as written takes a maximum over i = 1..n, but the term (sum_{j=i+1}^n s_j)/(n-i) is undefined when i = n; the intended range should presumably be i = 1..n-1, or the empty-sum case should be defined separately.
- [Section 4, Change Score Measures] The notation b(I1, In, Ij) conflicts with the training setup, in which each anchor consists of c = 3 consecutive images. The paper should specify how the anchor sets are constructed at inference time, and how queries that coincide with images inside an anchor set are handled.
- [Figure 7 caption] The caption states that row 6 is incorrectly labeled and 'should be 0,' which is a self-admitted error in a displayed example. The figure should be corrected, and the text should clarify whether this correction was applied to the evaluation labels used in Tables 1 and 2.
- [Table 2] The row for OPTIMUS (512x512) reports an F1 score but no AUROC; if the AUROC is unavailable or undefined for that configuration, this should be stated explicitly rather than left as a dash.
Circularity Check
No significant circularity: the self-supervised training objective, the pivot-score change measure, and the human-labeled evaluation are independent of one another.
full rationale
The derivation chain is not circular. OPTIMUS is trained with self-supervised temporal-order labels constructed from timestamps (query Q placed before A1 or after A2), with no human change labels used in training. At test time, the pivot score is computed from the model's learned confidence outputs S = {b(I1, In, Ij)} and compared against independently human-annotated binary change labels; no fitted parameter is renamed as a prediction and no evaluation constant is derived from the labels. The invariance principle ('if a model can recover relative order, then there are long-lasting changes') is an empirical hypothesis that is tested rather than assumed by construction. The only self-citation is the use of Satlas pretrained weights [1], whose authors overlap with the present paper, but the ablation in Appendix C.2 shows Random and ImageNet initializations still achieve AUROC 0.851 and 0.857 versus 0.876 for Satlas, so the central result does not rest on that citation. The untested distribution shift to in-interval queries and the inclusion of anchor-coincident endpoints in S are validity and robustness concerns, not circular reductions: the change score is not equivalent to the training objective or to the ground-truth labels by definition. The evaluation is also against external human labels and baseline methods, making the main AUROC claim self-contained rather than derived from the paper's own assumptions.
Assumptions & free parameters
free parameters (4)
- Context size c =
3
- Top 50% filtering threshold for iterative localization =
50%
- Cloud cover threshold =
20%
- Learning rate, batch size, epochs =
3e-4, 5, 5
assumptions (4)
- domain assumption If a model can recover the long-term ordering of images in a time series, then the series contains long-lasting changes.
- ad hoc to paper Queries outside the anchor interval generalize to queries inside the interval.
- domain assumption Human annotations of persistent change are accurate and consistent.
- domain assumption The model's ordering signal is not dominated by nuisance variables such as clouds, shadows, or lighting changes.
Cite this review
Pith. "Pith review of OPTIMUS: Observing Persistent Transformations in Multi-temporal Unlabeled Satellite-data." pith.science (2026). https://pith.science/paper/BAC476X5
@misc{pith2026250613902,
author = {Pith},
title = {Pith review of: OPTIMUS: Observing Persistent Transformations in Multi-temporal Unlabeled Satellite-data},
year = {2026},
howpublished = {\url{https://pith.science/paper/BAC476X5}},
note = {Machine review of arXiv:2506.13902}
}
read the original abstract
In the face of pressing environmental issues in the 21st century, monitoring surface changes on Earth is more important than ever. Large-scale remote sensing, such as satellite imagery, is an important tool for this task. However, using supervised methods to detect changes is difficult because of the lack of satellite data annotated with change labels, especially for rare categories of change. Annotation proves challenging due to the sparse occurrence of changes in satellite images. Even within a vast collection of images, only a small fraction may exhibit persistent changes of interest. To address this challenge, we introduce OPTIMUS, a self-supervised learning method based on an intuitive principle: if a model can recover information about the relative order of images in the time series, then that implies that there are long-lasting changes in the images. OPTIMUS demonstrates this principle by using change point detection methods on model outputs in a time series. We demonstrate that OPTIMUS can directly detect interesting changes in satellite images, achieving an improvement in AUROC score from 56.3% to 87.6% at distinguishing changed time series from unchanged ones compared to baselines. Our code and dataset are available at https://huggingface.co/datasets/optimus-change/optimus-dataset/.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
SatlasPretrain: A Large- Scale Dataset for Remote Sensing Image Understanding,
Favyen Bastani, Piper Wolters, Ritwik Gupta, Joe Ferdi- nando, and Aniruddha Kembhavi. SatlasPretrain: A Large- Scale Dataset for Remote Sensing Image Understanding,
-
[2]
Change Detection in Optical Aerial Images by a Multilayer Conditional Mixed Markov Model
Csaba Benedek and Tam ´As Sziranyi. Change Detection in Optical Aerial Images by a Multilayer Conditional Mixed Markov Model. IEEE Transactions on Geoscience and Re- mote Sensing, 47(10):3416–3430, 2009. 4
work page 2009
-
[3]
Hao Chen and Zhenwei Shi. A spatial-temporal attention- based method and a new dataset for remote sensing image change detection. Remote Sensing, 12(10):1662, 2020. 1
work page 2020
-
[4]
Urban Change Detection for Multispectral Earth Observation Using Convolutional Neural Networks
Rodrigo Caye Daudt, Bertr Le Saux, Alexandre Boulch, and Yann Gousseau. Urban Change Detection for Multispectral Earth Observation Using Convolutional Neural Networks. In IGARSS 2018 - 2018 IEEE International Geoscience and Re- mote Sensing Symposium, pages 2115–2118, 2018. 1
work page 2018
-
[5]
Rodrigo Caye Daudt, Bertrand Le Saux, Alexandre Boulch, and Yann Gousseau. Urban Change Detection for Multi- spectral Earth Observation Using Convolutional Neural Net- works, 2018. 1, 2, 7
work page 2018
-
[6]
Imagenet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, 2009. 11
work page 2009
-
[7]
xBD: A Dataset for Assessing Building Damage from Satellite Imagery, 2019
Ritwik Gupta, Richard Hosfelt, Sandra Sajeev, Nirav Patel, Bryce Goodman, Jigar Doshi, Eric Heim, Howie Choset, and Matthew Gaston. xBD: A Dataset for Assessing Building Damage from Satellite Imagery, 2019. 1
work page 2019
-
[8]
Deep Residual Learning for Image Recognition, 2015
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition, 2015. 6
work page 2015
Show all 30 references
-
[9]
AFDE-Net: Building Change Detection using Attention- Based Feature Differential Enhancement for Satellite Im- agery
Shimaa Holail, Tamer Saleh, Xiongwu Xiao, and Deren Li. AFDE-Net: Building Change Detection using Attention- Based Feature Differential Enhancement for Satellite Im- agery. IEEE Geoscience and Remote Sensing Letters, 20:1– 5, 2023. 3
2023
-
[10]
Change detection from remotely sensed images: From pixel-based to object-based approaches
Masroor Hussain, Dongmei Chen, Angela Cheng, Hui Wei, and David Stanley. Change detection from remotely sensed images: From pixel-based to object-based approaches. ISPRS Journal of Photogrammetry and Remote Sensing , 80:91–106, 2013. 1
2013
-
[11]
ChangeNet: Multi-Temporal Asymmetric Change De- tection Dataset, 2024
Deyi Ji, Siqi Gao, Mingyuan Tao, Hongtao Lu, and Feng Zhao. ChangeNet: Multi-Temporal Asymmetric Change De- tection Dataset, 2024. 3
2024
-
[12]
Siamese Neural Networks for One-shot Image Recognition
Gregory Koch, Richard Zemel, and Ruslan Salakhutdinov. Siamese Neural Networks for One-shot Image Recognition. In ICML Deep Learning Workshop, 2015. 6
2015
-
[13]
The Outcome of the 2021 IEEE GRSS Data Fusion Con- test—Track MSD: Multitemporal Semantic Change Detec- tion
Zhuohong Li, Fangxiao Lu, Hongyan Zhang, Lilin Tu, Jiayi Li, Xin Huang, Caleb Robinson, Nikolay Malkin, Nebojsa Jojic, Pedram Ghamisi, Ronny H ¨ansch, and Naoto Yokoya. The Outcome of the 2021 IEEE GRSS Data Fusion Con- test—Track MSD: Multitemporal Semantic Change Detec- tion...
2021
-
[14]
BCE- Net: Reliable Building Footprints Change Extraction based on Historical Map and Up-to-date Images using Contrastive Learning
Cheng Liao, Han Hu, Xuekun Yuan, Haifeng Li, Chao Liu, Chunyang Liu, Gui Fu, Yulin Ding, and Qing Zhu. BCE- Net: Reliable Building Footprints Change Extraction based on Historical Map and Up-to-date Images using Contrastive Learning. ISPRS Journal of Photogrammetry and Remote ...
2023
-
[15]
A CNN-transformer network with multiscale context aggrega- tion for fine-grained cropland change detection
Mengxi Liu, Zhuoqun Chai, Haojun Deng, and Rong Liu. A CNN-transformer network with multiscale context aggrega- tion for fine-grained cropland change detection. IEEE Jour- nal of Selected Topics in Applied Earth Observations and Remote Sensing, 15:4297–4306, 2022. 2
2022
-
[16]
Swin transformer v2: Scaling up capacity and resolution, 2022
Ze Liu, Han Hu, Yutong Lin, Zhuliang Yao, Zhenda Xie, Yixuan Wei, Jia Ning, Yue Cao, Zheng Zhang, Li Dong, Furu Wei, and Baining Guo. Swin transformer v2: Scaling up capacity and resolution, 2022. 11
2022
-
[17]
Decoupled Weight Decay Regularization, 2019
Ilya Loshchilov and Frank Hutter. Decoupled Weight Decay Regularization, 2019. 6
2019
-
[18]
A PolSAR Change Detection In- dex Based on Neighborhood Information for Flood Mapping
Sahel Mahdavi, Bahram Salehi, Weimin Huang, Meisam Amani, and Brian Brisco. A PolSAR Change Detection In- dex Based on Neighborhood Information for Flood Mapping. Remote Sensing, 11(16), 2019. 1
2019
-
[19]
Change Event Dataset for Discovery from Spatio-temporal Remote Sensing Imagery
Utkarsh Mall, Bharath Hariharan, and Kavita Bala. Change Event Dataset for Discovery from Spatio-temporal Remote Sensing Imagery. In NeurIPS Datasets and Benchmarks Track, 2022. 4
2022
-
[20]
Change- Aware Sampling and Contrastive Learning for Satellite Im- ages
Utkarsh Mall, Bharath Hariharan, and Kavita Bala. Change- Aware Sampling and Contrastive Learning for Satellite Im- ages. In CVPR, 2023. 1, 4, 7
2023
-
[21]
Seasonal contrast: Unsuper- vised pre-training from uncurated remote sensing data, 2021
Oscar Ma ˜nas, Alexandre Lacoste, Xavier Giro i Nieto, David Vazquez, and Pau Rodriguez. Seasonal contrast: Unsuper- vised pre-training from uncurated remote sensing data, 2021. 4, 7
2021
-
[22]
S2Looking: A Satellite Side-Looking Dataset for Building Change Detec- tion
Li Shen, Yao Lu, Hao Chen, Hao Wei, Donghai Xie, Jiabao Yue, Rui Chen, Shouye Lv, and Bitao Jiang. S2Looking: A Satellite Side-Looking Dataset for Building Change Detec- tion. Remote Sensing, 13(24):5094, 2021. 3
2021
-
[23]
A deeply supervised attention metric-based network and an open aerial image dataset for remote sensing change detection
Qian Shi, Mengxi Liu, Shengchen Li, Xiaoping Liu, Fei Wang, and Liangpei Zhang. A deeply supervised attention metric-based network and an open aerial image dataset for remote sensing change detection. IEEE transactions on geo- science and remote sensing, 60:1–16, 2021. 1, 2
2021
-
[24]
Hi- UCD: A Large-scale Dataset for Urban Semantic Change Detection in Remote Sensing Imagery
Shiqi Tian, Ailong Ma, Zhuo Zheng, and Yanfei Zhong. Hi- UCD: A Large-scale Dataset for Urban Semantic Change Detection in Remote Sensing Imagery. arXiv preprint arXiv:2011.03247, 2020. 3
2011 arXiv
-
[25]
DynamicEarthNet: Daily Multi-Spectral Satellite Dataset for Semantic Change Segmentation, 2022
Aysim Toker, Lukas Kondmann, Mark Weber, Marvin Eisenberger, Andr´es Camero, Jingliang Hu, Ariadna Pregel Hoderlein, C ¸ a˘glar S ¸enaras, Timothy Davis, Daniel Cremers, Giovanni Marchisio, Xiao Xiang Zhu, and Laura Leal-Taix´e. DynamicEarthNet: Daily Multi-Spectral Satellite ...
2022
-
[26]
SSL4EO- S12: A Large-Scale Multi-Modal, Multi-Temporal Dataset for Self-Supervised Learning in Earth Observation, 2023
Yi Wang, Nassim Ait Ali Braham, Zhitong Xiong, Chenying Liu, Conrad M Albrecht, and Xiao Xiang Zhu. SSL4EO- S12: A Large-Scale Multi-Modal, Multi-Temporal Dataset for Self-Supervised Learning in Earth Observation, 2023. 4
2023
-
[27]
Chen Wu, Bo Du, and L. Zhang. Fully Convolutional Change Detection Framework With Generative Adversarial Network for Unsupervised, Weakly Supervised and Regional Super- vised Change Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45:9774–9788, 2022. 3
2022
-
[28]
Asymmetric Siamese Networks for Semantic Change Detection in Aerial Images
Kunping Yang, Gui-Song Xia, Zicheng Liu, Bo Du, Wen Yang, Marcello Pelillo, and Liangpei Zhang. Asymmetric Siamese Networks for Semantic Change Detection in Aerial Images. IEEE Transactions on Geoscience and Remote Sens- ing, 60:1–18, 2022. 4
2022
-
[29]
A deeply supervised image fusion network for change de- tection in high resolution bi-temporal remote sensing im- ages
Chenxiao Zhang, Peng Yue, Deodato Tapete, Liangcun Jiang, Boyi Shangguan, Li Huang, and Guangchao Liu. A deeply supervised image fusion network for change de- tection in high resolution bi-temporal remote sensing im- ages. ISPRS Journal of Photogrammetry and Remote Sens- ing, ...
2020
-
[30]
Change is Everywhere: Single-Temporal Supervised Object Change Detection in Remote Sensing Imagery, 2023
Zhuo Zheng, Ailong Ma, Liangpei Zhang, and Yanfei Zhong. Change is Everywhere: Single-Temporal Supervised Object Change Detection in Remote Sensing Imagery, 2023. 3, 4 A. Annotation Guidelines For the manual annotation process, we provided annota- tors with a detailed set of g...
2023
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.