REVIEW 4 major objections 5 minor 103 references
TiMo: Spatiotemporal Foundation Model for Satellite Image Time Series
T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read TiMo, a hierarchical vision transformer pretrained on the new MillionST dataset, outperforms prior spatiotemporal foundation models on deforestation, land cover, crop type, and flood tasks.
desk verdict MillionST is a real asset and D-STGA is a genuinely distinctive attention variant, but pretraining uses plain MHSA and the fine-tuning-only gains over strong baselines are small, unbarred, and thinly ablated. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is spatiotemporal gyroscope attention (STGA), together with its fast variant D-STGA. In the $T\times H\times W$ spatiotemporal token cube, STGA gives each query a key/value set shaped like a gyroscope or cross: every token at the same time as the query plus every token at the same spatial location. D-STGA replaces the spatial half of this set with an approximation that carries the paper's efficiency argument: the temporal-invariant similarity $I = M_Q M_K^{\top}$ is computed once from the median-over-time query and key features, and each timestamp's spatial map is $S_{:,t}=D'_t+I$, where $D'_t$ is a projected version of the temporal difference $Q_t - M_Q$. This turns the dominant attention cost into $O(T)$ rather than $O(T^2)$, and it is what lets the first two stages of TiMo model multiscale spatiotemporal structure cheaply. The same machinery is reused during fine-tuning by swapping the pretrained multi-head self-attention weights into the gyroscope blocks, so pretraining and downstream use share one parameter set.
What would settle it
Compare, on real SITS inputs, the exact spatiotemporal attention map of STGA or multi-head self-attention with the D-STGA approximation $S_{:,t}=D'_t+I$ from equations (4)-(5). If the approximate maps disagree materially—for example, if the locations attended to most are different—while TiMo still performs well downstream, the reported gains are not explained by faithfully modeling spatial similarity; if they agree closely, the median-plus-correction assumption is doing its stated work.
Extended reading notes
Core claim
The paper's central claim is that its hierarchical vision transformer, TiMo, learns spatiotemporal representations that transfer better than prior SITS foundation models, and that the transfer is driven by attention that explicitly separates space and time. In STGA, each query attends only to tokens at the same timestamp plus tokens at the same spatial location, so the model sees a cross-shaped neighborhood instead of all token pairs; D-STGA then approximates the spatial half of that cross by computing similarity between temporal median features, $I = M_Q M_K^{\top}$, and adding a per-timestamp correction $D'_t$ built from the query differences $Q_t - M_Q$. The paper shows results across four tasks: deforestation (MultiEarth), land cover (MultiSenGE), crop type classification (MTLCC), and flood mapping (Sen12Flood, KuroSiwo), with TiMo variants ahead of the compared spatiotemporal foundation models on every benchmark and with larger variants consistently better. It also contributes MillionST, a million-image pretraining dataset with ten temporal phases over five years, and reports that random temporal sampling during pretraining beats fixed sampling. The authors' conclusion is that TiMo is a general, scalable, and label-efficient spatiotemporal foundation model for SITS analysis.
Load-bearing premise
The load-bearing premise is that at any time step, the spatial similarity between any two locations is fully captured by the similarity of their median-over-time features plus a single scalar adjustment for that time step, so the per-timestamp temporal differences add no location-specific spatial structure.
Editorial extensions
If this is right
- A single TiMo backbone pretrained once on MillionST transfers to at least four SITS tasks, so task-specific architectures may not be needed for deforestation, land cover, crop mapping, or flood monitoring.
- Because D-STGA reduces attention cost from $O(T^2)$ to $O(T)$, longer or denser temporal series become practical at the same compute budget.
- TiMo-Large trained on 50% of KuroSiwo labels beats a strong spatiotemporal model trained on the full set, so the model lowers the labeling burden for flood mapping and similar disaster-response tasks.
- TiMo-Huge improves over TiMo-Base on both tested benchmarks, so scaling the same architecture continues to buy accuracy.
Reading between the lines
- Editorial inference: if the median-plus-correction decomposition is as general as D-STGA assumes, it could be applied to other aligned three-dimensional token domains, such as weather reanalysis or sparse video, to cut attention cost the same way.
- Editorial inference: the paper does not compare D-STGA to exact STGA under identical pretraining, so whether the approximation is lossless or acts as an implicit regularizer remains untested; an ablation that holds everything else fixed would separate the two.
- Editorial inference: because MillionST is drawn mainly from Europe, North Africa, and West Asia, the claimed generalization is bounded by that geography; sampling the same way on other continents would indicate whether the learned spatiotemporal structure is truly global.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This manuscript proposes TiMo, a hierarchical vision transformer for satellite image time series (SITS), with a spatiotemporal gyroscope attention (STGA) mechanism and a differential variant (D-STGA) intended to capture multiscale spatiotemporal relationships efficiently. The authors curate MillionST, a pretraining dataset of one million Sentinel-2 patches from 100,000 locations with 10 timestamps, and pretrain TiMo with masked image modeling. They evaluate on deforestation (MultiEarth), land cover (MultiSenGE), crop classification (MTLCC), and flood detection (Sen12Flood, KuroSiwo), reporting improvements over SOTA SITS foundation models, plus data efficiency and scaling studies. The paper also includes ablations on attention configuration and temporal sampling, and a datasheet for MillionST.
Significance. If the reported results are reproducible, TiMo would be a useful SITS backbone, and MillionST is a potentially valuable resource for the community. The paper covers a broad set of downstream tasks and includes data efficiency and scaling analyses. However, the central novelty—the gyroscope attention mechanism—is not isolated in the experiments, and the pretraining protocol undermines the claim that the pretrained model encodes spatiotemporal patterns through that mechanism. The dataset contribution and the empirical gains across tasks are real if confirmed, but the evidence for the proposed attention mechanism as the cause is currently weak.
major comments (4)
- [Sec. 3.4] The pretraining procedure uses MHSA in all blocks, and STGA/D-STGA is introduced only during fine-tuning, with the D-STGA linear projection randomly initialized. Consequently, the model pretrained on MillionST is a plain hierarchical ViT, not TiMo as described in Secs. 3.2–3.3, and the pretrained weights contain no information about the gyroscope attention patterns. The claims in the abstract and Section 5 that TiMo 'learns and encodes generalizable spatiotemporal representations' via the proposed mechanism are therefore not supported. To support the claim, the authors should either pretrain with the actual STGA/D-STGA architecture or provide a controlled experiment that isolates the attention mechanism from the pretraining data and the hierarchical backbone.
- [Table 7 / Sec. 4.6] The ablation does not include a feasible MHSA baseline (M-M-M-M runs out of memory), and the only direct comparison involving the proposed mechanism is S-M-M-M (0.7827) versus D-M-M-M (0.7836), a difference of 0.0009 mIoU that is within likely noise. There is no configuration that uses MHSA in the first two stages and the proposed attention in later stages, or vice versa, so the reported gains of D-D-M-M over competing models cannot be attributed to the attention mechanism rather than to the hierarchical architecture, parameter count, or MillionST pretraining. Additional ablations with an MHSA-only configuration trained at reduced resolution or with gradient checkpointing, or a from-scratch comparison, are needed.
- [Sec. 3.3, Eqs. (4)-(5)] The D-STGA approximation assumes that spatial similarity at each timestamp can be expressed as the temporal-invariant similarity I = M_Q M_K^T plus a rank-1 projection D'_t of the per-timestamp difference. This assumption is not validated anywhere in the paper. If the approximation is inaccurate, the attention maps in the first two stages are systematically biased, yet no experiment measures the approximation error, e.g., by comparing D-STGA against full STGA in a configuration where both fit in memory (which the S-M-M-M row of Table 7 shows is possible), or by analyzing the learned D'_t maps.
- [Tables 2-6] All benchmark results are reported without error bars, standard deviations, or significance tests. Several decisive comparisons are very small (e.g., Table 2 TiMo-Base 0.7860 vs SeCo 0.7846; Table 3 TiMo-Base 0.2977 vs SatMAE 0.2942), so it is unclear whether the claimed improvements are statistically meaningful. The paper should report multiple seeds or at least provide variance estimates, particularly for the ablations in Table 7.
minor comments (5)
- [Sec. 4.8 / Figure 7] The text states that 'TiMo-Huge demonstrates a 9.14% higher mIoU than TiMo-Base on MultiEarth dataset,' but Figure 7 shows only MultiSenGE and KuroSiwo; the MultiEarth scaling result is not displayed. Please add the corresponding figure or correct the reference.
- [Table 1] The TiMo-Huge dimension '20488' appears to be a typo for '2048'.
- [Supplementary Material D.2/D.4] The datasheet states 'A1: No' for preprocessing, but the collection section describes cloud removal, filtering, and temporal/spatial sampling. Please clarify whether preprocessing was applied.
- [Supplementary Material D.2, A10] The datasheet's claim that the dataset is 'self-contained' is contradicted by the statement that samples 'can be downloaded from official websites'; please clarify the distribution mechanism.
- [Table 5] TiMo-Large (86.78) slightly underperforms TiMo-Base (86.89) on Sen12Flood; the explanation of overfitting would be more convincing with training/validation curves or regularization details.
Circularity Check
No circularity found: TiMo is evaluated against external public benchmarks and its reported gains are empirical, not defined in terms of its own inputs.
full rationale
TiMo's central claims are supported by comparisons on public datasets (MultiEarth, MultiSenGE, MTLCC, Sen12Flood, KuroSiwo), and MillionST is used only for self-supervised pretraining. The reported mIoU and accuracy numbers are external measurements, not consequences of a fitted parameter being renamed as a prediction. The D-STGA approximation in Sec. 3.3 is an architectural modeling assumption; if it does not capture true spatiotemporal relationships, that is a correctness or validation concern, not circularity. Self-citations ([5], [19], [35], [47]) appear in related-work or implementation-reuse contexts and are not used to define the benchmark results. The observation that pretraining uses MHSA while STGA/D-STGA is introduced during fine-tuning is a legitimate attribution concern about the ablation and the source of the reported gains, but it does not make any derivation circular: no quantity in the paper equals its own input by construction, and no prediction is statistically forced by a fit to the same data. Overall score 0.
Assumptions & free parameters
assumptions (5)
- domain assumption Multi-temporal observations contain both variant and invariant regions, so the spatial status at any time can be inferred from temporal differences.
- domain assumption Randomly sampling 3 of 10 timestamps during pretraining preserves enough temporal structure to transfer to downstream tasks with up to 30 timestamps.
- domain assumption The Hiera masked-image-modeling objective transfers to satellite image time series when the mask unit is set to 1x32x32 pixels.
- domain assumption MillionST, sampled near populous cities in Europe, North Africa, and West Asia, provides a useful pretraining distribution for global downstream tasks.
- standard math Softmax attention, matrix multiplication, and the architecture details of Swin Transformer blocks are used as described in Eqs. 1-5.
Cite this review
Pith. "Pith review of TiMo: Spatiotemporal Foundation Model for Satellite Image Time Series." pith.science (2026). https://pith.science/paper/UIXQXLXZ
@misc{pith2026250508723,
author = {Pith},
title = {Pith review of: TiMo: Spatiotemporal Foundation Model for Satellite Image Time Series},
year = {2026},
howpublished = {\url{https://pith.science/paper/UIXQXLXZ}},
note = {Machine review of arXiv:2505.08723}
}
read the original abstract
Satellite image time series (SITS) provide continuous observations of the Earth's surface, making them essential for applications such as environmental management and disaster assessment. However, existing spatiotemporal foundation models rely on plain vision transformers, which encode entire temporal sequences without explicitly capturing multiscale spatiotemporal relationships between land objects. This limitation hinders their effectiveness in downstream tasks. To overcome this challenge, we propose TiMo, a novel hierarchical vision transformer foundation model tailored for SITS analysis. At its core, we introduce a spatiotemporal gyroscope attention mechanism that dynamically captures evolving multiscale patterns across both time and space. For pre-training, we curate MillionST, a large-scale dataset of one million images from 100,000 geographic locations, each captured across 10 temporal phases over five years, encompassing diverse geospatial changes and seasonal variations. Leveraging this dataset, we adapt masked image modeling to pre-train TiMo, enabling it to effectively learn and encode generalizable spatiotemporal representations.Extensive experiments across multiple spatiotemporal tasks-including deforestation monitoring, land cover segmentation, crop type classification, and flood detection-demonstrate TiMo's superiority over state-of-the-art methods. Code, model, and dataset will be released at https://github.com/MiliLab/TiMo.
Figures
Figures from the paper (8 more)
Reference graph
Works this paper leans on
-
[1]
Vits for sits: Vision transformers for satellite image time series
Michail Tarasiou, Erik Chavez, and Stefanos Zafeiriou. Vits for sits: Vision transformers for satellite image time series. In CVPR, pages 10418–10428, 2023. 1, 7
2023
-
[2]
Multisenge: A multimodal and multitemporal benchmark dataset for land use/land cover remote sensing applications
Romain Wenger, Anne Puissant, Jonathan Weber, Lhas- sane Idoumghar, and Germain Forestier. Multisenge: A multimodal and multitemporal benchmark dataset for land use/land cover remote sensing applications. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Informa- tion Sciences, 3:635–640, 2022. 1, 6, 12
2022
-
[3]
Using difference features effectively: A multi-task network for exploring change areas and change moments in time series remote sensing images
Jialu Li and Chen Wu. Using difference features effectively: A multi-task network for exploring change areas and change moments in time series remote sensing images. ISPRS Jour- nal of Photogrammetry and Remote Sensing , 218:487–505, 2024
2024
-
[4]
Panoptic seg- mentation of satellite image time series with convolutional temporal attention networks
Vivien Sainte Fare Garnot and Loic Landrieu. Panoptic seg- mentation of satellite image time series with convolutional temporal attention networks. In ICCV, pages 4872–4881, 2021
2021
-
[5]
Spatiotemporal masked pre-training for advancing crop mapping on satellite image time series with limited labels
Xiaolei Qin, Haonan Guo, Xin Su, Zhenghui Zhao, Di Wang, and Liangpei Zhang. Spatiotemporal masked pre-training for advancing crop mapping on satellite image time series with limited labels. International Journal of Applied Earth Ob- servation and Geoinformation, 137:104426, 2025. 1
2025
-
[6]
Satmae: Pre-training transformers for tem- poral and multi-spectral satellite imagery
Yezhen Cong, Samar Khanna, Chenlin Meng, Patrick Liu, Erik Rozi, Yutong He, Marshall Burke, David Lobell, and Stefano Ermon. Satmae: Pre-training transformers for tem- poral and multi-spectral satellite imagery. NeurIPS, 35:197– 211, 2022. 2, 3, 5, 6, 7, 8, 12
2022
-
[7]
Momentum contrast for unsupervised visual rep- resentation learning
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual rep- resentation learning. In CVPR, pages 9729–9738, 2020. 2
2020
-
[8]
Seasonal contrast: Un- supervised pre-training from uncurated remote sensing data
Oscar Manas, Alexandre Lacoste, Xavier Gir ´o-i Nieto, David Vazquez, and Pau Rodriguez. Seasonal contrast: Un- supervised pre-training from uncurated remote sensing data. In ICCV, pages 9414–9423, 2021. 2, 3, 6, 7, 8
2021
Show all 103 references
-
[9]
Geography-aware self-supervised learning
Kumar Ayush, Burak Uzkent, Chenlin Meng, Kumar Tan- may, Marshall Burke, David Lobell, and Stefano Ermon. Geography-aware self-supervised learning. In ICCV, pages 10181–10190, 2021. 2, 3, 6, 7, 8
2021
-
[10]
Skysense: A multi-modal remote sens- ing foundation model towards universal interpretation for earth observation imagery
Xin Guo, Jiangwei Lao, Bo Dang, Yingying Zhang, Lei Yu, Lixiang Ru, Liheng Zhong, Ziyuan Huang, Kang Wu, Dingxiang Hu, et al. Skysense: A multi-modal remote sens- ing foundation model towards universal interpretation for earth observation imagery. In CVPR, pages 27672–27683,
-
[11]
Masked autoencoders are scalable vision learners
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll´ar, and Ross Girshick. Masked autoencoders are scalable vision learners. In CVPR, pages 16000–16009, 2022. 2, 4, 5, 12
2022
-
[12]
BEit: BERT pre-training of image transformers
Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. BEit: BERT pre-training of image transformers. In ICLR, 2022. 2
2022
-
[13]
Videomae: Masked autoencoders are data-efficient learners for self-supervised video pre-training
Zhan Tong, Yibing Song, Jue Wang, and Limin Wang. Videomae: Masked autoencoders are data-efficient learners for self-supervised video pre-training. NeurIPS, 35:10078– 10093, 2022. 2
2022
-
[14]
Prithvi-eo- 2.0: A versatile multi-temporal foundation model for earth observation applications
Daniela Szwarcman, Sujit Roy, Paolo Fraccaro, THorsteinn El ´ı G ´ıslason, Benedikt Blumenstiel, Rinki Ghosal, Pedro Henrique de Oliveira, Joao Lucas de Sousa Almeida, Rocco Sedona, Yanghui Kang, et al. Prithvi-eo- 2.0: A versatile multi-temporal foundation model for earth obs...
-
[15]
An image is worth 16x16 words: Transformers for image recognition at scale
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, 9 Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognitio...
2021
-
[16]
Functional map of the world
Gordon Christie, Neil Fendley, James Wilson, and Ryan Mukherjee. Functional map of the world. In CVPR, pages 6172–6180, 2018. 2, 3, 6
2018
-
[17]
Hi- era: A hierarchical vision transformer without the bells-and- whistles
Chaitanya Ryali, Yuan-Ting Hu, Daniel Bolya, Chen Wei, Haoqi Fan, Po-Yao Huang, Vaibhav Aggarwal, Arkabandhu Chowdhury, Omid Poursaeed, Judy Hoffman, et al. Hi- era: A hierarchical vision transformer without the bells-and- whistles. In ICML, pages 29441–29454. PMLR, 2023. 2, 3, 5
2023
-
[18]
A billion-scale foundation model for remote sensing images
Keumgang Cha, Junghoon Seo, and Taekyung Lee. A billion-scale foundation model for remote sensing images. IEEE Journal of Selected Topics in Applied Earth Observa- tions and Remote Sensing, pages 1–17, 2024. 2
2024
-
[19]
Hypersigma: Hy- perspectral intelligence comprehension foundation model
Di Wang, Meiqi Hu, Yao Jin, Yuchun Miao, Jiaqi Yang, Yichu Xu, Xiaolei Qin, Jiaqi Ma, Lingyu Sun, Chenxing Li, Chuan Fu, Hongruixuan Chen, Chengxi Han, Naoto Yokoya, Jing Zhang, Minqiang Xu, Lin Liu, Lefei Zhang, Chen Wu, Bo Du, Dacheng Tao, and Liangpei Zhang. Hypersigma: Hy-...
2025
-
[20]
Lexie Yang, and Dalton Lunga
Philipe Dias, Aristeidis Tsaris, Jordan Bowman, Ab- hishek Potnis, Jacob Arndt, H. Lexie Yang, and Dalton Lunga. Oreole-fm: Successes and challenges toward billion- parameter foundation models for high-resolution satellite imagery. In ACM SIGSPATIAL, 2024. 2
2024
-
[21]
Swin transformer v2: Scaling up capacity and resolution
Ze Liu, Han Hu, Yutong Lin, Zhuliang Yao, Zhenda Xie, Yixuan Wei, Jia Ning, Yue Cao, Zheng Zhang, Li Dong, Furu Wei, and Baining Guo. Swin transformer v2: Scaling up capacity and resolution. In CVPR, pages 12009–12019, June
-
[22]
Scaling vision with sparse mix- ture of experts
Carlos Riquelme, Joan Puigcerver, Basil Mustafa, Maxim Neumann, Rodolphe Jenatton, Andr ´e Susano Pinto, Daniel Keysers, and Neil Houlsby. Scaling vision with sparse mix- ture of experts. In NeurIPS, volume 34, pages 8583–8595, 2021
2021
-
[23]
Scaling vision transformers
Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, and Lu- cas Beyer. Scaling vision transformers. In CVPR, pages 12104–12113, June 2022
2022
-
[24]
Vi- TAE: Vision transformer advanced by exploring intrinsic in- ductive bias
Yufei Xu, Qiming Zhang, Jing Zhang, and Dacheng Tao. Vi- TAE: Vision transformer advanced by exploring intrinsic in- ductive bias. In NeurIPS, volume 34, pages 28522–28535, 2021
2021
-
[25]
Internimage: Exploring large-scale vi- sion foundation models with deformable convolutions
Wenhai Wang, Jifeng Dai, Zhe Chen, Zhenhang Huang, Zhiqi Li, Xizhou Zhu, Xiaowei Hu, Tong Lu, Lewei Lu, Hongsheng Li, et al. Internimage: Exploring large-scale vi- sion foundation models with deformable convolutions. In CVPR, pages 14408–14419, 2023. 2
2023
-
[26]
Satlaspretrain: A large- scale dataset for remote sensing image understanding
Favyen Bastani, Piper Wolters, Ritwik Gupta, Joe Ferdi- nando, and Aniruddha Kembhavi. Satlaspretrain: A large- scale dataset for remote sensing image understanding. In ICCV, pages 16772–16782, 2023. 2
2023
-
[27]
An empirical study of remote sensing pretraining.IEEE Transactions on Geoscience and Remote Sensing , 61:1–20,
Di Wang, Jing Zhang, Bo Du, Gui-Song Xia, and Dacheng Tao. An empirical study of remote sensing pretraining.IEEE Transactions on Geoscience and Remote Sensing , 61:1–20,
-
[28]
Swin transformer: Hierarchical vision transformer using shifted windows
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In ICCV, pages 10012–10022, October 2021. 2, 4
2021
-
[29]
Vi- taev2: Vision transformer advanced by exploring inductive bias for image recognition and beyond
Qiming Zhang, Yufei Xu, Jing Zhang, and Dacheng Tao. Vi- taev2: Vision transformer advanced by exploring inductive bias for image recognition and beyond. International Jour- nal of Computer Vision, 131(5):1141–1162, 2023. 2, 8
2023
-
[30]
Spectralgpt: Spec- tral remote sensing foundation model
Danfeng Hong, Bing Zhang, Xuyang Li, Yuxuan Li, Chenyu Li, Jing Yao, Pedram Ghamisi, Naoto Yokoya, Hao Li, Xiuping Jia, Antonio Plaza, et al. Spectralgpt: Spec- tral remote sensing foundation model. IEEE Transac- tions on Pattern Analysis and Machine Intelligence , 2024. DOI:1...
2024
-
[31]
Usat: A unified self-supervised encoder for multi-sensor satellite imagery
Jeremy Irvin, Lucas Tao, Joanne Zhou, Yuntao Ma, Langston Nashold, Benjamin Liu, and Andrew Y Ng. Usat: A unified self-supervised encoder for multi-sensor satellite imagery. arXiv preprint arXiv:2312.02199, 2023. 2
2023 arXiv
-
[32]
CSP: Self-supervised contrastive spatial pre-training for geospatial-visual representations
Gengchen Mai, Ni Lao, Yutong He, Jiaming Song, and Stefano Ermon. CSP: Self-supervised contrastive spatial pre-training for geospatial-visual representations. In ICML, pages 23498–23515. PMLR, 2023. 2
2023
-
[33]
GeoCLIP: Clip-inspired alignment between locations and images for effective worldwide geo-localization
Vicente Vivanco, Gaurav Kumar Nayak, and Mubarak Shah. GeoCLIP: Clip-inspired alignment between locations and images for effective worldwide geo-localization. InNeurIPS, 2023
2023
-
[34]
SatCLIP: Global, general- purpose location embeddings with satellite imagery
Konstantin Klemmer, Esther Rolf, Caleb Robinson, Lester Mackey, and Marc Rußwurm. SatCLIP: Global, general- purpose location embeddings with satellite imagery. ArXiv, abs/2311.17179, 2023. 2
2023 arXiv
-
[35]
Advancing plain vision transformer toward remote sensing foundation model
Di Wang, Qiming Zhang, Yufei Xu, Jing Zhang, Bo Du, Dacheng Tao, and Liangpei Zhang. Advancing plain vision transformer toward remote sensing foundation model. IEEE Transactions on Geoscience and Remote Sensing , 61:1–15,
-
[36]
RingMo: A remote sensing foundation model with masked image modeling
Xian Sun, Peijin Wang, Wanxuan Lu, Zicong Zhu, Xiao- nan Lu, Qibin He, Junxi Li, Xuee Rong, Zhujun Yang, Hao Chang, et al. RingMo: A remote sensing foundation model with masked image modeling. IEEE Transactions on Geo- science and Remote Sensing, 61:1–22, 2022
2022
-
[37]
Self- supervised material and texture representation learning for remote sensing tasks
Peri Akiva, Matthew Purri, and Matthew Leotta. Self- supervised material and texture representation learning for remote sensing tasks. In CVPR, pages 8203–8215, June 2022
2022
-
[38]
Cross-scale mae: A tale of multiscale exploitation in remote sensing
Maofeng Tang, Andrei Liviu Cozma, Konstantinos Geor- giou, and Hairong Qi. Cross-scale mae: A tale of multiscale exploitation in remote sensing. In NeurIPS, 2023. 5
2023
-
[39]
Masked angle-aware autoencoder for remote sensing images
Zhihao Li, Biao Hou, Siteng Ma, Zitong Wu, Xianpeng Guo, Bo Ren, and Licheng Jiao. Masked angle-aware autoencoder for remote sensing images. In ECCV, pages 260–278, 2024. 2, 5
2024
-
[40]
Bridging remote sensors with multisensor geospatial foundation models
Boran Han, Shuai Zhang, Xingjian Shi, and Markus Reich- stein. Bridging remote sensors with multisensor geospatial foundation models. In CVPR, pages 27852–27862, June
-
[41]
Neu- ral plasticity-inspired foundation model for observing the earth crossing modalities
Zhitong Xiong, Yi Wang, Fahong Zhang, Adam J Stewart, Jo¨elle Hanna, Damian Borth, Ioannis Papoutsis, Bertrand Le Saux, Gustau Camps-Valls, and Xiao Xiang Zhu. Neu- ral plasticity-inspired foundation model for observing the earth crossing modalities. arXiv preprint arXiv:2403....
2024
-
[42]
SenPa-MAE: Sensor parameter aware masked autoencoder for multi-satellite self- supervised pretraining
Jonathan Prexl and Michael Schmitt. SenPa-MAE: Sensor parameter aware masked autoencoder for multi-satellite self- supervised pretraining. ArXiv, abs/2408.11000, 2024
2024 arXiv
-
[43]
OmniSat: Self-supervised modality fusion for Earth observation
Guillaume Astruc, Nicolas Gonthier, Clement Mallet, and Loic Landrieu. OmniSat: Self-supervised modality fusion for Earth observation. ECCV, 2024
2024
-
[44]
AnySat: An Earth observation model for any resolutions, scales, and modalities
Guillaume Astruc, Nicolas Gonthier, Clement Mallet, and Loic Landrieu. AnySat: An Earth observation model for any resolutions, scales, and modalities. arXiv preprint arXiv:2412.14123, 2024. 2
2024 arXiv
-
[45]
Scale-mae: A scale-aware masked autoencoder for multiscale geospatial representation learning
Colorado J Reed, Ritwik Gupta, Shufan Li, Sarah Brock- man, Christopher Funk, Brian Clipp, Kurt Keutzer, Salvatore Candido, Matt Uyttendaele, and Trevor Darrell. Scale-mae: A scale-aware masked autoencoder for multiscale geospatial representation learning. In ICCV, pages 4088–...
2023
-
[46]
Towards geospatial foundation models via con- tinual pretraining
Mat ´ıas Mendieta, Boran Han, Xingjian Shi, Yi Zhu, and Chen Chen. Towards geospatial foundation models via con- tinual pretraining. In ICCV, pages 16806–16816, 2023
2023
-
[47]
MTP: Advancing remote sensing foun- dation model via multi-task pretraining
Di Wang, Jing Zhang, Minqiang Xu, Lin Liu, Dongsheng Wang, Erzhong Gao, Chengxi Han, Haonan Guo, Bo Du, Dacheng Tao, et al. MTP: Advancing remote sensing foun- dation model via multi-task pretraining. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sen...
2024
-
[48]
Consecutive pre-training: A knowledge transfer learning strategy with relevant unla- beled data for remote sensing domain
Tong Zhang, Peng Gao, Hao Dong, Yin Zhuang, Guanqun Wang, Wei Zhang, and He Chen. Consecutive pre-training: A knowledge transfer learning strategy with relevant unla- beled data for remote sensing domain. Remote Sensing , 14(22), 2022
2022
-
[49]
TOV: The original vision model for optical re- mote sensing image understanding via self-supervised learn- ing
Chao Tao, Ji Qi, Guo Zhang, Qing Zhu, Weipeng Lu, and Haifeng Li. TOV: The original vision model for optical re- mote sensing image understanding via self-supervised learn- ing. IEEE Journal of Selected Topics in Applied Earth Ob- servations and Remote Sensing, 2023
2023
-
[50]
CMID: A unified self-supervised learning framework for remote sensing image understanding
Dilxat Muhtar, Xueliang Zhang, Pengfeng Xiao, Zhenshi Li, and Feng Gu. CMID: A unified self-supervised learning framework for remote sensing image understanding. IEEE Transactions on Geoscience and Remote Sensing , 61:1–17,
-
[51]
Change- aware sampling and contrastive learning for satellite images
Utkarsh Mall, Bharath Hariharan, and Kavita Bala. Change- aware sampling and contrastive learning for satellite images. In CVPR, pages 5261–5270, 2023. 2, 3, 6, 7, 8
2023
-
[52]
A2-MAE: A spatial-temporal-spectral unified remote sens- ing pre-training method based on anchor-aware masked au- toencoder
Lixian Zhang, Yi Zhao, Runmin Dong, Jinxiao Zhang, Shuai Yuan, Shilei Cao, Mengxuan Chen, Juepeng Zheng, Weijia Li, Wei Liu, Wayne Zhang, Litong Feng, and Haohuan Fu. A2-MAE: A spatial-temporal-spectral unified remote sens- ing pre-training method based on anchor-aware masked ...
2024 arXiv
-
[53]
SSL4EO- S12: A large-scale multimodal, multitemporal dataset for self-supervised learning in earth observation [software and data sets]
Yi Wang, Nassim Ait Ali Braham, Zhitong Xiong, Chenying Liu, Conrad M Albrecht, and Xiao Xiang Zhu. SSL4EO- S12: A large-scale multimodal, multitemporal dataset for self-supervised learning in earth observation [software and data sets]. IEEE Geoscience and Remote Sensing Magaz...
2023
-
[54]
Spatio-temporal swin- mae: A swin transformer based multiscale representa- tion learner for temporal satellite imagery
Yohei Nakayama and Jiawei Su. Spatio-temporal swin- mae: A swin transformer based multiscale representa- tion learner for temporal satellite imagery. arXiv preprint arxiv:2405.02512, 2024. 2
2024 arXiv
-
[55]
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. NeurIPS, 30, 2017. 4
2017
-
[56]
MViTv2: Improved multiscale vision transformers for classification and detection
Yanghao Li, Chao-Yuan Wu, Haoqi Fan, Karttikeya Man- galam, Bo Xiong, Jitendra Malik, and Christoph Feichten- hofer. MViTv2: Improved multiscale vision transformers for classification and detection. In CVPR, pages 4804–4814, June 2022. 5
2022
-
[57]
Exploring plain vision transformer backbones for object de- tection
Yanghao Li, Hanzi Mao, Ross Girshick, and Kaiming He. Exploring plain vision transformer backbones for object de- tection. In ECCV, pages 280–296. Springer, 2022. 6
2022
-
[58]
Multiearth 2023– multimodal learning for earth and environment workshop and challenge
Miriam Cha, Gregory Angelides, Mark Hamilton, Andy Soszynski, Brandon Swenson, Nathaniel Maidel, Phillip Isola, Taylor Perron, and Bill Freeman. Multiearth 2023– multimodal learning for earth and environment workshop and challenge. arXiv preprint arXiv:2306.04738, 2023. 6, 12
2023 arXiv
-
[59]
Multi-temporal land cover classification with sequential recurrent encoders
Marc Rußwurm and Marco K ¨orner. Multi-temporal land cover classification with sequential recurrent encoders. IS- PRS International Journal of Geo-Information , 7(4):129,
-
[60]
Sen12-flood: a sar and multispectral dataset for flood detec- tion
Cl ´ement Rambour, Nicolas Audebert, Elise Koeniguer, Bertrand Le Saux, Michel Crucianu, and Mihai Datcu. Sen12-flood: a sar and multispectral dataset for flood detec- tion. IEEE: Piscataway, NJ, USA, 2020. 7, 12
2020
-
[61]
Kuro siwo: 33 billion mˆ2 under the water
Nikolaos Ioannis Bountos, Maria Sdraka, Angelos Za- vras, Andreas Karavias, Ilektra Karasante, Themistocles Herekakis, Angeliki Thanasou, Dimitrios Michail, and Ioan- nis Papoutsis. Kuro siwo: 33 billion mˆ2 under the water. a global multi-temporal satellite dataset for rapid ...
2024
-
[62]
Can foundation models wrangle your data? Proceedings of the VLDB Endowment, 16(4):738–746, December 2022
Avanika Narayan, Ines Chami, Laurel Orr, and Christopher R´e. Can foundation models wrangle your data? Proceedings of the VLDB Endowment, 16(4):738–746, December 2022. 8
2022
-
[63]
Towards foundation models for scientific machine learning: Characterizing scaling and transfer be- havior
Shashank Subramanian, Peter Harrington, Kurt Keutzer, Wahid Bhimji, Dmitriy Morozov, Michael W Mahoney, and Amir Gholami. Towards foundation models for scientific machine learning: Characterizing scaling and transfer be- havior. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M...
2023
-
[64]
12 2019/07/31 2020/08/04 2021/05/06 TiMo-BaseGTImage TiMo-Large Deforestation Forest Figure 8
For what purpose was the dataset created? Was there a specific task in mind? Was there a specific gap that needed to be filled? Please provide a description. 12 2019/07/31 2020/08/04 2021/05/06 TiMo-BaseGTImage TiMo-Large Deforestation Forest Figure 8. Visualization of the pre...
2019
-
[65]
Who created this dataset (e.g., which team, research group) and on behalf of which entity (e.g., company, in- stitution, organization)? A2: This dataset is created by the authors of this paper
-
[66]
Who funded the creation of the dataset? If there is an associated grant, please provide the name of the grantor and the grant name and number. A3: N/A. D.2. Composition
-
[67]
A1: MillionST is comprised of image subsets captured by the Sentinel-2 satellite
What do the instances that comprise the dataset rep- resent (e.g., documents, photos, people, countries)? Are there multiple types of instances(e.g., movies, users, and ratings; people and interactions between them; nodes and edges)? Please provide a description. A1: MillionST...
-
[68]
How many instances are there in total (of each type, if appropriate)? A2: MillionST has 100,000 instances, totally including 1,000,000 image patches
-
[69]
If it is not representative of the larger set, please describe why not (e.g., to cover a more diverse range of instances, because instances were withheld or unavailable)
Does the dataset contain all possible instances or is it a sample (not necessarily random) of instances from a larger set? If the dataset is a sample, then what is the larger set? Is the sample representative of the larger set (e.g., geographic coverage)? If so, please describ...
-
[70]
Visualization of the prediction results on MTLCC dataset
What data does each instance consist of? “Raw” data 13 TiMo-BaseGTImage TiMo-Large Unknown Rapeseed Winter spelt Winter wheat Winter barley Summer barley Maize Soybeans Figure 9. Visualization of the prediction results on MTLCC dataset. GT denotes ground truth. (e.g., unproces...
-
[71]
A5: No, since this dataset is intended for spatiotemporal self-supervised learning
Is there a label or target associated with each in- stance? If so, please provide a description. A5: No, since this dataset is intended for spatiotemporal self-supervised learning
-
[72]
This does not include intentionally removed informa- tion, but might include, e.g., redacted text
Is any information missing from individual instances? If so, please provide a description, explaining why this information is missing (e.g., because it was unavailable). This does not include intentionally removed informa- tion, but might include, e.g., redacted text. A6: No
-
[73]
Are relationships between individual instances made explicit (e.g., users’ movie ratings, social network links)? If so, please describe how these relationships are made explicit. A7: Yes. The relationships between different instances are shown in the folder name and metadata.json
-
[74]
A8: Yes, we recommend utilizing the whole dataset for spatiotemporal self-supervised pre-training
Are there recommended data splits (e.g., training, de- velopment/validation, testing)? If so, please provide a description of these splits, explaining the rationale be- hind them. A8: Yes, we recommend utilizing the whole dataset for spatiotemporal self-supervised pre-training
-
[75]
Are there any errors, sources of noise, or redundan- cies in the dataset? If so, please provide a description. A9: No
-
[76]
Is the dataset self-contained, or does it link to or oth- erwise rely on external resources (e.g., websites, tweets, other datasets)? If it links to or relies on external re- sources, a) are there guarantees that they will exist, and remain constant, over time; b) are there of...
-
[77]
Does the dataset contain data that might be con- sidered confidential (e.g., data that is protected by legal privilege or by doctorpatient confidentiality, data that includes the content of individuals non-public commu- nications)? If so, please provide a description. A11: No
-
[78]
Does the dataset contain data that, if viewed directly, might be offensive, insulting, threatening, or might oth- erwise cause anxiety? If so, please describe why. A12: No. 15 D.3. Collection Process
-
[79]
A1: The data associated with each instance are directly observable, as they are stored in the GeoTIFF format and can be accessed via Rasterio
How was the data associated with each instance ac- quired? Was the data directly observable (e.g., raw text, movie ratings), reported by subjects (e.g., survey re- sponses), or indirectly inferred/derived from other data (e.g., part-of-speech tags, model-based guesses for age ...
-
[80]
All operations are controlled by Python scripts to operate the Google Earth Engine
What mechanisms or procedures were used to col- lect the data (e.g., hardware apparatus or sensor, man- ual human curation, software program, software API)? How were these mechanisms or procedures validated? A2: The data after radiometric calibration and atmo- spheric correcti...
-
[81]
Spatially, for each sample, the process begins with uni- form sampling from 1,317 cities in Europe, North Africa, and West Asia
If the dataset is a sample from a larger set, what was the sampling strategy (e.g., deterministic, probabilistic with specific sampling probabilities)? A3: The images in the dataset are probabilistically sam- pled. Spatially, for each sample, the process begins with uni- form ...
2017
-
[82]
Who was involved in the data collection process (e.g., students, crowdworkers, contractors) and how were they compensated (e.g., how much were crowdworkers paid)? A4: The authors of this paper
-
[83]
A5: Since all operations are performed online, down- loading is seriously affected by the network connection sta- tus, and collecting data costs about 1 month
Over what timeframe was the data collected? Does this timeframe match the creation timeframe of the data associated with the instances (e.g., recent crawl of old news articles)? If not, please describe the timeframe in which the data associated with the instances was cre- ated...
-
[84]
If not, you may skip the remainder of the questions in this section
Was any preprocessing/cleaning/labeling of the data done (e.g., discretization or bucketing, tokenization, part-of-speech tagging, SIFT feature extraction, re- moval of instances, processing of missing values)? If so, please provide a description. If not, you may skip the rema...
-
[85]
Was the “raw” data saved in addition to the pre- processed/cleaned/labeled data (e.g., to support unantic- ipated future uses)? If so, please provide a link or other access point to the “raw” data. A2: N/A
-
[86]
Is the software used to preprocess/clean/label the in- stances available? If so, please provide a link or other access point. A3: N/A. D.5. Uses
-
[87]
Has the dataset been used for any tasks already? If so, please provide a description. A1: No
-
[88]
Is there a repository that links to any or all papers or systems that use the dataset? If so, please provide a link or other access point. A2: N/A
-
[89]
What (other) tasks could the dataset be used for? A3: It can be used for the research of RS self-supervised learning, especially for spatiotemporal pre-training
-
[90]
Is there anything a future user could do to mitigate these undesirable harms? A4: No
Is there anything about the composition of the dataset or the way it was collected and prepro- cessed/cleaned/labeled that might impact future uses? For example, is there anything that a future user might need to know to avoid uses that could result in unfair treatment of indi...
-
[91]
Are there tasks for which the dataset should not be used? If so, please provide a description. A5: No. D.6. Distribution
-
[92]
Will the dataset be distributed to third parties outside of the entity (e.g., company, institution, organization) on 16 behalf of which the dataset was created? If so, please provide a description. A1: Yes. The dataset will be publicly available
-
[93]
How will the dataset will be distributed (e.g., tarball on website, API, GitHub)? Does the dataset have a digi- tal object identifier (DOI)? A2: It will be publicly available on the project website
-
[94]
When will the dataset be distributed? A3: The dataset will be distributed once the paper is ac- cepted after peer review
-
[95]
A4: It will be distributed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 License
Will the dataset be distributed under a copyright or other intellectual property (IP) license, and/or under ap- plicable terms of use (ToU)? If so, please describe this license and/or ToU, and provide a link or other access point to, or otherwise reproduce, any relevant licens...
-
[96]
Have any third parties imposed IP-based or other re- strictions on the data associated with the instances? If so, please describe these restrictions, and provide a link or other access point to, or otherwise reproduce, any rel- evant licensing terms, as well as any fees associ...
-
[97]
Do any export controls or other regulatory restric- tions apply to the dataset or to individual instances? If so, please describe these restrictions, and provide a link or other access point to, or otherwise reproduce, any supporting documentation. A6: No. D.7. Maintenance
-
[98]
Who will be supporting/hosting/maintaining the dataset? A1: The authors
-
[99]
How can the owner/curator/manager of the dataset be contacted (e.g., email address)? A2: They can be contacted via email available on the project website
-
[100]
Is there an erratum? If so, please provide a link or other access point. A3: No
-
[101]
Will the dataset be updated (e.g., to correct label- ing errors, add new instances, delete instances)? If so, please describe how often, by whom, and how up- dates will be communicated to users (e.g., mailing list, GitHub)? A4: No
-
[102]
If not, please describe how its obsolescence will be commu- nicated to users
Will older versions of the dataset continue to be sup- ported/hosted/maintained? If so, please describe how. If not, please describe how its obsolescence will be commu- nicated to users. A5: N/A
-
[103]
Will these contribu- tions be validated/verified? If so, please describe how
If others want to extend/augment/build on/contribute to the dataset, is there a mechanism for them to do so? If so, please provide a description. Will these contribu- tions be validated/verified? If so, please describe how. If not, why not? Is there a process for communicat- i...
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.