Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

SMARTIES: Spectrum-Aware Multi-Sensor Auto-Encoder for Remote Sensing Images

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SMARTIES claims a single, sensor-agnostic transformer can replace per-sensor pretraining for remote sensing by projecting every spectral band into a shared spectrum-aware space, and shows the same frozen backbone outperforming…

desk verdict Solid seen-sensor results and a clean spectrum-aware design, but the unseen-sensor interpolation claim is effectively nearest-layer selection in the only test. read the letter →

arxiv 2506.19585 v1 pith:GSFNIOI5 submitted 2025-06-24 cs.CV

Gencer Sumbul , Chang Xu , Emanuele Dalsasso , Devis Tuia This is my paper
classification cs.CV
keywords remotesensingfoundationmodelmulti-sensorspectrum-awareprojectioncross-sensortokenmixupmaskedautoencoderself-supervisedlearningsensortransferEarthobservation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SMARTIES aims to prove that a single, sensor-agnostic transformer can replace the usual practice of training a separate model for each remote sensing instrument. It does this by sending every spectral band through its own small projection layer defined by the band's wavelength range, so that images from optical, multispectral, and radar sensors all land in one shared 'spectrum-aware' space. The model is pretrained to reconstruct masked patches from cross-sensor mixed tokens, and the authors report that it outperforms models that were pretrained specifically on a single sensor on both single- and multi-modal tasks across ten datasets. A further claim is that a new, never-seen sensor can be handled by interpolating the projection layers of the closest known wavelength ranges, as long as its bands fall inside the spectral span seen during pretraining.

What carries the argument

The load-bearing object is a bank of seventeen fully-connected projection layers, each tied to a spectral interval (twelve for Sentinel-2 bands, three for Maxar RGB, two for Sentinel-1 VV and VH). Every input patch is tokenized by sending each of its bands through the corresponding projector, averaging the resulting embeddings, and scaling the average by the largest number of bands seen during pretraining to prevent sensors with many bands from dominating. Training uses a masked autoencoder objective with a 75 percent masking ratio and mean-squared-error reconstruction loss on cross-sensor token mixup pairs, where a binary mask exchanges tokens between aligned pairs from different sensors. For unseen sensors, tokenization is done either by selecting existing projectors that overlap the new band's wavelength range or by linearly interpolating the two nearest projectors using the normalized distance between center wavelengths.

What would settle it

Train SMARTIES without Sentinel-1 data and then linear-probe on SAR (BigEarthNet-S1): the supplementary table already shows only about 2 percent mAP gain from projection extrapolation, far below the roughly 16.5 percent gain obtained when SAR is included in pretraining. A direct test of the interpolation assumption would take a held-out band whose center wavelength sits between two trained projectors but whose spatial resolution or radiometric profile differs strongly from both neighbors, and check whether interpolated projections beat randomly reinitializing that projector.

Watch

Extended reading notes

Core claim

The central claim is that sensor heterogeneity in remote sensing can be absorbed by a fixed set of wavelength-indexed linear projectors rather than by sensor-specific backbones. Each band of any input image is tokenized by a projector matched to the band's spectral range, the resulting per-band embeddings are averaged and scaled by a constant to balance sensors with different band counts, and a vanilla ViT encoder-decoder is trained with masked reconstruction on mixed tokens from paired Sentinel-2, RGB, and Sentinel-1 images. After pretraining, the same encoder and projectors serve any band combination at inference; for bands whose center wavelength lies between two trained ranges, the corresponding projectors are blended by normalized distance from central wavelengths. The paper shows this configuration beating single-sensor pretrained baselines and matching or exceeding multi-sensor ones, including on datasets and sensors never seen in pretraining.

Load-bearing premise

The claim that SMARTIES transfers to unseen sensors rests on the assumption that a new band's optimal projector is a linear interpolation of the projectors of the two neighboring spectral ranges, with no retraining of the transformer; the paper itself shows this fails when the new band lies outside the spectral range covered during pretraining.

Editorial extensions

If this is right

  • A single pretrained ViT backbone plus a bank of wavelength-indexed projectors can replace sensor-specific encoders in future remote sensing foundation models, cutting pretraining cost to about 496K images and 300 epochs.
  • Downstream tasks can mix arbitrary band combinations at inference, including SAR plus optical pairs, without architectural changes or sensor-specific retraining.
  • New sensors whose bands overlap the pretraining spectral range can be used with the backbone frozen, needing only a task head, as demonstrated on BurnScars, SpaceNet7, and SICKLE.
  • The model shows robustness to spatial-scale variation without scale-specific pretraining, improving kNN classification accuracy on EuroSAT, WHU-RS19, and UCMerced.
  • The approach scales with model size: ViT-L consistently improves over ViT-B, suggesting larger backbones would benefit further.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The interpolation rule is effectively a physics prior that band semantics vary smoothly with wavelength; a testable extension would replace linear interpolation with a small learned function of center wavelength and bandwidth, which the authors do not explore.
  • If the claim holds, the same projector-bank trick could extend to hyperspectral, thermal, or LiDAR intensity channels that map to wavelength or frequency, potentially unifying even more disparate Earth observation data types.
  • Because pretraining relies on paired, co-registered images for the mixup objective, the method's benefits may shrink where such pairs are scarce; an unpaired or weakly aligned pretraining variant would be the natural next test.
  • The demonstrated failure of extrapolation bounds the method: gains are guaranteed only within the pretraining spectral envelope, so adding a genuinely new part of the spectrum, such as thermal infrared beyond roughly 2.4 micrometers, would require new projectors and continued pretraining rather than pure zero-shot transfer.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces SMARTIES, a single-transformer foundation model for remote sensing that projects images from heterogeneous sensors into a shared spectrum-aware space via wavelength-dependent projection layers. Pretraining uses masked reconstruction with cross-sensor token mixup on paired Sentinel-2/Sentinel-1 and RGB data. The authors evaluate the model on ten datasets covering single- and multi-modal classification and segmentation, including transfer to sensors not seen during pretraining, and report competitive or state-of-the-art results against sensor-specific and multi-sensor foundation models. The claimed contributions are scalability to diverse sensors, data efficiency, and generalization to unseen sensors through projection-layer interpolation.

Significance. If the central claims hold, SMARTIES would be a useful step toward a single remote sensing foundation model that avoids sensor-specific backbones and pretraining. The paper has several concrete strengths: the proposed spectrum-aware projection is simple and architecture-agnostic; the pretraining budget (496K images, 300 epochs) is comparatively small; the code and pretrained models are promised to be public; and the supplementary material honestly reports the failure of extrapolation beyond the pretraining spectral range. The cross-sensor token mixup ablation provides evidence for the value of that design choice. The main risk is the overstatement of the unseen-sensor generalization claim, which is supported by limited and partly degenerate experimental evidence.

major comments (4)
  1. [Abstract, Sec. 5, Table 3] The abstract and conclusion state that SMARTIES 'outperforms previous models that rely on sensor-specific pretraining' on both single- and multi-modal tasks. Table 3 shows that on RESISC-45, SMARTIES ViT-L obtains 95.8% top-1 accuracy, which is below SatMAE++ (RGB) ViT-L at 97.5%. This is a direct counterexample to the unqualified claim. The text in Sec. 4.3 correctly softens this to 'highly competitive performance', but the abstract and conclusion do not. Please either restrict the claim or provide repeated-run statistics showing that the difference is within noise.
  2. [Sec. 3.4, Table 6, Sec. S3] The unseen-sensor interpolation experiment on SICKLE does not establish that interpolation to a spectral range outside the pretraining set works. The thermal band B10 is interpolated between S2 SWIR f12 (center about 2.2 um) and S1 VV f16 (center about 55 um); with inverse-distance weighting, the weight on f16 is on the order of 10^-4, so the interpolated projection is numerically almost identical to f12 alone. The 14.8 mIoU gain of SMARTIES (w PI) over SMARTIES (w/o PI) in Table 6 may be driven by the in-range interpolation of the Landsat-8 blue band between S2 B01 and B02, not by handling the out-of-range thermal band. Please add an ablation that isolates the thermal band (e.g., compare nearest-layer reuse vs. interpolation for B10 while keeping the blue-band handling fixed) and report the actual interpolation weights used.
  3. [Tables 1-8] All experimental results are reported as single numbers without standard deviations or the number of runs, and many baseline numbers are taken from previous papers (Tables 2, 3, 5) without re-running under identical protocols. Several margins are small, for example Table 1 BEN-S2 FT 87.7 vs. 87.6 for CROMA and Table 2 EuroSAT FT 99.6 vs. 99.2. Without variance estimates or a shared protocol, the claimed superiority is not fully quantifiable. Please add repeated-run statistics for at least the main comparisons, or clearly mark which baselines are re-run locally and which are transcribed.
  4. [Sec. 3.4, Sec. 1, Sec. 5, Table S4] The paper's framing as 'unprecedented generalization' to unseen sensors is stronger than the evidence. The supplementary Table S4 shows that extrapolation beyond the pretraining spectral range fails, and the only truly out-of-range transfer test (SICKLE thermal band) reduces to nearest-layer reuse as argued above. The in-range unseen-sensor cases (BurnScars/HLS, SpaceNet7/Planet) use bands that overlap the pretraining spectra and do not involve interpolation. The claim should be scoped to sensors whose bands fall within the pretraining spectral range, with interpolation demonstrated in a nontrivial regime, rather than stated as general capability for unseen sensors.
minor comments (4)
  1. [Sec. S1.2] The text says 'using AdamW optimizer (β1 = 0.9, β1 = 0.95)'; the second subscript should be β2.
  2. [Eq. (3)] The denominator 'RNW NH' is ambiguous; it should be written as (R · N_W · N_H) or defined explicitly as the number of masked tokens.
  3. [Sec. 4.2] The phrase 'We try to compare our models with all the existing models' is vague; consider replacing 'try to compare' with 'we compare'.
  4. [Sec. 3.1] Minor wording: 'provided in the Sec. S1.1' should be 'provided in Sec. S1.1'.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation; the spectrum-aware projection, self-supervised pretraining, and downstream evaluations are self-contained, with only benign data-source self-citations.

full rationale

SMARTIES's derivation chain is architectural and empirical rather than definitional. Projection layers f_i are learned fully-connected mappings per spectral range (Sec. 3.1, Tab. S1); the transformer is pretrained with masked reconstruction and cross-sensor mixup (Eqs. 1-3); and every headline result is measured against external benchmarks (EuroSAT, RESISC-45, BigEarthNet, PANGAEA, SICKLE). No fitted parameter is renamed as a prediction: the only interpolated quantity in Sec. 3.4 is a weighted average of pretrained projection layers, and its SICKLE test uses held-out Landsat-8 data with external crop-type labels, so the reported mIoU is not an input to the model. Self-citations ([39] BigEarthNet-MM dataset, [22] and [40] earlier SSL work, [43] a survey) are data sources or background context, not load-bearing premises, and no uniqueness theorem or ansatz is imported from the authors' prior work. The weakest point is experimental: the single true interpolation case (SICKLE thermal B10) is numerically dominated by the nearest projection layer f12 because the other endpoint f16 lies at roughly 55 um, so it approximates nearest-layer reuse rather than a graded interpolation; and the paper itself limits interpolation to in-range bands and shows extrapolation fails (Sec. S3). This is a limitation of the generalization evidence, not a circular reduction of the claimed result to its inputs.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central method rests on hand-chosen scaling and mixing hyperparameters plus several domain assumptions about spectral alignment and the validity of interpolation. No physical constants or fitted external benchmarks are used, and no new physical entities are introduced.

free parameters (3)
  • C_max = 12
    Chosen as the largest number of spectral bands in pretraining, used to scale averaged band embeddings to prevent imbalance between sensors with different band counts (Sec. 3.1).
  • mixup ratio = 0.5
    Determines the proportion of tokens exchanged in cross-sensor token mixup; fixed by hand, not tuned (Sec. 3.2, Sec. 4.2).
  • masking ratio R = 0.75
    Masked-image-modeling ratio inherited from vanilla MAE; a hand-chosen hyperparameter that affects the reconstruction task (Sec. 4.2).
assumptions (4)
  • domain assumption All RS sensors measure subsets of the electromagnetic spectrum, and a linear projection per spectral range is sufficient to embed them into a shared representation space.
    Underpins Sec. 3.1; if SAR amplitude, RGB reflectance, and multispectral radiance cannot be aligned by band-range-specific linear layers, the unified model fails.
  • domain assumption Paired, co-located multi-sensor images are available for pretraining and are used to define the cross-sensor token mixup objective.
    Sec. 3.2 and Sec. 4.1 rely on fMoW and BigEarthNet pairs; unpaired single-sensor data cannot provide the mixup signal.
  • domain assumption Interpolation between learned projection layers weighted by central wavelength yields valid token embeddings for unseen bands falling within the pretraining spectral range.
    Sec. 3.4 and the SICKLE experiments (Sec. 4.4) depend on this; the authors show it does not generalize to extrapolation (Tab. S4).
  • domain assumption Masked autoencoding with MSE reconstruction in the spectrum-aware space induces representations transferable to downstream classification and segmentation tasks.
    This is the standard SSL assumption inherited from MAE [17], used throughout Sec. 4 without independent validation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SMARTIES: Spectrum-Aware Multi-Sensor Auto-Encoder for Remote Sensing Images." pith.science (2026). https://pith.science/paper/GSFNIOI5

@misc{pith2026250619585,
  author       = {Pith},
  title        = {Pith review of: SMARTIES: Spectrum-Aware Multi-Sensor Auto-Encoder for Remote Sensing Images},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GSFNIOI5}},
  note         = {Machine review of arXiv:2506.19585}
}
read the original abstract

From optical sensors to microwave radars, leveraging the complementary strengths of remote sensing (RS) sensors is crucial for achieving dense spatio-temporal monitoring of our planet. In contrast, recent deep learning models, whether task-specific or foundational, are often specific to single sensors or to fixed combinations: adapting such models to different sensory inputs requires both architectural changes and re-training, limiting scalability and generalization across multiple RS sensors. On the contrary, a single model able to modulate its feature representations to accept diverse sensors as input would pave the way to agile and flexible multi-sensor RS data processing. To address this, we introduce SMARTIES, a generic and versatile foundation model lifting sensor-specific/dependent efforts and enabling scalability and generalization to diverse RS sensors: SMARTIES projects data from heterogeneous sensors into a shared spectrum-aware space, enabling the use of arbitrary combinations of bands both for training and inference. To obtain sensor-agnostic representations, we train a single, unified transformer model reconstructing masked multi-sensor data with cross-sensor token mixup. On both single- and multi-modal tasks across diverse sensors, SMARTIES outperforms previous models that rely on sensor-specific pretraining. Our code and pretrained models are available at https://gsumbul.github.io/SMARTIES.

Figures

Figures reproduced from arXiv: 2506.19585 by the authors.

Figure 1
Figure 1. (a) An example of RGB, multispectral, and SAR images [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. SMARTIES lifts sensor-dependent efforts for multi-sensor RS image representation learning by leveraging: (1) spectrum-aware [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Spectrum-aware RS image projection and reconstruction [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: An example of downstream transfer to an unseen spec [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: kNN classification accuracy (%) on EuroSAT versus pretraining epoch for ViT-B and ViT-L. 8 segmentation task by using both OLI and TIRS bands, we apply interpolation for the thermal infrared bands, and learn a single layer segmentation head to assess its unseen sen￾sor…

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. CanadaFireSat: Toward high-resolution wildfire forecasting with multiple modalities

    cs.CV 2025-06 conditional novelty 6.5 of 10

    Introduces a multi-modal 100m wildfire forecasting benchmark for Canada and shows deep learning models benefit from fusing Sentinel-2 imagery with environmental predictors.

Reference graph

Works this paper leans on

48 extracted references · 42 canonical work pages · cited by 1 Pith paper

  1. [1]

    Self-supervised learning from images with a joint-embedding predictive architecture

    Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bo- janowski, Pascal Vincent, Michael Rabbat, Yann LeCun, and Nicolas Ballas. Self-supervised learning from images with a joint-embedding predictive architecture. InIEEE/CVF Conf. Comput. Vis. Pattern Recog. (CVPR), pages 15619–15629,

    work page

  2. [2]

    Geography-aware self-supervised learning

    Kumar Ayush, Burak Uzkent, Chenlin Meng, Kumar Tan- may, Marshall Burke, David Lobell, and Stefano Ermon. Geography-aware self-supervised learning. InInt. Conf. Comput. Vis. (ICCV), pages 10181–10190, 2021. 6

    work page 2021

  3. [3]

    Multimae: Multi-modal multi-task masked autoen- coders

    Roman Bachmann, David Mizrahi, Andrei Atanov, and Amir Zamir. Multimae: Multi-modal multi-task masked autoen- coders. InEur. Conf. Comput. Vis. (ECCV), pages 348–367. Springer, 2022. 2

    work page 2022

  4. [4]

    Satlaspretrain: A large- scale dataset for remote sensing image understanding

    Favyen Bastani, Piper Wolters, Ritwik Gupta, Joe Ferdi- nando, and Aniruddha Kembhavi. Satlaspretrain: A large- scale dataset for remote sensing image understanding. InInt. Conf. Comput. Vis. (ICCV), pages 16772–16782, 2023. 6

    work page 2023

  5. [5]

    Encoder-decoder with atrous separable convolution for semantic image segmentation

    Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Eur. Conf. Comput. Vis. (ECCV), pages 801–818, 2018. 7

    work page 2018

  6. [6]

    Remote sens- ing image scene classification: Benchmark and state of the art.Proceedings of the IEEE, 105(10):1865–1883, 2017

    Gong Cheng, Junwei Han, and Xiaoqiang Lu. Remote sens- ing image scene classification: Benchmark and state of the art.Proceedings of the IEEE, 105(10):1865–1883, 2017. 6

    2017

  7. [7]

    Big data for remote sensing: Challenges and opportunities.Proceed- ings of the IEEE, 104(11):2207–2219, 2016

    Mingmin Chi, Antonio Plaza, Jón Atli Benediktsson, Zhongyi Sun, Jinsheng Shen, and Yangyong Zhu. Big data for remote sensing: Challenges and opportunities.Proceed- ings of the IEEE, 104(11):2207–2219, 2016. 1

    work page 2016

  8. [8]

    Functional map of the world

    Gordon Christie, Neil Fendley, James Wilson, and Ryan Mukherjee. Functional map of the world. InIEEE/CVF Conf. Comput. Vis. Pattern Recog. (CVPR), pages 6172– 6180, 2018. 5

    work page 2018

Show all 48 references
  1. [9]

    Lobell, and Stefano Ermon

    Yezhen Cong, Samar Khanna, Chenlin Meng, Patrick Liu, Erik Rozi, Yutong He, Marshall Burke, David B. Lobell, and Stefano Ermon. Satmae: Pre-training transformers for tem- poral and multi-spectral satellite imagery. InAdv. Neural Inform. Process. Syst. (NeurIPS), pages 197–211,...

    2022

  2. [10]

    Satellite image classification via two-layer sparse coding with biased image representa- tion.IEEE Geosci

    Dengxin Dai and Wen Yang. Satellite image classification via two-layer sparse coding with biased image representa- tion.IEEE Geosci. Remote Sens. Lett. (GRSL), 8(1):173– 176, 2010. 6, 7

    2010

  3. [11]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition ...

    2021

  4. [12]

    Anthony Fuller, Koreen Millard, and James R. Green. CROMA: Remote sensing representations with contrastive radar-optical masked autoencoders. InAdv. Neural Inform. Process. Syst. (NeurIPS), pages 5506–5538, 2023. 1, 2, 6, 7, 3

    2023

  5. [13]

    MCMAE: Masked convolution meets masked au- toencoders

    Peng Gao, Teli Ma, Hongsheng Li, Ziyi Lin, Jifeng Dai, and Yu Qiao. MCMAE: Masked convolution meets masked au- toencoders. InAdv. Neural Inform. Process. Syst. (NeurIPS), pages 35632–35644, 2022. 7

    2022

  6. [14]

    Multimodal classification of remote sensing images: A review and future directions.Proceedings of the IEEE, 103(9):1560–1584, 2015

    Luis Gómez-Chova, Devis Tuia, Gabriele Moser, and Gustau Camps-Valls. Multimodal classification of remote sensing images: A review and future directions.Proceedings of the IEEE, 103(9):1560–1584, 2015. 1

    2015

  7. [15]

    SkySense: A multi-modal remote sensing foundation model towards universal interpretation for earth observation imagery

    Xin Guo, Jiangwei Lao, Bo Dang, Yingying Zhang, Lei Yu, Lixiang Ru, Liheng Zhong, Ziyuan Huang, et al. SkySense: A multi-modal remote sensing foundation model towards universal interpretation for earth observation imagery. In IEEE/CVF Conf. Comput. Vis. Pattern Recog. (CVPR), ...

    2024

  8. [16]

    Bridging remote sensors with multisensor geospatial foundation models

    Boran Han, Shuai Zhang, Xingjian Shi, and Markus Reich- stein. Bridging remote sensors with multisensor geospatial foundation models. InIEEE/CVF Conf. Comput. Vis. Pattern Recog. (CVPR), pages 27852–27862, 2024. 6

    2024

  9. [17]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. InIEEE/CVF Conf. Comput. Vis. Pattern Recog. (CVPR), pages 16000–16009, 2022. 2, 5, 7, 1, 4

    2022

  10. [18]

    Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification.IEEE J

    Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification.IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. (JSTARS), 12(7): 2217–2226, 2019. 6

    2019

  11. [19]

    SpectralGPT: Spectral remote sensing foundation model.IEEE Trans

    Danfeng Hong, Bing Zhang, Xuyang Li, Yuxuan Li, Chenyu Li, Jing Yao, Naoto Yokoya, Hao Li, et al. SpectralGPT: Spectral remote sensing foundation model.IEEE Trans. Pat- tern Anal. Mach. Intell. (TPAMI), 46(8):5227–5244, 2024. 1, 2, 6, 3, 4

    2024

  12. [20]

    Johannes Jakubik, Sujit Roy, C. E. Phillips, Paolo Fraccaro, Denys Godwin, Bianca Zadrozny, Daniela Szwarcman, Car- los Gomes, et al. Foundation models for generalist geospa- tial artificial intelligence.arXiv preprint arXiv:2310.18660,

    arXiv

  13. [21]

    TerraMind: Large- scale generative multimodality for earth observation.arXiv preprint arXiv:2504.11171, 2025

    Johannes Jakubik, Felix Yang, Benedikt Blumenstiel, Erik Scheurer, Rocco Sedona, Stefano Maurogiovanni, Jente Bosmans, Nikolaos Dionelis, et al. TerraMind: Large- scale generative multimodality for earth observation.arXiv preprint arXiv:2504.11171, 2025. 7

    2025 arXiv

  14. [22]

    Self-supervised pre-training enhances change de- tection in Sentinel-2 images

    Marrit Leenstra, Diego Marcos, Francesca Bovolo, and De- vis Tuia. Self-supervised pre-training enhances change de- tection in Sentinel-2 images. InInt. Conf. Pattern Recog. (ICPR), Workshop Pattern Recog. in Remote Sens., pages 578–590, 2021. 2

    2021

  15. [23]

    S2mae: A spatial-spectral pretraining foundation model for spectral remote sensing data

    Xuyang Li, Danfeng Hong, and Jocelyn Chanussot. S2mae: A spatial-spectral pretraining foundation model for spectral remote sensing data. InIEEE/CVF Conf. Comput. Vis. Pat- tern Recog. (CVPR), pages 27696–27705, 2024. 1, 2, 6, 3

    2024

  16. [24]

    9 Exploring plain vision transformer backbones for object de- tection.arXiv preprint arXiv:2203.16527, 2022

    Yanghao Li, Hanzi Mao, Ross Girshick, and Kaiming He. 9 Exploring plain vision transformer backbones for object de- tection.arXiv preprint arXiv:2203.16527, 2022. 2

    2022 arXiv

  17. [25]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. InInt. Conf. Learn. Represent. (ICLR), 2019. 5, 1

    2019

  18. [26]

    Seasonal contrast: Un- supervised pre-training from uncurated remote sensing data

    Oscar Mañas, Alexandre Lacoste, Xavier Giró-i Nieto, David Vazquez, and Pau Rodríguez. Seasonal contrast: Un- supervised pre-training from uncurated remote sensing data. InInt. Conf. Comput. Vis. (ICCV), pages 9414–9423, 2021. 6, 2

    2021

  19. [27]

    Change- aware sampling and contrastive learning for satellite images

    Utkarsh Mall, Bharath Hariharan, and Kavita Bala. Change- aware sampling and contrastive learning for satellite images. InIEEE/CVF Conf. Comput. Vis. Pattern Recog. (CVPR), pages 5261–5270, 2023. 6

    2023

  20. [28]

    PANGAEA: A global and inclusive benchmark for geospatial foundation models.arXiv preprint 2412.04204, 2024

    Valerio Marsocci, Yuru Jia, Georges Le Bellier, David Kerekes, Liang Zeng, Sebastian Hafner, Sebastian Gerard, Eric Brune, et al. PANGAEA: A global and inclusive benchmark for geospatial foundation models.arXiv preprint 2412.04204, 2024. 7, 2, 3

    2024 arXiv

  21. [29]

    Towards geospatial foundation models via con- tinual pretraining

    Matías Mendieta, Boran Han, Xingjian Shi, Yi Zhu, and Chen Chen. Towards geospatial foundation models via con- tinual pretraining. InInt. Conf. Comput. Vis. (ICCV), pages 16806–16816, 2023. 6, 7

    2023

  22. [30]

    4M: Massively multimodal masked modeling.Adv

    David Mizrahi, Roman Bachmann, Oguzhan Kar, Teresa Yeo, Mingfei Gao, Afshin Dehghan, and Amir Zamir. 4M: Massively multimodal masked modeling.Adv. Neural In- form. Process. Syst. (NeurIPS), pages 58363–58408, 2023. 2

    2023

  23. [31]

    Mmearth: Explor- ing multi-modal pretext tasks for geospatial representation learning

    Vishal Nedungadi, Ankit Kariryaa, Stefan Oehmcke, Serge Belongie, Christian Igel, and Nico Lang. Mmearth: Explor- ing multi-modal pretext tasks for geospatial representation learning. InEur. Conf. Comput. Vis. (ECCV), pages 164– 182, 2024. 2

    2024

  24. [32]

    Rethinking transformers pre-training for multi- spectral satellite imagery

    Mubashir Noman, Muzammal Naseer, Hisham Cholakkal, Rao Muhammad Anwar, Salman Khan, and Fahad Shah- baz Khan. Rethinking transformers pre-training for multi- spectral satellite imagery. InIEEE/CVF Conf. Comput. Vis. Pattern Recog. (CVPR), pages 27811–27819, 2024. 1, 6, 7, 3

    2024

  25. [33]

    SenPa-MAE: Sen- sor parameter aware masked autoencoder for multi-satellite self-supervised pretraining

    Jonathan Prexl and Michael Schmitt. SenPa-MAE: Sen- sor parameter aware masked autoencoder for multi-satellite self-supervised pretraining. InGerman Conf. Pattern Recog. (GCPR), 2024. 2, 7

    2024

  26. [34]

    Scale-mae: A scale-aware masked autoencoder for multiscale geospatial representation learning

    Colorado J Reed, Ritwik Gupta, Shufan Li, Sarah Brock- man, Christopher Funk, Brian Clipp, Kurt Keutzer, Salvatore Candido, Matt Uyttendaele, and Trevor Darrell. Scale-mae: A scale-aware masked autoencoder for multiscale geospatial representation learning. InInt. Conf. Comput....

    2023

  27. [35]

    U- Net: Convolutional networks for biomedical image segmen- tation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U- Net: Convolutional networks for biomedical image segmen- tation. InInt. Conf. Medical Image Comput. Computer- assisted Intervention (MICCAI), pages 234–241. Springer,

  28. [36]

    Lightweight temporal self-attention for classifying satellite images time series.arXiv preprint arXiv:2007.00586, 2020

    Vivien Sainte Fare Garnot and Loic Landrieu. Lightweight temporal self-attention for classifying satellite images time series.arXiv preprint arXiv:2007.00586, 2020. 3

    2007 arXiv

  29. [37]

    SICKLE: A multi- sensor satellite imagery dataset annotated with multiple key cropping parameters

    Depanshu Sani, Sandeep Mahato, Sourabh Saini, Harsh Ku- mar Agarwal, Charu Chandra Devshali, Saket Anand, Gau- rav Arora, and Thiagarajan Jayaraman. SICKLE: A multi- sensor satellite imagery dataset annotated with multiple key cropping parameters. InIEEE/CVF Winter Conf. on Ap...

    2024

  30. [38]

    Contrastive self-supervised data fusion for satellite imagery

    Linus Scheibenreif, Michael Mommert, and Damian Borth. Contrastive self-supervised data fusion for satellite imagery. ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci., 3: 705–711, 2022. 2

    2022

  31. [39]

    BigEarthNet-MM: A large scale multi-modal multi-label benchmark archive for remote sensing image classification and retrieval.IEEE Geosci

    Gencer Sumbul, Arne de Wall, Tristan Kreuziger, Filipe Marcelino, Hugo Costa, Pedro Benevides, Mário Caetano, Begüm Demir, and V olker Markl. BigEarthNet-MM: A large scale multi-modal multi-label benchmark archive for remote sensing image classification and retrieval.IEEE Geos...

    2021

  32. [40]

    A novel self-supervised cross-modal image retrieval method in re- mote sensing

    Gencer Sumbul, Markus Müller, and Begüm Demir. A novel self-supervised cross-modal image retrieval method in re- mote sensing. InIEEE Int. Conf. Image Process. (ICIP), pages 2426–2430, 2022. 2

    2022

  33. [41]

    Cross-scale MAE: A tale of multi- scale exploitation in remote sensing

    Maofeng Tang, Andrei Liviu Cozma, Konstantinos Geor- giou, and Hairong Qi. Cross-scale MAE: A tale of multi- scale exploitation in remote sensing. InAdv. Neural Inform. Process. Syst. (NeurIPS), pages 20054–20066, 2023. 7

    2023

  34. [42]

    DynamicEarthNet: Daily multi- spectral satellite dataset for semantic change segmentation

    Aysim Toker, Lukas Kondmann, Mark Weber, Marvin Eisen- berger, Camero Andres, Jingliang Hu, Ariadna Hoderlein, Caglar Senaras, et al. DynamicEarthNet: Daily multi- spectral satellite dataset for semantic change segmentation. InIEEE/CVF Conf. Comput. Vis. Pattern Recog. (CVPR),

  35. [43]

    Towards a collective agenda on AI for earth science data analysis

    Devis Tuia, Ribana Roscher, Jan Dirk Wegner, Nathan Ja- cobs, Xiao Xiang Zhu, and Gustua Camps-Valls. Towards a collective agenda on AI for earth science data analysis. IEEE Geosci. Remote Sens. Magazine (GRSM), 9(2):88–104,

  36. [44]

    The multi- temporal urban development spacenet dataset

    Adam Van Etten, Daniel Hogan, Jesus Martinez Manso, Ja- cob Shermeyer, Nicholas Weir, and Ryan Lewis. The multi- temporal urban development spacenet dataset. InIEEE/CVF Conf. Comput. Vis. Pattern Recog. (CVPR), pages 6394– 6403, 2021. 7

    2021

  37. [45]

    Self-supervised learn- ing in remote sensing: A review.IEEE Geosci

    Yi Wang, Conrad M Albrecht, Nassim Ait Ali Braham, Lichao Mou, and Xiao Xiang Zhu. Self-supervised learn- ing in remote sensing: A review.IEEE Geosci. Remote Sens. Magazine (GRSM), 10(4):213–247, 2022. 2

    2022

  38. [46]

    Unified perceptual parsing for scene understand- ing

    Tete Xiao, Yingcheng Liu, Bolei Zhou, Yuning Jiang, and Jian Sun. Unified perceptual parsing for scene understand- ing. InEur. Conf. Comput. Vis. (ECCV), 2018. 7

    2018

  39. [47]

    Neural plasticity-inspired multimodal foundation model for earth observation.arXiv preprint arXiv:2403.15356, 2024

    Zhitong Xiong, Yi Wang, Fahong Zhang, Adam J Stewart, Joëlle Hanna, Damian Borth, Ioannis Papoutsis, Bertrand Le Saux, Gustau Camps-Valls, and Xiao Xiang Zhu. Neural plasticity-inspired multimodal foundation model for earth observation.arXiv preprint arXiv:2403.15356, 2024. 1, 2, 7

    2024

  40. [48]

    Bag-of-visual-words and spatial extensions for land-use classification

    Yi Yang and Shawn Newsam. Bag-of-visual-words and spatial extensions for land-use classification. InSIGSPA- TIAL Int. Conf. Advances in Geographic Inf. Systems, page 270–279, 2010. 6, 7 10 SMARTIES: Spectrum-Aware Multi-Sensor Auto-Encoder for Remote Sensing Images Supplementa...

    2010

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.