Pith. sign in

REVIEW 3 major objections 6 minor 52 references

MAPEX: Modality-Aware Pruning of Experts for Remote Sensing Foundation Models

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MAPEX shows that a mixture-of-modality experts model pre-trained on six remote sensing modalities can be pruned into per-modality specialists that match or beat much larger single-modality foundation models on flood, wildfire, and…

desk verdict Useful efficiency idea, but the main land-cover benchmark may be contaminated by pre-training overlap; the external flood and wildfire results are the cleaner evidence. read the letter →

arxiv 2507.07527 v1 pith:WKWVHKKF submitted 2025-07-10 cs.CV

Joelle Hanna , Linus Scheibenreif , Damian Borth This is my paper
classification cs.CV
keywords mixture-of-expertsmodality-awareroutingexpertpruningremotesensingfoundationmodelsself-supervisedpre-trainingmaskedautoencodingflooddetectionwildfire
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MAPEX aims to resolve a mismatch in remote sensing foundation models: pre-training is usually done on one modality, while real tasks use SAR, SWIR, elevation, or other inputs. The paper's proposal is to pre-train one mixture-of-modality experts transformer on six co-located modalities, with routing that forces input tokens of each modality onto dedicated experts, and then, for each downstream task, prune the network down to the top experts of the task's modality. This produces a small, modality-specialized model that the authors show can be fine-tuned as easily as a single-modality model. The central quantitative claim is that these pruned models outperform or match much larger single-modality foundation models: on flood detection from SAR, MAPEX beats SatMAE by 3% and Scale-MAE by 1% in accuracy, and on wildfire detection from SWIR it beats SatMAE by 2% and matches Scale-MAE while using far fewer parameters. If correct, this means one multi-modal pre-training run can supply many cheap per-modality specialists, including for modalities that currently have no dedicated foundation model.

What carries the argument

The central mechanism is the mixture-of-modality experts block, where the router does not look at token content but at a learnable [MODALITY] embedding associated with each input modality; this guarantees that every token of a modality is routed to the same subset of experts, producing explicit expert-to-modality structure. Around this sit a load-balancing loss that keeps expert utilization uniform, a shared expert that processes all modalities to absorb common features, and modality dropout that randomly zeros entire modalities during pre-training. Downstream, pruning simply keeps the top-k experts by routing weight for the modality of interest and drops the rest, turning a 360M-parameter pre-training model into 130M-parameter task models.

What would settle it

Check whether any sample identifiers in the ben-ge-8k evaluation subset also appear in the ben-ge pre-training split; if they do, or if the split is not documented, re-run Tables 1, 5, 6, 7, and 10 on a held-out set and compare. A second check would be to train the same pipeline with token-based routing and show that pruned experts no longer concentrate on their intended modality, which would indicate the routing mechanism, not the benchmark, is responsible for specialization.

Watch

Extended reading notes

Core claim

The paper claims that modality-conditioned routing during masked autoencoding pre-training creates experts that are genuinely aligned with modalities, and that this alignment survives pruning. Concretely, the authors report that after keeping only the two experts most strongly routed to a given modality, the resulting sub-network performs best on that modality's data, and that these pruned models reach 75.6% fine-tuned accuracy on RGB land-cover classification, 77.8% on red edge, 79.8% on SWIR, 59.4% on NIR, 73.6% on SAR, and 58.8% on elevation, topping the comparison on NIR, SAR, and elevation. They also report that modality dropout of 50% during pre-training improves downstream k-NN accuracy by about 1.5% absolute on average, indicating that forcing the model to handle missing modalities helps the pruning stage. The paper's stated conclusion is that the architecture combines the benefit of large-scale multi-modal pre-training with the deployment convenience of small specialized models.

Load-bearing premise

The load-bearing assumption is that the 8,000 ben-ge samples used for land-cover evaluation were not part of the ben-ge data used for pre-training; the paper describes ben-ge-8k as a subset of ben-ge but never states that it was held out.

Editorial extensions

If this is right

  • A single multi-modal pre-training run can produce one specialized sub-model per modality, so new tasks on under-served modalities such as elevation, NIR, or SAR no longer need a purpose-built pre-trained model.
  • Downstream fine-tuning becomes cheaper because the pruned model is roughly a third of the pre-training size, and pruning is done before fine-tuning so memory and latency shrink.
  • Because routing is deterministic per modality, deployment can mix and match: keeping experts for RGB and SAR together produces a multimodal model with no further architecture changes.
  • The fixed top-2 pruning rule is a practical default: in the size-scaling experiment, most modalities peak at 2 experts per modality, while RGB prefers 5 smaller experts, so a fixed count is a compromise rather than an optimum.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the ben-ge-8k evaluation subset overlaps the ben-ge pre-training set, the reported accuracies, especially the k-NN results, would be inflated; the paper does not state that the 8,000 samples were held out, so the cleanest test is re-evaluating on a disjoint split.
  • The pruning rule is validated only on the same benchmark used to justify it; an independent check would be to pre-train with a random or token-based router and show that top-2 pruning then does not yield the same specialization.
  • The method is presented as modality-aware, but nothing in the routing stops the shared expert from carrying task information; ablating or shrinking the shared expert could reveal how much of the pruning benefit comes from the modality experts alone.
  • A natural extension is continual addition of new modalities: because routing is conditioned on an embedding, a new sensor could be inserted with its own [MODALITY] token and fine-tuned while leaving existing experts frozen, though the paper does not run this experiment.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces MAPEX, a remote-sensing foundation model that combines masked autoencoding with a mixture-of-modality experts encoder and a router conditioned on modality tokens. After pre-training on BigEarthNet-GeoEnvironment (ben-ge), the authors prune all experts except the top-k routed experts for a target modality, yielding a smaller modality-specialized model that is then evaluated on land-cover classification and segmentation on ben-ge-8k, flood detection on SEN12-FLOOD, and wildfire detection on California Fire Perimeters. The reported results show MAPEX matching or exceeding SatMAE and Scale-MAE on several modalities with fewer parameters, and the paper also presents ablations of modality dropout, shared expert, routing mechanism, and expert count.

Significance. If the ben-ge-8k evaluation were based on samples held out from pre-training, the paper would make a useful practical contribution: converting a large multi-modal MoE into compact per-modality models with competitive accuracy, plus external validation on flood and wildfire tasks. The promise of code release and the use of independent SEN12-FLOOD and California Wildfire benchmarks are strengths. However, the central land-cover comparisons and all ablations rely on ben-ge-8k, a subset of the very dataset used for pre-training, and the manuscript never states that this subset is disjoint from pre-training. This missing-support issue is currently the main barrier to accepting the quantitative claims.

major comments (3)
  1. [§4 and §5.1] The evaluation benchmark ben-ge-8k is described as “a subset of ben-ge” (§4), while ben-ge is used for pre-training (§4, §5.2); the paper never states that the 8,000 samples were excluded from pre-training. If they were not excluded, the k-NN and fine-tuning numbers in Tables 1, 4, 5, 6, 7, 8, 9, and 10 and Figures 4–6 could reflect memorization rather than transfer, and the comparison with SatMAE and Scale-MAE, which were not pre-trained on ben-ge, would be unfair. This is a load-bearing point for the paper’s central claim. Please report the exact split protocol, provide sample identifiers, and either confirm the holdout or re-run the ben-ge-8k experiments on a genuinely held-out subset.
  2. [§3.2 and Figure 6] Because the routing weights are computed as w_m^r = r([MODALITY]_m), the router input already encodes the modality identity, so the observation that each modality activates a consistent subset of experts is guaranteed by construction. The paper interprets the heatmap in Figure 6 as evidence that “routing successfully instills expert–modality relationships” into the model; this conflates a designed conditioning mechanism with emergent specialization. I would ask the authors to reframe the claim or test whether a token-based router with load balancing produces measurably different routing behavior, so that the “specialization” result is informative rather than circular.
  3. [§3.3 and Figure 5] The pruning rule in Equation (2) is validated mainly on the same ben-ge-8k benchmark, so the choice of top-2 experts and the expert-size trade-offs in Figure 5 inherit the contamination risk described above. In addition, the notation “arg topk(w_m^r)” is ambiguous: w_m^r is introduced as the routing probability for a modality, and the equation would benefit from an explicit definition of the top-k operator over the expert set. Please clarify the notation and, if possible, validate the pruning choice on one of the external benchmarks (SEN12-FLOOD or California Wildfire) as well.
minor comments (6)
  1. [§4] The phrase “ben-ge-8k consisting of ≈8 000samples” is missing a space; it should be “≈8,000 samples.”
  2. [§3.3] Equation (2) uses “arg topk” without a definition; please define it as returning the indices of the k experts with the largest routing probabilities for modality m.
  3. [§5.1, Tables 2 and 3] For the SEN12-FLOOD and California Wildfire datasets, please specify the exact train/test split and whether the split avoids spatial or temporal leakage (e.g., same-fire-perimeter patches appearing in both training and test).
  4. [Supplementary Material, Table 9] The “Supervised FS” column reports fine-tuning accuracy, but the caption does not state this explicitly; please clarify the protocol and architecture for that baseline.
  5. [Figure 6] The color scale and the “best and worst specialized models” highlights are difficult to read in the printed figure; adding a colorbar and explicit labels or annotations would improve interpretability.
  6. [§2, Related Work] The discussion of MoMa [29] and DAMEX [23] would be clearer with one sentence explicitly stating what MAPEX adds relative to MoMa’s modality-aware routing, since both use modality-specific experts.

Circularity Check

1 steps flagged · score 2.0 of 10

No central circularity: pruning-performance claims are empirically grounded; only the 'expert specialization' evidence is partly self-definitional because routing is conditioned on the modality label.

  1. self definitional [Sec. 3.2 (Modality routing) and Sec. 5.1 (Specialization of Experts, Fig. 6)]
    "Unlike standard MoE implementations, this routing strategy guarantees that tokens of a modality will always be routed to the same experts, even across samples. ... To test the effectiveness of these specialized models, we create modality-specific versions and evaluate them across all available data types. ... The heatmap reveals that each specialized model performs best when tested on its intended data type (e.g., RGB experts on RGB data, NIR experts on NIR data)."

    The 'RGB experts' are not discovered by the heatmap; they are defined by the routing rule wm_r = r([MODALITY]m), which the paper states guarantees that each modality's tokens always go to the same experts. A 'modality-specific version' is therefore, by construction, the set of experts that already received only that modality's tokens during pre-training (shared expert aside). Testing 'RGB experts on RGB data' and reading the diagonal as evidence that routing 'instills' expert↔modality relationships checks the same assignment used to build the specialized models. The performance magnitudes in Tables 1-4 remain empirical, but the claimed discovery of specialization is a restatement of the routing design rather than an independent prediction.

full rationale

The derivation chain is otherwise self-contained. The pruning rule (Eq. 2) selects experts by top-k routing weights for the downstream modality, which is a selection rule, not a fitted parameter renamed as a prediction. The headline comparisons on SEN12-FLOOD (SAR) and the California wildfire dataset (SWIR) are external to the pre-training distribution and do not reduce to MAPEX's own inputs. The ben-ge-8k land-cover results use a subset of the ben-ge dataset that is used for pre-training, and the paper never states that the 8,000 evaluation samples were held out; this is a data-contamination risk that could inflate the land-cover comparisons, but it is a missing-support/validation issue, not a circular derivation. The only construction-based element is the expert-specialization evidence in Figure 6, which partly confirms the routing design rather than independently demonstrating emergent modality-specificity. Because the central pruning-performance claims retain independent empirical content, the overall circularity score is low.

Assumptions & free parameters 5 free parameters · 5 assumptions · 3 invented entities

The central claims rest on standard self-supervised pre-training assumptions plus a set of hyperparameters (dropout, top-k) tuned on the downstream evaluation set. No new physical entities are introduced; the only invented objects are learned model components.

free parameters (5)
  • modality_dropout_rate = 0.5
    Selected based on k-NN accuracy on ben-ge-8k (Table 5), which is also the downstream evaluation set; a post hoc choice that affects all main results.
  • top_k_experts = 2
    Selected based on downstream accuracy vs k on ben-ge-8k (Figure 5); used for pruning in all main experiments.
  • load_balancing_weight_alpha = 0.01
    Chosen to yield near-uniform expert utilization during pre-training; not swept.
  • masked_token_ratio = 0.75
    Standard MAE masking ratio carried over from He et al. 2022; not specific to this paper.
  • pre_training_epochs = 50
    Reported in implementation details; no evidence of convergence or tuning.
assumptions (5)
  • domain assumption Masked autoencoding pre-training transfers to downstream classification and segmentation tasks.
    The paper uses MAE as the self-supervised objective and assumes the resulting representations are useful for downstream tasks, following SatMAE and Scale-MAE.
  • domain assumption The ben-ge dataset is representative of the target remote sensing modalities and tasks.
    The paper uses ben-ge for pre-training and evaluation, assuming its six modality groups cover the relevant sensor types and land-cover classes.
  • domain assumption Pruning experts by pre-training routing weights preserves the most useful experts for a modality.
    Eq. (2) keeps the top-k experts by routing probability; the paper assumes these are the best sub-networks, validated only empirically on the same benchmark used for hyperparameter selection.
  • domain assumption Modality dropout during pre-training improves robustness to missing modalities at test time.
    Setting entire modalities to zero in a batch is assumed to help single-modality deployment; Table 5 supports this only on the evaluation set.
  • domain assumption A shared expert that processes all modalities captures common features and helps modality experts specialize.
    The shared expert is added on the hypothesis that shared features exist; Table 6 shows mixed benefit, so this design choice is not independently justified.
invented entities (3)
  • Learnable [MODALITY] token
    purpose: Input to the routing module to condition expert selection on modality identity.
    A learned vector per modality; its efficacy is shown only via in-paper ablations (Table 7), with no external handle.
  • End-of-modality token
    purpose: Injected before attention to mark modality transitions in the token sequence.
    Architectural component with no independent validation.
  • Shared expert
    purpose: A permanently activated expert processing tokens from all modalities to capture common features.
    Ablated in Table 6 with mixed results; no external evidence.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MAPEX: Modality-Aware Pruning of Experts for Remote Sensing Foundation Models." pith.science (2026). https://pith.science/paper/WKWVHKKF

@misc{pith2026250707527,
  author       = {Pith},
  title        = {Pith review of: MAPEX: Modality-Aware Pruning of Experts for Remote Sensing Foundation Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WKWVHKKF}},
  note         = {Machine review of arXiv:2507.07527}
}
read the original abstract

Remote sensing data is commonly used for tasks such as flood mapping, wildfire detection, or land-use studies. For each task, scientists carefully choose appropriate modalities or leverage data from purpose-built instruments. Recent work on remote sensing foundation models pre-trains computer vision models on large amounts of remote sensing data. These large-scale models tend to focus on specific modalities, often optical RGB or multispectral data. For many important applications, this introduces a mismatch between the application modalities and the pre-training data. Moreover, the large size of foundation models makes them expensive and difficult to fine-tune on typically small datasets for each task. We address this mismatch with MAPEX, a remote sensing foundation model based on mixture-of-modality experts. MAPEX is pre-trained on multi-modal remote sensing data with a novel modality-conditioned token routing mechanism that elicits modality-specific experts. To apply the model on a specific task, we propose a modality aware pruning technique, which only retains experts specialized for the task modalities. This yields efficient modality-specific models while simplifying fine-tuning and deployment for the modalities of interest. We experimentally validate MAPEX on diverse remote sensing datasets and show strong performance compared to fully supervised training and state-of-the-art remote sensing foundation models. Code is available at https://github.com/HSG-AIML/MAPEX.

Figures

Figures reproduced from arXiv: 2507.07527 by the authors.

Figure 1
Figure 1. Overview of MAPEX. A multi-modal foundation model [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the pre-training and downstream application process. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Task specific modalities: Sentinel-1 bands provide valuable infor￾mation for detecting water bodies in the presence of clouds. Similarly, SWIR bands are more effective than RGB for identifying burned areas. For example, SAR (Sentinel￾1) data is more informative than RGB data for detecting water in cloudy weather, while the SWIR bands of Sentinel-2 provide better information than RGB data for detecting burned areas. … view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Downstream accuracy (%) for different expert numbers [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Downstream accuracy with different numbers of retained [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Expert←→modality specialization on the ben-ge-8k dataset [32]. Each row corresponds to the performance of MAPEX pruned for a specific modality and applied across different modal￾ities. The best and worst specialized models for each input modal￾ity are highlighted. 7 […
Figure 7
Figure 7. Figure 7: Reconstruction examples from mixture-of-modality ex [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

52 extracted references · 37 canonical work pages

  1. [1]

    Flood Detection with SAR: A re- view of Techniques and Datasets

    Donato Amitrano, Gerardo Di Martino, Alessio Di Simone, and Pasquale Imperatore. Flood Detection with SAR: A re- view of Techniques and Datasets. Remote Sensing, 16(4): 656, 2024. 2

    work page 2024

  2. [2]

    Multimodal Machine Learning: A Survey and Tax- onomy

    Tadas Baltru ˇsaitis, Chaitanya Ahuja, and Louis-Philippe Morency. Multimodal Machine Learning: A Survey and Tax- onomy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(2):423–443, 2018. 2

    work page 2018

  3. [3]

    On the Opportunities and Risks of Foundation Models.arXiv preprint arXiv:2108.07258, 2021

    Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Alt- man, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the Opportunities and Risks of Foundation Models.arXiv preprint arXiv:2108.07258, 2021. 2

    arXiv 2021

  4. [4]

    Language Models are Few-shot Learners

    Tom B Brown. Language Models are Few-shot Learners. arXiv preprint arXiv:2005.14165, 2020. 2

    arXiv 2005

  5. [5]

    CAL FIRE Incidents

    California Department of Forestry and Fire Protection. CAL FIRE Incidents. https : / / www . fire . ca . gov / incidents, . Accessed: 2024-11. 5

    work page 2024

  6. [6]

    Cali- fornia Fire Perimeters (all)

    California Department of Forestry and Fire Protection. Cali- fornia Fire Perimeters (all). https://catalog.data. gov/dataset/california- fire- perimeters- all-b3436, . Accessed: 2024-11. 5

    work page 2024

  7. [7]

    MV-MOE: A Visual Mixture-of-Experts Model for Optical-SAR Image Match- ing

    Jingyi Cao, Yanan You, and Jun Liu. MV-MOE: A Visual Mixture-of-Experts Model for Optical-SAR Image Match- ing. In IGARSS 2024-2024 IEEE International Geoscience and Remote Sensing Symposium , pages 9676–9679. IEEE,

    work page 2024

  8. [8]

    Big Data for Remote Sensing: Challenges and Opportunities

    Mingmin Chi, Antonio Plaza, Jon Atli Benediktsson, Zhongyi Sun, Jinsheng Shen, and Yangyong Zhu. Big Data for Remote Sensing: Challenges and Opportunities. Pro- ceedings of the IEEE, 104(11):2207–2219, 2016. 1

    work page 2016

Show all 52 references
  1. [9]

    Com- parison of Burn Severity Assessments using Differenced Normalized Burn Ratio and Ground Data

    Allison E Cocke, Peter Z Ful ´e, and Joseph E Crouse. Com- parison of Burn Severity Assessments using Differenced Normalized Burn Ratio and Ground Data. International Journal of Wildland Fire, 14(2):189–198, 2005. 2, 5

    2005

  2. [10]

    Burke, D

    Yezhen Cong, Samarth Khanna, Chenlin Meng, Patrick Liu, Erik Rozi, Yutong He, M. Burke, D. Lobell, and Stefano Er- mon. SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery. ArXiv, abs/2207.08051,

    arXiv

  3. [11]

    An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale.arXiv preprint arXiv:2010.11929, 2020

    Alexey Dosovitskiy. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale.arXiv preprint arXiv:2010.11929, 2020. 3

    2010 arXiv

  4. [12]

    Sentinel-2: ESA’s optical high-resolution mission for GMES operational services

    Matthias Drusch, Umberto Del Bello, S ´ebastien Carlier, Olivier Colin, Veronica Fernandez, Ferran Gascon, Bianca Hoersch, Claudia Isola, Paolo Laberinti, Philippe Martimort, et al. Sentinel-2: ESA’s optical high-resolution mission for GMES operational services. Remote sensing...

    2012

  5. [13]

    Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity

    William Fedus, Barret Zoph, and Noam Shazeer. Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity. Journal of Machine Learning Research, 23(120):1–39, 2022. 2

    2022

  6. [14]

    Normalized burn ratio (NBR)

    Geospatial Data Platform. Normalized burn ratio (NBR). https : / / docs . up42 . com / help / spectral - indexes/nbr. Accessed: 2024-11. 5

    2024

  7. [15]

    WV-Net: A Foundation Model for SAR WV-mode Satellite Imagery Trained using Contrastive Self-supervised Learning on 10 Million Images

    Yannik Glaser, Justin E Stopa, Linnea M Wolniewicz, Ralph Foster, Doug Vandemark, Alexis Mouche, Bertrand Chapron, and Peter Sadowski. WV-Net: A Foundation Model for SAR WV-mode Satellite Imagery Trained using Contrastive Self-supervised Learning on 10 Million Images. arXiv pr...

    2024 arXiv

  8. [16]

    Skysense: A Multi-modal Remote Sensing Foundation Model Towards Universal Interpreta- tion for Earth Observation Imagery

    Xin Guo, Jiangwei Lao, Bo Dang, Yingying Zhang, Lei Yu, Lixiang Ru, Liheng Zhong, Ziyuan Huang, Kang Wu, Dingxiang Hu, et al. Skysense: A Multi-modal Remote Sensing Foundation Model Towards Universal Interpreta- tion for Earth Observation Imagery. In Proceedings of the IEEE/CV...

    2024

  9. [17]

    Multisensory Geospatial Models via Cross-Sensor Pre- training

    Boran Han, Shuai Zhang, Xingjian Shi, and Markus Reich- stein. Multisensory Geospatial Models via Cross-Sensor Pre- training. 2

  10. [18]

    Sparse multimodal vision transformer for weakly supervised seman- tic segmentation

    Joelle Hanna, Michael Mommert, and Damian Borth. Sparse multimodal vision transformer for weakly supervised seman- tic segmentation. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 2145–2154, 2023. 1

    2023

  11. [19]

    Masked Autoencoders are Scal- able Vision Learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll´ar, and Ross Girshick. Masked Autoencoders are Scal- able Vision Learners. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 16000–16009, 2022. 3

    2022

  12. [20]

    Distilling the Knowledge in a Neural Net- work

    Geoffrey Hinton. Distilling the Knowledge in a Neural Net- work. arXiv preprint arXiv:1503.02531, 2015. 2

    2015 arXiv

  13. [21]

    SpectralGPT: Spectral Remote Sensing Foun- dation Model

    Danfeng Hong, Bing Zhang, Xuyang Li, Yuxuan Li, Chenyu Li, Jing Yao, Naoto Yokoya, Hao Li, Pedram Ghamisi, Xiup- ing Jia, et al. SpectralGPT: Spectral Remote Sensing Foun- dation Model. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024. 2, 3

    2024

  14. [22]

    LoRA: Low-rank Adaptation of Large Language Models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank Adaptation of Large Language Models. arXiv preprint arXiv:2106.09685, 2021. 2

    2021 arXiv

  15. [23]

    Damex: Dataset-aware mixture-of-experts for visual understanding of mixture-of-datasets

    Yash Jain, Harkirat Singh Behl, Zsolt Kira, and Vib- hav Vineet. Damex: Dataset-aware mixture-of-experts for visual understanding of mixture-of-datasets. ArXiv, abs/2311.04894, 2023. 2 9

    2023 arXiv

  16. [24]

    Scaling Laws for Neural Language Models

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling Laws for Neural Language Models. arXiv preprint arXiv:2001.08361,

    2001 arXiv

  17. [25]

    Segment Any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment Any- thing. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 4015–4026, 2023. 2

    2023

  18. [26]

    Multimodal Foundation Models: From Specialists to General-purpose Assistants

    Chunyuan Li, Zhe Gan, Zhengyuan Yang, Jianwei Yang, Linjie Li, Lijuan Wang, Jianfeng Gao, et al. Multimodal Foundation Models: From Specialists to General-purpose Assistants. Foundations and Trends® in Computer Graphics and Vision, 16(1-2):1–214, 2024. 2

    2024

  19. [27]

    Remote Sensing and Image Interpretation

    Thomas Lillesand, Ralph W Kiefer, and Jonathan Chipman. Remote Sensing and Image Interpretation . John Wiley & Sons, 2015. 2

    2015

  20. [28]

    RS-MoE: Mixture of Experts for Remote Sensing Image Captioning and Vi- sual Question Answering

    Hui Lin, Danfeng Hong, Shuhang Ge, Chuyao Luo, Kai Jiang, Hao Jin, and Congcong Wen. RS-MoE: Mixture of Experts for Remote Sensing Image Captioning and Vi- sual Question Answering. arXiv preprint arXiv:2411.01595,

    arXiv

  21. [29]

    MoMa: Efficient Early-fusion Pre-training with Mixture of Modality-aware Experts

    Xi Victoria Lin, Akshat Shrivastava, Liang Luo, Srinivasan Iyer, Mike Lewis, Gargi Gosh, Luke Zettlemoyer, and Ar- men Aghajanyan. MoMa: Efficient Early-fusion Pre-training with Mixture of Modality-aware Experts. arXiv preprint arXiv:2407.21770, 2024. 2

    2024 arXiv

  22. [30]

    Decoupled Weight De- cay Regularization

    Ilya Loshchilov and Frank Hutter. Decoupled Weight De- cay Regularization. In International Conference on Learning Representations, 2017. 7, 8

    2017

  23. [31]

    AI Foundation Models in Re- mote Sensing: A Survey

    Siqi Lu, Junlin Guo, James R Zimmer-Dauphinee, Jordan M Nieusma, Xiao Wang, Parker VanValkenburgh, Steven A Wernke, and Yuankai Huo. AI Foundation Models in Re- mote Sensing: A Survey. arXiv preprint arXiv:2408.03464,

    arXiv

  24. [32]

    Ben-ge: Extending BigEarthNet with geographical and environmen- tal data

    Michael Mommert, Nicolas Kesseli, Jo ¨elle Hanna, Linus Scheibenreif, Damian Borth, and Beg ¨um Demir. Ben-ge: Extending BigEarthNet with geographical and environmen- tal data. In IGARSS 2023-2023 IEEE International Geo- science and Remote Sensing Symposium , pages 1016–1019. ...

    2023

  25. [34]

    Belongie, Christian Igel, and Nico Lang

    Vishal Nedungadi, Ankit Kariryaa, Stefan Oehmcke, Serge J. Belongie, Christian Igel, and Nico Lang. Mmearth: Explor- ing multi-modal pretext tasks for geospatial representation learning. ArXiv, abs/2405.02771, 2024. 3

    2024 arXiv

  26. [35]

    Learning Transferable Visual Models from Natural Language Super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning Transferable Visual Models from Natural Language Super- vision. In International Conference on Machine Learning ,...

    2021

  27. [36]

    Sen12-flood: a SAR and Multispectral Dataset for Flood Detection

    C Rambour, N Audebert, E Koeniguer, B Le Saux, M Cru- cianu, and M Datcu. Sen12-flood: a SAR and Multispectral Dataset for Flood Detection. IEEE: Piscataway, NJ, USA ,

  28. [37]

    Scale-MAE: A Scale- Aware Masked Autoencoder for Multiscale Geospatial Rep- resentation Learning

    Colorado Reed, Ritwik Gupta, Shufan Li, Sara Brockman, Christopher Funk, Brian Clipp, Salvatore Candido, Matthew Uyttendaele, and Trevor Darrell. Scale-MAE: A Scale- Aware Masked Autoencoder for Multiscale Geospatial Rep- resentation Learning. 2023 IEEE/CVF International Confe...

    2023

  29. [38]

    Scaling Vision with Sparse Mix- ture of Experts

    Carlos Riquelme, Joan Puigcerver, Basil Mustafa, Maxim Neumann, Rodolphe Jenatton, Andr ´e Susano Pinto, Daniel Keysers, and Neil Houlsby. Scaling Vision with Sparse Mix- ture of Experts. Advances in Neural Information Processing Systems, 34:8583–8595, 2021. 2

    2021

  30. [39]

    U-net: Convolutional networks for biomedical image segmentation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. ArXiv, abs/1505.04597, 2015. 6

    2015 arXiv

  31. [40]

    Self-supervised Vision Transformers for Land-cover Segmentation and Classification

    Linus Scheibenreif, Jo ¨elle Hanna, Michael Mommert, and Damian Borth. Self-supervised Vision Transformers for Land-cover Segmentation and Classification. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 1422–1431, 2022. 3

    2022

  32. [41]

    Outrageously Large Neural Networks: The Sparsely-gated Mixture-of-Experts Layer

    Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. Outrageously Large Neural Networks: The Sparsely-gated Mixture-of-Experts Layer. arXiv preprint arXiv:1701.06538, 2017. 2, 4

    2017 arXiv

  33. [42]

    Neural Net Pruning-Why and How

    Sietsma and Dow. Neural Net Pruning-Why and How. In IEEE 1988 international conference on neural networks , pages 325–333. IEEE, 1988. 2

    1988

  34. [43]

    Ap- plications of Remote Sensing in Precision Agriculture: A Review

    Rajendra P Sishodia, Ram L Ray, and Sudhir K Singh. Ap- plications of Remote Sensing in Precision Agriculture: A Review. Remote sensing, 12(19):3136, 2020. 2

    2020

  35. [44]

    Gencer Sumbul, Arne De Wall, Tristan Kreuziger, Filipe Marcelino, Hugo Costa, Pedro Benevides, Mario Caetano, Beg¨um Demir, and V olker Markl. BigEarthNet-MM: A large-scale, multimodal, multilabel benchmark archive for remote sensing image classification and retrieval.IEEE Geo...

    2021

  36. [45]

    Remote Sensing Plat- forms and Sensors: A Survey

    Charles Toth and Grzegorz J ´o´zk´ow. Remote Sensing Plat- forms and Sensors: A Survey. ISPRS Journal of Photogram- metry and Remote Sensing, 115:22–36, 2016. 2

    2016

  37. [46]

    Attention is all you need

    A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017. 3, 4

    2017

  38. [47]

    HyperSIGMA: Hyperspectral Intelligence Comprehen- sion Foundation Model

    Di Wang, Meiqi Hu, Yao Jin, Yuchun Miao, Jiaqi Yang, Yichu Xu, Xiaolei Qin, Jiaqi Ma, Lingyu Sun, Chenxing Li, et al. HyperSIGMA: Hyperspectral Intelligence Comprehen- sion Foundation Model. arXiv preprint arXiv:2406.11519 ,

    arXiv

  39. [48]

    Foundation Models for Remote Sensing and Earth Observation: A Sur- vey

    Aoran Xiao, Weihao Xuan, Junjue Wang, Jiaxing Huang, Dacheng Tao, Shijian Lu, and Naoto Yokoya. Foundation Models for Remote Sensing and Earth Observation: A Sur- vey. arXiv preprint arXiv:2410.16602, 2024. 1

    2024 arXiv

  40. [49]

    Stewart, Joelle Hanna, Damian Borth, Ioannis Papoutsis, Bertrand Le Saux, Gustau Camps-Valls, and Xiao Xiang Zhu

    Zhitong Xiong, Yi Wang, Fahong Zhang, Adam J. Stewart, Joelle Hanna, Damian Borth, Ioannis Papoutsis, Bertrand Le Saux, Gustau Camps-Valls, and Xiao Xiang Zhu. Neural 10 plasticity-inspired multimodal foundation model for earth observation. 2024. 3

    2024

  41. [50]

    RingMo-SAM: A Foundation Model for Segment Any- thing in Multimodal Remote-sensing Images

    Zhiyuan Yan, Junxi Li, Xuexue Li, Ruixue Zhou, Wenkai Zhang, Yingchao Feng, Wenhui Diao, Kun Fu, and Xian Sun. RingMo-SAM: A Foundation Model for Segment Any- thing in Multimodal Remote-sensing Images. IEEE Trans- actions on Geoscience and Remote Sensing , 61:1–16, 2023. 2, 3

    2023

  42. [51]

    Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources

    Xiao Xiang Zhu, Devis Tuia, Lichao Mou, Gui-Song Xia, Liangpei Zhang, Feng Xu, and Friedrich Fraundorfer. Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources. IEEE Geoscience and Remote Sensing Magazine, 5(4):8–36, 2017. 1 11 MAPEX: Modality-Aware Pru...

    2017

  43. [52]

    In each k-shot experiment, we randomly select k samples for every class from the training set

    Few-shot Learning We perform few-shot experiments to evaluate the label ef- ficiency of our method and compare its performance with other approaches. In each k-shot experiment, we randomly select k samples for every class from the training set. We keep the same experimental se...

  44. [53]

    Multimodal Downstream Evaluation Since combining multiple modalities is often beneficial for some downstream tasks, we evaluate the multimodal capa- bilities of MAPEX by testing its performance on different data modalities: RGB, SAR, and their combination (RGB + SAR) from the ...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.