REVIEW 4 major objections 6 minor 1 cited by
Deploying Geospatial Foundation Models in the Real World: Lessons from WorldCereal
T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read In the WorldCereal operational crop-mapping system, fine-tuning the Presto geospatial foundation model outperforms the deployed CatBoost baseline on random, geographic, and temporal splits, with the largest gains for underrepresented crop…
desk verdict A credible operational case study showing fine-tuned Presto beats the deployed CatBoost baseline, though the generalization claims outpace what the splits actually test. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is Presto, a lightweight transformer-based foundation model designed for pixel-level time series of Sentinel-1, Sentinel-2, digital elevation, and weather data. Pre-training gives the model transferable representations, and supervised fine-tuning adapts those representations to WorldCereal's data pipeline, which differs from Presto's pre-training data in processing level and cloud filtering. The evaluation machinery is three train/validation splits: a random split, a geographic split that removes entire countries, and a temporal split that removes the most recent label year, each testing a different form of generalization the operational system needs.
What would settle it
Train the same fine-tuned Presto and CatBoost models on the paper's data but evaluate them on 2024/2025 reference data in the actual deployment regions; if CatBoost matches or beats Presto's F1 there, the central claim that the foundation model generalizes better in this operational setting is falsified.
Extended reading notes
Core claim
The paper's central claim is that fine-tuning a pre-trained geospatial foundation model, Presto, on task-specific labels yields a model that beats the fully supervised classifier currently deployed in WorldCereal. For binary cropland classification, fine-tuned Presto reaches F1 scores of 0.861 (random split), 0.829 (geographic split), and 0.886 (temporal split), compared with 0.856, 0.810, and 0.830 for the deployed CatBoost baseline. For multiclass crop-type classification, fine-tuned Presto reaches 0.809, 0.650, and 0.686 versus 0.728, 0.563, and 0.649 for the supervised baseline, with the millet/sorghum class improving from 0.415 to 0.530 F1. A randomly initialized version of the same architecture performs worst, which the paper reads as evidence that the pre-training, not just the transformer architecture, drives the gains. The paper also finds that an additional self-supervised learning round before fine-tuning does not improve results, so the simplest adaptation—supervised fine-tuning alone—is enough in this setting.
Load-bearing premise
The paper's evidence for real-world generalization rests on assuming that a few expert-chosen held-out countries and the withheld year 2021 stand in for all the places and seasons the deployed system will actually encounter.
Editorial extensions
If this is right
- A single fine-tuned foundation model can replace continent-specific trained classifiers and improve accuracy in the same operational pipeline.
- The improvement is concentrated in underrepresented crops, so label-scarce categories benefit most from switching to a pre-trained model.
- The additional self-supervised adaptation step can be skipped when the unlabelled data matches the fine-tuning data, saving compute without losing performance.
- Because the gains persist on geographic and temporal splits, users can produce maps for new regions and years with less local retraining.
- The three-step protocol gives other operational mapping services a concrete path for testing foundation models before deployment.
Reading between the lines
- Not tested in the paper: the models are evaluated on data through 2021, while deployment targets 2024/2025; validating on 2024/2025 reference data once available would be the direct check.
- Because the held-out countries are chosen by expert judgment, a practitioner aiming at different regions should re-run the geographic split with their own target countries before relying on the generalization claim.
- The null result for extra self-supervised learning is conditional on SSL and fine-tuning data coming from the same distribution; an unlabelled dataset from a genuinely new region or season might still help, and that experiment is not run here.
- If gains scale with label scarcity, foundation models should show even larger margins on mapping tasks with fewer labels than WorldCereal's cropland task, a prediction checkable on other land-cover problems.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a three-step protocol (requirements, adaptation, empirical testing) for integrating geospatial foundation models into operational remote sensing systems, and applies it to the WorldCereal crop mapping system. Using Presto as the foundation model, the authors fine-tune it on WorldCereal's cropland and crop-type classification datasets and compare against a deployed CatBoost baseline, an untuned CatBoost on raw inputs, and a randomly initialized Presto. Experiments are run under random, geographic, and temporal splits, with per-country and per-class metrics and qualitative patch inspection. The reported results show Presto generally outperforms the baselines, particularly on underrepresented crop types, and the additional SSL step does not help. The paper concludes that foundation models can be practically integrated into operational mapping.
Significance. If the empirical claims hold, this paper provides a useful template for practitioners and a concrete case study showing that a lightweight foundation model can be competitive in an operational crop mapping pipeline. The use of multiple splits, multiple baselines, per-country and per-class analyses, and visual patch quality assessment are strengths. The paper is also transparent about resource constraints (CPU deployment) and reports MAC counts to justify model choice. However, the central generalization claims are only weakly supported by the experimental design, and the lack of uncertainty quantification limits the strength of the conclusions. The paper has a clear protocol contribution, and the negative result for SSL is a useful data point, but the statistical and external-validity gaps prevent the strong conclusions from being fully established.
major comments (4)
- [Section 3.4, Tables 1-5] No confidence intervals, standard errors, or significance tests are reported for any metric. Several headline gaps are close to noise: in Table 1 (Random split) the Deployed baseline is 0.856 and Finetuned Presto is 0.861 (Δ=0.005), and SSL + Finetuned Presto ties Finetuned Presto at 0.861; in Table 4 several per-country deltas are ≤0.009 (Italy) or negative (Mozambique, −0.001). Given that some geographic-split validation sets contain only 0.2K–0.9K samples (Table 4, e.g., Brazil 0.9K, Italy 0.6K, Mozambique 0.4K, Ethiopia 0.2K), the high-variance estimates cannot support the word 'significantly' in the Abstract. Please report bootstrap confidence intervals over points or repeated fine-tuning runs with different seeds, or at least provide the number of independent runs and a paired significance test for the main comparisons.
- [Section 3.4.1, Tables 1-2] The 'Deployed Baseline' is an existing WorldCereal CatBoost classifier, but the paper does not state whether this baseline was retrained on the same training splits (random, geographic, temporal) used for the other models. If the deployed model was trained on data that include the held-out countries or the held-out year, then the geographic and temporal comparisons against it are confounded. Please specify the training distribution for the Deployed Baseline; if it was not retrained, either retrain it under identical splits (reporting the resulting performance) or explicitly state that the comparison is against a previously trained system and adjust the interpretation of H1 and H2 accordingly.
- [Abstract and Section 3.4] The claim of 'strong spatial and temporal generalization' is not established by the reported splits. The geographic split uses a small set of expert-selected countries (Appendix B) that may not be representative of the global deployment distribution, and several have very few validation samples (Table 4, e.g., Brazil 0.9K, Italy 0.6K, Madagascar 0.5K). The temporal split holds out only 2021, which is adjacent to the training years 2017–2020, whereas the stated deployment target is global maps for 2024/2025 (Section 3.2); a one-year gap does not emulate the 3–4 year drift in seasonality, data processing, and sensor characteristics. The paper should either add an evaluation on a later year (if any labels exist beyond 2021) or soften the generalization claim to 'generalizes to the held-out countries and years tested' and explicitly list this as a limitation.
- [Section 3.4.2, Table 3] The 'Unprocessed CatBoost' baseline is trained directly on raw pixel-time series, but the paper does not report the hyperparameter budget or tuning procedure for this baseline, nor whether a comparable effort was made to optimize it as in the deployed baseline (Section 3.4.1). If the baseline is under-tuned, the reported margin of Presto over it may overstate the advantage. Please report the number of trees, learning rate, early stopping criterion, and any hyperparameter search for the CatBoost baselines; if the raw-input CatBoost is meant to be a weaker point of comparison, say so explicitly.
minor comments (6)
- [Abstract and Section 3.5] The statement 'strong spatial and temporal generalization' is repeated in the conclusion; consider qualifying it to reflect the limited scope of the evaluated splits (see Major Comment 3).
- [Section 3.4.1, Table 1] The caption does not specify whether the reported F1 is for the positive (crop) class in the binary task or a macro average; please clarify for both Tables 1 and 2.
- [Section 3.4.1, paragraph after Table 1] The sentence 'The randomly initialized Presto architecture performs worst' is not true on the temporal split in Table 1, where Finetuned Presto-Rnd (0.806) is below Deployed baseline (0.830) and Unprocessed CatBoost (0.874), but the statement should be qualified as 'performs worst overall across the three splits' to avoid a false universal.
- [Appendix C, Figure 10 caption] The caption 'SSL + Finetuned model shows a much worse result comparable to the Unprocessed CatBoost model' appears to contradict the quantitative results in Table 3, where SSL+Finetuned is the best or close to best; please reconcile the qualitative description with the numeric results.
- [Section 2.1, MAC counts] The MAC counts for Presto, Galileo-Nano, and AnySat are given without a source or a footnote; please add a reference or state that they were computed by the authors.
- [References and Section 1] The citation 'Reuel-Lamparth et al., 2025' in Section 1 appears to be labeled inconsistently with the bibliography entry 'Reuel-Lamparth, A., Hardy, A., ...' ; please check the author name formatting.
Circularity Check
No significant circularity: the paper's central claims are empirical comparisons on held-out splits, with no fitted parameter or prior result doing the work.
full rationale
The paper's central claims (H1, H2, H3) are tested by direct experiments: fine-tuning Presto on WorldCereal training splits and evaluating on random, geographic, and temporal held-out splits against CatBoost baselines. No parameter is fitted to the evaluation set, and no result is defined in terms of the conclusion being drawn. Presto's self-supervised pretraining is not label-based and does not use the downstream WorldCereal labels used for evaluation. The deployed baseline is an existing operational system, and the comparison is an apples-to-apples empirical evaluation; if anything, the deployed baseline may have seen more data, which would disadvantage the Presto models rather than inflate their reported gains. The randomly-initialized Presto control shows that pretraining, not architecture alone, drives the improvement, further insulating the causal claim from circularity. The paper does cite prior work by overlapping authors (Presto, Tseng et al. 2024; WorldCereal, Van Tricht et al. 2023), but these citations supply the model and system under test or contextual claims such as computational cost; they are not the evidence for the paper's performance numbers, which are new measurements on external reference data. The noted limitations about the geographic split (expert-selected countries) and the temporal split (one-year gap to 2021 rather than the full gap to 2024/2025) are validity or external-proxy concerns, not circularity: they concern whether the evaluation supports the operational generalization claim, not whether the claim is constructed from its own inputs. No step in the derivation chain reduces by definition or by self-citation to the paper's conclusions.
Assumptions & free parameters
free parameters (1)
- Held-out country selection
assumptions (3)
- domain assumption F1 score is the appropriate evaluation metric for the operational goals
- domain assumption The held-out countries are a valid proxy for geographic generalization in the deployment scenario
- domain assumption The temporal split (2017-2020 train, 2021 test) generalizes to 2024/2025
Cite this review
Pith. "Pith review of Deploying Geospatial Foundation Models in the Real World: Lessons from WorldCereal." pith.science (2026). https://pith.science/paper/H53W3AJ5
@misc{pith2026250800858,
author = {Pith},
title = {Pith review of: Deploying Geospatial Foundation Models in the Real World: Lessons from WorldCereal},
year = {2026},
howpublished = {\url{https://pith.science/paper/H53W3AJ5}},
note = {Machine review of arXiv:2508.00858}
}
read the original abstract
The increasing availability of geospatial foundation models has the potential to transform remote sensing applications such as land cover classification, environmental monitoring, and change detection. Despite promising benchmark results, the deployment of these models in operational settings is challenging and rare. Standardized evaluation tasks often fail to capture real-world complexities relevant for end-user adoption such as data heterogeneity, resource constraints, and application-specific requirements. This paper presents a structured approach to integrate geospatial foundation models into operational mapping systems. Our protocol has three key steps: defining application requirements, adapting the model to domain-specific data and conducting rigorous empirical testing. Using the Presto model in a case study for crop mapping, we demonstrate that fine-tuning a pre-trained model significantly improves performance over conventional supervised methods. Our results highlight the model's strong spatial and temporal generalization capabilities. Our protocol provides a replicable blueprint for practitioners and lays the groundwork for future research to operationalize foundation models in diverse remote sensing applications. Application of the protocol to the WorldCereal global crop-mapping system showcases the framework's scalability.
Figures
Figures from the paper (6 more)
Forward citations
Cited by 1 Pith paper
-
Visuo-Acoustic Hand Pose and Contact Estimation
VibeMesh combines RGB-D vision, bone-conducted acoustic signals, and a graph attention network to estimate hand pose and dense contact, outperforming vision-only baselines in occluded settings.
Reference graph
Works this paper leans on
-
[1]
Agency, T. E. S. Copernicus Digital Elevation Model (DEM) GLO-30. https://dataspace.copernicus. eu/explore-data/data-collections/ copernicus-contributing-missions/ collections-description/COP-DEM. Ac- cessed: 2025-02-28. An, X., He, W., Zou, J., Yang, G., and Zhang, H. Pretrain a remote sensing foundation model by promoting intra- instance similarity.IEEE...
work page 2025
-
[3]
URL https://arxiv. org/abs/2412.14123. Bastani, F., Wolters, P., Gupta, R., Ferdinando, J., and Kembhavi, A. Satlaspretrain: A large-scale dataset for remote sensing image understanding,
-
[5]
Beukema, P., Bastani, F., Wolters, P., Herzog, H., and Fer- dinando, J. Satellite imagery and ai: a new era in ocean conservation, from research to deployment and impact. arXiv preprint arXiv:2312.03207,
-
[6]
Boogaard, H., Schubert, J., De Wit, A., Lazebnik, J., Hutjes, R., Van der Grijn, G., et al. Agrometeorological indicators from 1979 to present derived from reanalysis.Copernicus Climate Change Service (C3S) Climate Data Store (CDS), 10,
work page 1979
-
[9]
Fuller, A., Millard, K., and Green, J
URL https://ieeexplore.ieee.org/abstract/ document/10642780/. Fuller, A., Millard, K., and Green, J. Croma: Remote sens- ing representations with contrastive radar-optical masked autoencoders.Advances in Neural Information Process- ing Systems, 36:5506–5538,
-
[10]
Garnot, V . S. F., Landrieu, L., Giordano, S., and Chehata, N. Time-space tradeoff in deep learning models for crop classification on satellite multi-spectral image time series. InIGARSS 2019-2019 IEEE International Geoscience and Remote Sensing Symposium, pp. 6247–6250. IEEE,
work page 2019
-
[12]
Jiang, W., Zhang, J., Wang, D., Zhang, Q., Wang, Z., and Du, B
URL https: //arxiv.org/abs/2312.02199. Jiang, W., Zhang, J., Wang, D., Zhang, Q., Wang, Z., and Du, B. Lemevit: Efficient vision transformer with learn- able meta tokens for remote sensing image interpreta- tion,
-
[13]
SeaMo: A Season-Aware Multimodal Foundation Model for Remote Sensing
URL https://arxiv.org/abs/ 2412.19237. Marsocci, V ., Jia, Y ., Bellier, G. L., Kerekes, D., Zeng, L., Hafner, S., Gerard, S., Brune, E., Yadav, R., Shi- bli, A., et al. Pangaea: A global and inclusive bench- mark for geospatial foundation models.arXiv preprint arXiv:2412.04204,
Show all 22 references
-
[15]
Szwarcman, D., Roy, S., Fraccaro, P., Þorsteinn Elí Gís- lason, Blumenstiel, B., Ghosal, R., de Oliveira, P
URL https://arxiv.org/abs/2309.07207. Szwarcman, D., Roy, S., Fraccaro, P., Þorsteinn Elí Gís- lason, Blumenstiel, B., Ghosal, R., de Oliveira, P. H., de Sousa Almeida, J. L., Sedona, R., Kang, Y ., Chakraborty, S., Wang, S., Gomes, C., Kumar, A., Truong, M., Godwin, D., Lee, ...
-
[16]
Tseng, G., Zvonkov, I., Nakalembe, C
URL https://arxiv.org/abs/2412.02732. Tseng, G., Zvonkov, I., Nakalembe, C. L., and Kerner, H. Cropharvest: A global dataset for crop-type classification. InThirty-fifth Conference on Neural Information Process- ing Systems Datasets and Benchmarks Track (Round 2),
-
[17]
Tseng, G., Fuller, A., Reil, M., Herzog, H., Beukema, P., Bastani, F., Green, J
URL https:// arxiv.org/abs/2304.14065. Tseng, G., Fuller, A., Reil, M., Herzog, H., Beukema, P., Bastani, F., Green, J. R., Shelhamer, E., Kerner, H., and Rolnick, D. Galileo: Learning global & local features of many remote sensing modalities. InForty-second International Conf...
-
[18]
USDA-NASS, USDA NASS Marketing and Information Services Office, Washington, DC
Published crop-specific data layer. USDA-NASS, USDA NASS Marketing and Information Services Office, Washington, DC. Accessed: 2025-02-23. Van De Kerchove, R., Zanaga, D., Keersmaecker, W., Sou- verijns, N., Wevers, J., Brockmann, C., Grosu, A., Paccini, A., Cartus, O., Santoro...
2025
-
[19]
Xiong, Z., Wang, Y ., Zhang, F., and Zhu, X
URL https://arxiv.org/ abs/2403.13430. Xiong, Z., Wang, Y ., Zhang, F., and Zhu, X. X. One for all: Toward unified foundation models for earth vision,
-
[20]
Zvonkov, I., Tseng, G., Nakalembe, C., and Kerner, H
URLhttps://arxiv.org/abs/2401.07527. Zvonkov, I., Tseng, G., Nakalembe, C., and Kerner, H. Openmapflow: A library for rapid map creation with machine learning and remote sensing data. InProceed- ings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 14655–14663,
-
[21]
A. Dataset Description Thecroplanddataset comprises sampled point data with the following key characteristics: • Total Samples:Approximately 1.3 million points, with 26% labeled as cropland. • Geographic Coverage:Data from 176 countries; no- tably, the USA, Spain, and Belgium ...
2017
-
[22]
other_crop
with a relatively even distribution (see Figure 6). • Data Sources:Aggregated from 121 sources, includ- ing prominent contributions from the USDA Crop Data Layer (USDA National Agricultural Statistics Service, 2021), the LUCAS Copernicus 2018 dataset (d’Andrimont et al., 2021)...
2021
-
[2013]
M., Bradbury, K., and Malof, J
Huang, B., Reichman, D., Collins, L. M., Bradbury, K., and Malof, J. M. Tiling and stitching segmentation output for remote sensing: Basic challenges and recommendations. arXiv preprint arXiv:1805.12219,
-
[2021]
doi: 10.5194/essd-13-1119-2021
ISSN 1866-3508. doi: 10.5194/essd-13-1119-2021. Fibaek, C., Camilleri, L., Luyts, A., Dionelis, N., and Le Saux, B. Phileo bench: Evaluating geo- spatial foundation models. InIGARSS 2024-2024 9 Deploying Geospatial Foundation Models in the Real World: Lessons from WorldCereal ...
2021 doi
-
[2022]
Lucas copernicus 2018: Earth- observation-relevant in situ data on land cover and use throughout the european union.Earth System Science Data, 13(3):1119–1133, March
d’Andrimont, R., Verhegghen, A., Meroni, M., Lemoine, G., Strobl, P., Eiselt, B., Yordanov, M., Martinez-Sanchez, L., and van der Velde, M. Lucas copernicus 2018: Earth- observation-relevant in situ data on land cover and use throughout the european union.Earth System Science ...
2018
-
[2023]
Becker-Reshef, I., Barker, B., Whitcraft, A., Oliva, P., Mob- ley, K., Justice, C., and Sahajpal, R
URL https://arxiv.org/abs/2211.15660. Becker-Reshef, I., Barker, B., Whitcraft, A., Oliva, P., Mob- ley, K., Justice, C., and Sahajpal, R. Crop type maps for operational global agricultural monitoring.Scientific Data, 10(1):172,
-
[2024]
2024.3469568
doi: 10.1109/TGRS. 2024.3469568. Astruc, G., Gonthier, N., Mallet, C., and Landrieu, L. Anysat: An earth observation model for any resolutions, scales, and modalities,
2024
-
[2025]
F., Ortiz, A., Sederholm, T., Dodhia, R., Birge, C., Richards, K., Pitcher, K., Duarte, P., and Ferres, J
Robinson, C., Nsutezo, S. F., Ortiz, A., Sederholm, T., Dodhia, R., Birge, C., Richards, K., Pitcher, K., Duarte, P., and Ferres, J. M. L. Rapid building damage assess- ment workflow: An implementation for the 2023 rolling fork, mississippi tornado event. InProceedings of the ...
2023
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.