Pith. sign in

REVIEW 5 major objections 7 minor 33 references

Utilizing Strategic Pre-training to Reduce Overfitting: Baguan -- A Pre-trained Weather Forecasting Model

T0 review · 5 major / 7 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A Siamese masked-autoencoder pre-training task, tuned to the right difficulty, gives a plain vision-transformer weather model lower error than Pangu-Weather and IFS.

desk verdict A clean internal ablation of Siamese-MAE pretraining for weather forecasting, wrapped in headline claims against Pangu/IFS that need stronger baselines and multi-year evaluation. read the letter →

arxiv 2505.13873 v1 pith:2KAFCYQQ submitted 2025-05-20 cs.LG cs.AI

Peisong Niu , Ziqing Ma , Tian Zhou , Weiqi Chen , Lefei Shen , Rong Jin , Liang Sun This is my paper · ORCID
classification cs.LGcs.AI
keywords weatherforecastingpre-trainingSiamesemaskedautoencoderoverfittingvisiontransformerlocalitybiasmedium-rangeself-supervisedlearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Weather reanalysis data spans only a few decades, so data-driven forecast models tend to overfit by learning arbitrary global correlations; this paper claims that a properly chosen pre-training task can inject locality bias and suppress that overfitting without changing the model architecture. The chosen task is Siamese masked-autoencoder pre-training — reconstructing a future frame with 75% of its patches masked, aided by an unmasked current frame — whose difficulty sits between vanilla MAE and no pre-training. The resulting model, Baguan, is a plain vision transformer that reports an average 10.1% RMSE improvement over its from-scratch twin (vanilla MAE pre-training: 4.5%), lower RMSE than Pangu-Weather for 75.2% of key variables with a 4.7% average improvement, and a 23.2% average improvement over IFS. The paper's case matters because pre-training is model-agnostic: if it really controls overfitting by tuning task difficulty, any vision-transformer-based weather model could adopt it, and the same encoder transfers to subseasonal and kilometer-scale regional forecasting.

What carries the argument

The load-bearing object is the Siamese masked-image-modeling pre-training task: a weight-shared encoder processes an unmasked frame $X_{t_0}$ alongside a frame $X_{t_0+\Delta t}$ with 75% of patches masked, and the cross-self decoder reconstructs the masked frame by letting its tokens attend to the source frame. Its defining property is task difficulty, formalized as $D_{\text{task}} \propto 1/(\lambda_t(1-r_t)+\lambda_{t-1}(1-r_{t-1}))$: vanilla MAE sets $r_{t-1}=1$ (the past frame carries no information), while Siamese MAE sets $r_{t-1}=0$, placing the task between the too-easy MAE and the hardest case of no pre-training. The masking ratio $r$ is the difficulty dial (0.75 chosen for stability; 0.95 slightly better but loss-spiky). The theoretical support is an explicitly simplified linear-regression analogue: a denoising filter $M^*=\Sigma(\Sigma+\gamma I)^{-1}$ shrinks small-eigenvalue directions, and the paper assumes transformer attention maps behave like that covariance matrix, with the top-k% spectral energy of attention scores as the measurable signature of overfitting control. The three-stage training pipeline (pre-train, fixed-lead fine-tune, iterative autoregressive fine-tune) and the pruned lead-time ensemble strategy carry the operational performance.

What would settle it

Re-run the published baselines on the same ERA5 test data with identical pressure levels, variables, and normalization, and extend the evaluation across multiple years (2018–2022) under a standardized protocol; if Baguan's average RMSE advantage over Pangu-Weather and IFS falls below 4.7% and 23.2% or reverses, the headline claim fails. The companion check is the overfitting story itself: if the Siamese-MAE-pretrained Baguan's average 10.1% advantage over its from-scratch twin shrinks or disappears on the multi-year test window, then pre-training is not conferring the claimed generalization benefit.

Watch

Extended reading notes

Core claim

Baguan is a vision-transformer weather model whose encoder is pre-trained by a Siamese masked autoencoder: the atmospheric state at $t_0$ is fed unmasked while the state at $t_0+\Delta t$ has 75% of its patches masked with Gaussian noise, and a cross-self decoder must reconstruct the masked frame with the two frames sharing one encoder. The paper's central claim is that this task sits at the right difficulty level and is what introduces locality bias into the attention maps, which controls overfitting; the masking ratio is the difficulty dial, with $r=0.95$ performing best but being unstable, so $r=0.75$ is chosen. After fixed-lead-time fine-tuning (1, 6, 24 hours) and autoregressive iterative fine-tuning, Baguan reports lower RMSE than Pangu-Weather for 75.2% of key variables (average 4.7%) on the 2018 test set, consistently beats IFS (average 23.2%), and the pre-training stage alone delivers an average 10.1% RMSE improvement over the same model trained without pre-training, versus 4.5% for vanilla MAE. The causal story is supported by a linear-regression analysis in which denoising pre-training prunes the small-eigenvalue subspace of the covariance matrix, improving the generalization bound from $O(d^{1/4}/n^{1/2})$ to $O(1/\sqrt{n})$, and by attention evidence that Siamese MAE activates 0.25% of tokens versus 1.61% for MAE. The paper is explicit that it did not compare against GNN- or local-attention-based systems such as GraphCast, FengWu, and FuXi because it lacks training codes for a fair comparison, and that the S2S results come from a companion paper.

Load-bearing premise

The load-bearing premise is that the published Pangu-Weather and IFS scores, evaluated on a single test year (2018) without re-running either baseline, are a fair and representative basis for the claimed 4.7% and 23.2% improvements; if those published numbers used different variable sets, pressure levels, or normalization, or if 2018 was atypically easy, the headline margins overstate Baguan's real skill advantage.

Editorial extensions

If this is right

  • Pre-training becomes an architecture-agnostic remedy for overfitting: any vision-transformer-based weather model that currently skips an independent pre-training stage could add Siamese MAE without redesigning for locality.
  • The benefit scales with capacity and resolution: pre-training's contribution grows from +4.20% to +10.1% as models get larger at 5.625°, and moving to 0.25° inputs cuts validation MSE by 38.8–74.0% relative to coarser grids.
  • The pre-trained encoder is a transferable foundation: fine-tuned Baguan reports S2S skill above ECMWF-S2S (via the companion optimization approach) and a 57.8% average RMSE improvement over EC-IFS on 0.05° regional forecasting, including variables absent from pre-training.
  • Mixing lead-time combinations such as [6,6,6,6] and [24] with a pruning rule adds roughly 2% accuracy at longer leads, directly attacking the error accumulation of autoregressive rollout.
  • The train-validation loss gap shrinks with pre-training (3.6e−4 versus 4.0e−4), and attention energy concentrates in leading eigenvectors — two measurable signs the paper offers for overfitting control.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If locality bias is the active ingredient, grafting Siamese MAE pre-training onto models that already build in locality — Swin-based or GNN-based systems — should compound the gain; the paper names exactly this combination as future work, making it the most direct test of the mechanism.
  • The masking-ratio pattern (0.95 best but unstable, 0.75 chosen) implies a difficulty sweet spot; an untested extension is an annealing schedule that raises the mask ratio during pre-training to reach the better optimum without the observed loss spikes.
  • The companion S2S result implies the pre-trained encoder retains skill in low signal-to-noise regimes where the paper argues overfitting is worst; a direct test is whether Siamese-MAE pre-training alone, without the companion's specialized optimization, already improves S2S skill over from-scratch training.
  • The attention-energy diagnostic could double as a general overfitting monitor for weather models — a model whose attention stays diffuse on validation data is likely memorizing — an application the paper does not itself develop.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 7 minor

Summary. The manuscript studies self-supervised pre-training for data-driven global weather forecasting. It proposes Baguan, a ViT-based model pre-trained with a Siamese masked autoencoder in which the first frame is unmasked and the second frame is masked at ratio r, then fine-tuned for fixed lead times and iteratively fine-tuned for autoregressive rollout. The main claims are: (i) pre-training with a task of intermediate difficulty introduces a locality bias that regularizes attention and reduces overfitting, supported by a controlled comparison of Siamese MAE, MAE, and no pre-training; (ii) Baguan outperforms Pangu-Weather and IFS on 0.25-degree ERA5 forecasts; and (iii) the pre-trained model transfers to S2S and regional forecasting. The paper also includes a linear-regression analysis intended to explain pre-training as a spectral regularizer.

Significance. If substantiated, the core idea that a model-agnostic pre-training stage can inject locality bias and reduce overfitting in a plain ViT weather model is genuinely useful, and it addresses an under-explored axis relative to architectural locality in Swin/GNN designs. The controlled Siamese-MAE versus MAE versus no-pre-training ablation in Section 4.2.1, the masking-ratio sweep in Section 4.2.4, and the model/data scaling analyses in Section 4.5 are valuable and go beyond a single benchmark table. The theoretical regularization analysis in Section 3.5 and Appendix A.12 is a concrete attempt at a mechanism, and the paper states it will release code and checkpoints, which would help reproducibility. The main weakness is that the headline comparisons against Pangu-Weather and IFS rest on a single test year and on published baseline numbers that are not re-run under the paper's protocol; the overfitting evidence is also thinner than the claims require.

major comments (5)
  1. [Section 4.3 / Fig. 5] The headline comparison against Pangu-Weather and IFS is not a controlled evaluation. The test set is a single year (2018, at 00 and 12 UTC), and neither baseline is re-run under the paper's protocol; there is no evidence that the published Pangu/IFS values use the same variable list (Table 3), pressure levels, normalization, latitude weighting, or ACC climatology as Eqs. (9)-(12). Any mismatch changes per-variable RMSE directly, and a single test year cannot protect against an atypical circulation regime. Please re-run Pangu (and, if possible, IFS/AIFS) under the Baguan protocol or report results through a maintained benchmark such as WeatherBench2, evaluate over multiple years, and report per-initialization spreads or confidence intervals before claiming the 4.7% and 23.2% improvements.
  2. [Section 4.3] The paper explicitly excludes FengWu, FuXi, and GraphCast, saying 'we lack training codes for these algorithms'; this weakens the benchmark claim because the strongest available reference results are omitted, and for GraphCast public code exists. If the goal is to demonstrate the value of pre-training rather than to claim a competitive SOTA position, the claims should be reworded to 'compared with published Pangu/IFS numbers' rather than implying a current SOTA ranking, or the missing baselines should be added.
  3. [Section 4.2.1] The evidence for the core overfitting claim is thin. The reported train-validation loss gap after pre-training (3.6e-4) versus without (4.0e-4) is computed over the last 50 training steps with no variance estimate, no number of seeds, and no significance test; the absolute difference is small relative to the quantities involved. Please report full learning curves, multiple seeds, and a statistical comparison. The attention-map and spectrum analyses in Sections 4.2.2-4.2.3 are qualitative and depend on the stated assumption that attention maps behave like covariance matrices; they should be presented as supporting intuition rather than as direct evidence.
  4. [Section 4.4.1] The S2S result is claimed as a contribution ('achieves SOTA performance in S2S forecasting, surpassing ECMWF-S2S'), but the full experiment is not in this manuscript; it refers to companion paper [10] and to Fig. 6 whose 'detailed results' are also deferred to [10]. A reader cannot verify the data, baselines, metrics, or statistical significance from this paper. Either include the complete S2S protocol and results here, or clearly mark the result as external work and remove it from the contribution list.
  5. [Appendix A.2, Eq. (11)] The ACC formula as written is not the standard latitude-weighted ACC: the denominator is sqrt(Σ L f'^2 g'^2), whereas the correct definition is sqrt((Σ L f'^2)(Σ L g'^2)). If Eq. (11) is what was implemented, the ACC curves in Figs. 5 and 10 are not standard ACC; if not, the equation must be corrected. Since ACC is one of the two headline metrics, this needs to be fixed and clarified.
minor comments (7)
  1. [Throughout] Please fix typos: 'exmaple' in Section 3.4, 'flexibity' in the introduction, 'Reigional' in the Table 2 caption, and 'Groud Truth' in Figs. 8 and 12.
  2. [Section 3.3.1, Eq. (5)] The quantities λ_t and λ_{t-1} in Eq. (5) are not defined, and since the relation is only a proportionality, the 'formally defined' task difficulty cannot be computed from the text; please define the λs or state explicitly that Eq. (5) is a schematic.
  3. [Fig. 3] The right panel caption says the RMSE is 'relative to mask_ratio=1.0', while the text says the results are presented relative to the w/o pre-train version; please make the normalization consistent.
  4. [Section 2 / Table 1] The 'Impact of Pre-training' column in Table 1 uses vague categories such as Positive, Negative, and Mildly Positive; please define the criterion and cite the specific experiments behind each entry.
  5. [Fig. 1(a)] The train/validation curves in Fig. 1(a) lack axis labels and units, which makes the Gap_train/Gap_val comparison unverifiable.
  6. [Appendix A.12] After the sentence 'The resulting linear regression model w2 is given by', the displayed equation is labelled w1; this is a typo that should be corrected.
  7. [Section 4.1.2 / Reproducibility] The paper says it intends to open-source the code and checkpoint but provides no repository link; please add one in the final version.

Circularity Check

1 steps flagged · score 4.0 of 10

The medium-range forecasting and Siamese-MAE ablation are self-contained; the S2S 'SOTA' claim is evidence-delegated to a co-authored companion paper, a load-bearing self-citation that prevents a fully clean circularity finding.

  1. self citation load bearing [Section 4.4.1 ('Baguan for S2S') and Fig. 6]
    "By fine-tuning Baguan with a novel multi-stage optimization approach, [10] demonstrates an impressive achievement in the S2S task. Their results outperform SOTA NWP systems(ECMWF-S2S), as illustrated in Fig. 6. ... The detailed results can be found in [10]."

    Contribution (3) claims that 'Baguan achieves SOTA performance in S2S forecasting, surpassing ECMWF-S2S,' but this claim is not demonstrated by an experiment reported in this paper. The curves in Fig. 6 are presented, yet their detailed results are explicitly deferred to [10], a companion preprint sharing four co-authors with the present paper (Tian Zhou, Liang Sun, Rong Jin, plus Yizhen Guo, Wanyi Jiang, and Bo Wu). The load-bearing evidence for the S2S success therefore reduces to a self-citation that is not independently verified or machine-checked here. The medium-range and pre-training-ablation results are computed in this paper and are not circular, so the score is moderate rather than high.

full rationale

The core result—that Siamese MAE pre-training mitigates overfitting and improves 6-hour forecasts—is supported by a controlled three-way ablation (Siamese MAE vs MAE vs no pre-training) computed in this paper (Fig. 3, Sec. 4.2.1), so it does not reduce by construction to a fitted parameter or to its own definition. The linear-regression regularization analysis (Sec. 3.5, Appendix A.12) is an independent mathematical argument under stated assumptions, and the attention-spectrum experiment is an interpretive analogy rather than a circular derivation. The comparison with Pangu-Weather and IFS (Sec. 4.3) relies on published curves and a single test year (2018) without re-running either baseline; this is a benchmark-fairness and robustness risk, not a circularity under the hard rules. The task-difficulty definition, Eq. (5), is the paper's own definition but is used only to index an external response variable (RMSE), so it is not self-definitional in a circular way. The one genuine circularity-adjacent step is the S2S claim: Fig. 6's supporting detail is explicitly deferred to [10], a companion paper sharing four co-authors with this one, and no independent verification is supplied in the present text. That makes the 'SOTA S2S' portion partially self-citation-load-bearing, while the central weather-forecasting derivation remains self-contained. The paper also concedes in Sec. 4.3 that current SOTA models (FengWu, FuXi, GraphCast) are excluded, which is a benchmark-coverage limitation, not evidence of circularity.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

Baguan is a model architecture, not a new physical entity; no new particles, forces, dimensions, or conserved quantities are proposed. The 'locality bias' is a property of learned attention, not an independent entity. The free parameters are the masking ratio, which is tuned via ablation and carries the main mechanism, and the undisclosed variable loss weights. The axioms list the theoretical and benchmark assumptions that the claims rest on.

free parameters (2)
  • Pre-training masking ratio r = 0.75 (ablation optimum 0.95 at 1.40625 degrees)
    Section 4.2.4: the authors choose r=0.75 for training stability even though r=0.95 gives the best RMSE in the ablation. This directly sets the 'task difficulty' that the paper argues is the key mechanism.
  • Variable-specific loss weights omega(v) = Not disclosed
    Equations (13) and (14) use pressure-weighted variable weights omega(v), normalized to sum to 1, but the actual values are not listed. These weights shape the training objective and could affect the reported gains.
assumptions (5)
  • ad hoc to paper Attention maps of a ViT behave like the covariance matrix in linear regression, so eigen-spectrum pruning arguments transfer.
    Section 4.2.2 and Section 3.5: the theoretical explanation of pre-training-as-regularization depends on this analogy. No empirical validation links linear-regression eigen-structure to transformer attention.
  • domain assumption The ERA5 1979-2015 / 2016-2017 / 2018 split and published Pangu/IFS scores are directly comparable to Baguan's setup.
    Section 4.3: the comparison uses published Pangu-Weather and IFS numbers without re-running them. The paper assumes equivalent variable sets, pressure levels, normalization, and that 2018 skill is representative.
  • domain assumption Denoising-based pre-training can be modeled as perfect knowledge of M* = Sigma (Sigma + gamma I)^{-1} with infinite unlabeled data.
    Appendix A.12: the claimed O(1/sqrt(n)) bound assumes an infinite-data oracle for the denoising matrix M*, which is not what the finite-sample Siamese MAE stage provides.
  • domain assumption The target regression vector w* lies in the top-K eigen-subspace of the input covariance matrix, and the spectrum follows a power law lambda_k = R / sqrt(k).
    Appendix A.12: both assumptions are needed for the dimension-independent O(1/sqrt(n)) bound. The paper does not verify either assumption for weather data.
  • standard math Matrix Chernoff and vector concentration inequalities apply as stated.
    Appendix A.12: these are standard tools for the error bounds, though the stated conditions (R, lambda_K) are not checked against the actual weather data distribution.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Utilizing Strategic Pre-training to Reduce Overfitting: Baguan -- A Pre-trained Weather Forecasting Model." pith.science (2026). https://pith.science/paper/2KAFCYQQ

@misc{pith2026250513873,
  author       = {Pith},
  title        = {Pith review of: Utilizing Strategic Pre-training to Reduce Overfitting: Baguan -- A Pre-trained Weather Forecasting Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2KAFCYQQ}},
  note         = {Machine review of arXiv:2505.13873}
}
read the original abstract

Weather forecasting has long posed a significant challenge for humanity. While recent AI-based models have surpassed traditional numerical weather prediction (NWP) methods in global forecasting tasks, overfitting remains a critical issue due to the limited availability of real-world weather data spanning only a few decades. Unlike fields like computer vision or natural language processing, where data abundance can mitigate overfitting, weather forecasting demands innovative strategies to address this challenge with existing data. In this paper, we explore pre-training methods for weather forecasting, finding that selecting an appropriately challenging pre-training task introduces locality bias, effectively mitigating overfitting and enhancing performance. We introduce Baguan, a novel data-driven model for medium-range weather forecasting, built on a Siamese Autoencoder pre-trained in a self-supervised manner and fine-tuned for different lead times. Experimental results show that Baguan outperforms traditional methods, delivering more accurate forecasts. Additionally, the pre-trained Baguan demonstrates robust overfitting control and excels in downstream tasks, such as subseasonal-to-seasonal (S2S) modeling and regional forecasting, after fine-tuning.

Figures

Figures reproduced from arXiv: 2505.13873 by the authors.

Figure 1
Figure 1. A brief review. (a) Pre-training can effectively alleviate overfitting. (b) A proper task difficulty leads to better [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of Baguan’s architecture and its three training stages. The process begins with the aggregation of the [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. The total spectrum energy of top k% attention scores. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figures from the paper (7 more)
Figure 5
Figure 5. Figure 5: Comparison of global latitude-weighted RMSE and ACC against forecast lead time for Baguan (green), Pangu-Weather (orange) and IFS (blue). Key variables analyzed include T2M, U10, T500, U500, Z500 and T850. More detailed results can be found in Appendix A.6. Train: Fuxi…
Figure 6
Figure 6. Figure 6: (a) The train loss and valid ACC of Baguan-S, Baguan￾P and FuXi (swin-based model) on S2S task. *-S represents train from sratch and *-P represents pre-training. (b-c) The TCC and PCC examined for Baguan-S2S and ECMWF-S2S models in forecasting T2M and Z500. *-15 and *-…
Figure 7
Figure 7. Figure 7: (Left): the scaling analysis of model. We present an analysis of GFlops against test MSE for a series of models varying in size from 10M to 1300M, indicated by the size of the circles. Models trained at resolutions of 5.625◦ , 1.40625◦ , and 0.25◦ are depicted in orang…
Figure 9
Figure 9. Figure 9: Comparison of global latitude-weighted RMSE of [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Results of 1.40625◦ and 5.625◦ for Baguan and ClimaX. Layer 0 Layer 7 Layer 15 ^ŝĂŵĞƐĞ D MAE dƌĂŝŶĨƌŽŵ ^ĐƌĂƚĐŚ [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]
Figure 11
Figure 11. Figure 11: Detailed attention map of Siamese MAE, MAE and the model trained from scratch. [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]
Figure 12
Figure 12. Figure 12: Detailed visualization of Baguan and Pangu-Weather. [PITH_FULL_IMAGE:figures/full_fig_p016_12.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

33 extracted references · 9 canonical work pages

  1. [10]

    Yizhen Guo, Tian Zhou, Wanyi Jiang, Bo Wu, Liang Sun, and Rong Jin. 2024. Maximizing the Impact of Deep Learning on Subseasonal-to-Seasonal Climate Forecasting: The Essential Role of Optimization. arXiv preprint arXiv:2411.16728 (2024)

    arXiv 2024

  2. [1]

    Ido Amos, Jonathan Berant, and Ankit Gupta. 2024. Never Train from Scratch: Fair Comparison of Long-Sequence Models Requires Data-Driven Priors. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024 . OpenReview.net. https://openreview.net/forum?id= PdaPky8MUn

    work page 2024

  3. [2]

    Kaifeng Bi, Lingxi Xie, Hengheng Zhang, Xin Chen, Xiaotao Gu, and Qi Tian. 2023. Accurate medium-range global weather forecasting with 3D neural networks. Nature 619, 7970 (2023), 533–538

    2023

  4. [3]

    Bruinsma, Ana Lucic, Megan Stanley, Johannes Brandstetter, Patrick Garvan, Maik Riechert, Jonathan A

    Cristian Bodnar, Wessel P. Bruinsma, Ana Lucic, Megan Stanley, Johannes Brandstetter, Patrick Garvan, Maik Riechert, Jonathan A. Weyn, Haiyu Dong, Anna Vaughan, Jayesh K. Gupta, Kit Thambiratnam, Alex Archibald, Eliza- beth Heider, Max Welling, Richard E. Turner, and Paris Perdikaris. 2024. Au- rora: A Foundation Model of the Atmosphere. ArXiv abs/2405.13...

    arXiv 2024

  5. [4]

    Kan Chen, Tao Han, Junchao Gong, Lei Bai, Fenghua Ling, Jingyao Luo, Xi Chen, Lei Ma, Tianning Zhang, Rui Su, Yuanzheng Ci, Bin Li, Xiaokang Yang, and Wanli Ouyang. 2023. FengWu: Pushing the Skillful Global Medium-range Weather Forecast beyond 10 Days Lead. ArXiv abs/2304.02948 (2023). https: //api.semanticscholar.org/CorpusID:257985330

    arXiv 2023

  6. [5]

    Lei Chen, Xiaohui Zhong, Feng jun Zhang, Yuan Cheng, Yinghui Xu, Yuan Qi, and Hao Li. 2023. FuXi: a cascade machine learning forecasting system for 15-day global weather forecast. npj Climate and Atmospheric Science 6 (2023), 1–11. https://api.semanticscholar.org/CorpusID:259224665

    work page 2023

  7. [6]

    Lei Chen, Xiaohui Zhong, Feng Zhang, Yuan Cheng, Yinghui Xu, Yuan Qi, and Hao Li. 2023. FuXi: a cascade machine learning forecasting system for 15-day global weather forecast. npj Climate and Atmospheric Science 6, 1 (2023), 190. https://doi.org/10.1038/s41612-023-00512-1

    doi:10.1038/s41612-023-00512-1 2023

  8. [7]

    Xinlei Chen and Kaiming He. 2020. Exploring Simple Siamese Representation Learning. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion (CVPR) (2020), 15745–15753. https://api.semanticscholar.org/CorpusID: 227118869

    work page 2020

Show all 33 references
  1. [8]

    Minjong Cheon, Yo-Hwan Choi, Seon-Yu Kang, Yumi Choi, Jeong-Gil Lee, and Daehyun Kang. 2024. Karina: An efficient deep learning model for global weather forecast. arXiv preprint arXiv:2403.10555 (2024)

    2024 arXiv

  2. [9]

    Jean Coiffier. 2011. Fundamentals of numerical weather prediction . Cambridge University Press

    2011

  3. [11]

    Agrim Gupta, Jiajun Wu, Jia Deng, and Fei-Fei Li. 2023. Siamese masked autoen- coders. Advances in Neural Information Processing Systems36 (2023), 40676–40693

    2023

  4. [12]

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick

  5. [13]

    Girshick

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross B. Girshick. 2022. Masked Autoencoders Are Scalable Vision Learners. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022. IEEE, 15979–15988...

    2022

  6. [14]

    Hans Hersbach, Bill Bell, Paul Berrisford, Shoji Hirahara, András Horányi, Joaquín Muñoz-Sabater, Julien Nicolas, Carole Peubey, Raluca Radu, Dinand Schepers, et al. 2020. The ERA5 global reanalysis. Quarterly Journal of the Royal Meteoro- logical Society 146, 730 (2020), 1999–2049

    2020

  7. [15]

    Thorsten Kurth, Shashank Subramanian, Peter Harrington, Jaideep Pathak, Morteza Mardani, David Hall, Andrea Miele, Karthik Kashinath, and Anima Anandkumar. 2023. FourCastNet: Accelerating Global High-Resolution Weather Forecasting Using Adaptive Fourier Neural Operators. In Pr...

    2023

  8. [16]

    Remi Lam, Alvaro Sanchez-Gonzalez, Matthew Willson, Peter Wirnsberger, Meire Fortunato, Ferran Alet, Suman Ravuri, Timo Ewalds, Zach Eaton- Rosen, Weihua Hu, Alexander Merose, Stephan Hoyer, George Holland, Oriol Vinyals, Jacklynn Stott, Alexander Pritzel, Shakir Mohamed, and ...

  9. [17]

    Simon Lang, Mihai Alexe, Matthew Chantry, Jesper Dramsch, Florian Pin- ault, Baudouin Raoult, Mariana C. A. Clare, Christian Lessig, Michael Maier- Gerber, Linus Magnusson, Zied Ben Bouallègue, Ana Prieto Nemesio, Peter D. Dueben, Andrew Brown, Florian Pappenberger, and Floren...

    2024 arXiv

  10. [18]

    Xin Man, Chenghong Zhang, Jin Feng, Changyu Li, and Jie Shao. 2023. W-mae: Pre-trained weather model with masked autoencoder for multi-variable weather forecasting. arXiv preprint arXiv:2304.08754 (2023)

    2023 arXiv

  11. [19]

    Brenowitz, Yair Cohen, Jaideep Pathak, Chieh-Yu Chen, Cheng-Chin Liu, Arash Vahdat, Karthik Kashinath, Jan Kautz, and Mike Pritchard

    Morteza Mardani, Noah D. Brenowitz, Yair Cohen, Jaideep Pathak, Chieh-Yu Chen, Cheng-Chin Liu, Arash Vahdat, Karthik Kashinath, Jan Kautz, and Mike Pritchard

  12. [20]

    Juan Nathaniel, Yongquan Qu, Tung Nguyen, Sungduk Yu, Julius Busecke, Aditya Grover, and Pierre Gentine. 2024. Chaosbench: A multi-channel, physics- based benchmark for subseasonal-to-seasonal climate prediction. arXiv preprint arXiv:2402.00712 (2024)

    2024 arXiv

  13. [21]

    Gupta, and Aditya Grover

    Tung Nguyen, Johannes Brandstetter, Ashish Kapoor, Jayesh K. Gupta, and Aditya Grover. 2023. ClimaX: A foundation model for weather and climate. In Interna- tional Conference on Machine Learning. https://api.semanticscholar.org/CorpusID: 256231457

    2023

  14. [22]

    CoRR abs/2309.15214 (2023)

    Generative Residual Diffusion Modeling for Km-scale Atmospheric Down- scaling. CoRR abs/2309.15214 (2023). https://doi.org/10.48550/ARXIV.2309.15214 arXiv:2309.15214

    2023 doi arXiv

  15. [23]

    Peebles and Saining Xie

    William S. Peebles and Saining Xie. 2022. Scalable Diffusion Models with Trans- formers. 2023 IEEE/CVF International Conference on Computer Vision (ICCV) (2022), 4172–4182. https://api.semanticscholar.org/CorpusID:254854389

    2022

  16. [24]

    Courville

    Ethan Perez, Florian Strub, Harm de Vries, Vincent Dumoulin, and Aaron C. Courville. 2018. FiLM: Visual Reasoning with a General Conditioning Layer. In AAAI

    2018

  17. [25]

    Foster, and Aditya Grover

    Tung Nguyen, Rohan Shah, Hritik Bansal, Troy Arcomano, Sandeep Madireddy, Romit Maulik, Veerabhadra Kotamarthi, Ian T. Foster, and Aditya Grover. 2023. Scaling transformer neural networks for skillful and reliable medium-range weather forecasting. CoRR abs/2312.03876 (2023). h...

    2023 doi arXiv

  18. [26]

    Johannes Schmude, Sujit Roy, Will Trojak, Johannes Jakubik, Daniel Salles Civ- itarese, Shraddha Singh, Julian Kuehnert, Kumar Ankur, Aman Gupta, Christo- pher E Phillips, et al . 2024. Prithvi wxc: Foundation model for weather and climate. arXiv preprint arXiv:2409.13598 (202...

    2024 arXiv

  19. [27]

    Jimeng Shi, Azam Shirali, Bowen Jin, Sizhe Zhou, Wei Hu, Rahuul Rangaraj, Shaowen Wang, Jiawei Han, Zhaonan Wang, Upmanu Lall, Yanzhao Wu, Leonardo Bobadilla, and Giri Narasimhan. 2025. Deep Learning and Foun- dation Models for Weather Prediction: A Survey. arXiv:2501.06907 [c...

    2025 arXiv

  20. [28]

    Stephan Rasp, Stephan Hoyer, Alexander Merose, Ian Langmore, Peter Battaglia, Tyler Russel, Alvaro Sanchez-Gonzalez, Vivian Yang, Rob Carver, Shreya Agrawal, Matthew Chantry, Zied Ben Bouallegue, Peter Dueben, Carla Bromberg, Jared Sisk, Luke Barrington, Aaron Bell, and Fei Sh...

    2024 arXiv

  21. [29]

    Zhenda Xie, Zheng Zhang, Yue Cao, Yutong Lin, Jianmin Bao, Zhuliang Yao, Qi Dai, and Han Hu. 2022. SimMIM: A Simple Framework for Masked Image Modeling. In International Conference on Computer Vision and Pattern Recognition (CVPR)

    2022

  22. [30]

    Type" column categorizes variables as static (non-time-varying) properties, time-varying, single-level (surface) properties, or time-varying atmospheric properties. The

    Xiaohui Zhong, Lei Chen, Xu Fan, Wenxu Qian, Jun Liu, and Hao Li. 2024. FuXi-2.0: Advancing machine learning weather forecasting model for practical applications. CoRR abs/2409.07188 (2024). https://doi.org/10.48550/ARXIV.2409. 07188 arXiv:2409.07188 Utilizing Strategic Pre-tr...

    2024 doi arXiv

  23. [31]

    Frederic Vitart and Andrew Robertson. 2018. The sub-seasonal to seasonal prediction project (S2S) and the prediction of extreme events. npj Climate and Atmospheric Science 1 (03 2018). https://doi.org/10.1038/s41612-018-0013-0

    2018 doi

  24. [2022]

    In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 16000–16009

  25. [2023]

    Sci- ence 382, 6677 (2023), 1416–1421

    Learning skillful medium-range global weather forecasting. Sci- ence 382, 6677 (2023), 1416–1421. https://doi.org/10.1126/science.adi2336 arXiv:https://www.science.org/doi/pdf/10.1126/science.adi2336

    2023 doi

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.