Pith. sign in

REVIEW 3 major objections 6 minor 26 references

Timestep-Conditioned Transformers for Global Weather Forecasting

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A single weather model can switch its forecast timestep at inference time, balancing short-range detail against long-range stability without retraining.

desk verdict Genuinely new timestep-conditioning mechanism with honest bounds, but the abstract overstates consistency and breadth. read the letter →

arxiv 2608.06241 v1 pith:ZICRJPDC submitted 2026-08-06 cs.LG cs.OS

Sam Levang , Fran Bartolic , Ty Dickinson , Chase Dwelle , Paulius Rauba , Viktor Cikojevic This is my paper
classification cs.LGcs.OS
keywords probabilisticweatherforecastingmulti-timestepinferencetimestepconditioningadaptivelayernormalizationautoregressiverolloutstabilityanomaly-spacemodelingneighborhoodattentiontransformerensemble
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Existing machine-learning weather models commit to one autoregressive timestep at design time: short steps resolve the diurnal cycle but accumulate error over deep rollouts, while long steps are stable but lose sub-daily detail. This paper claims that the timestep can instead be a configurable inference-time input, with a single set of weights serving steps from 1 to 24 hours. The payoff of the claim is practical: one model could produce detailed short-range forecasts and stable month-long rollouts by switching its step size rather than being retrained. The authors report that mixed-timestep training also stabilizes short-step rollouts relative to single-step specialists, and that the resulting operational model is competitive with leading machine-learning weather systems while beating the benchmark operational ensemble on medium- and extended-range probabilistic skill. If correct, this turns the choice of forecast timestep from an architectural commitment into a per-forecast operational decision.

What carries the argument

The load-bearing mechanism is sequential AdaLN-Zero conditioning: a Fourier embedding of the timestep $\Delta t$ is projected, per block, into per-channel shift, scale, and gate parameters that are applied before the ensemble noise $z$ in every transformer block. This low-dimensional modulation lets one shared weight set act as a family of transition kernels $q_\theta(\cdot \mid X_t, C, \Delta t, z)$; a second mechanism, anomaly-space modeling, subtracts a lowpass-filtered climatology from temperature, geopotential, and pressure fields so the backbone learns a stable residual state. A timestep-dependent loss scaling $\sqrt{\Delta t_{\mathrm{base}}/\Delta t}$ with $\Delta t_{\mathrm{base}} = 24$ h balances the mixed-timestep training objective across step sizes.

What would settle it

Train a generalist on 6 h and 24 h steps only and then evaluate it at a held-out 18 h timestep against specialists trained at 6, 18, and 24 h; if the 18 h generalist forecast is far worse than the specialists bracketing it, the model has memorized discrete step sizes rather than learned a continuously configurable transition operator.

Watch

Extended reading notes

Core claim

The paper's central discovery is that a low-dimensional adaptive-layer-norm modulation of a shared transformer stack is sufficient to make one network behave as a family of transition operators indexed by forecast step. Each block receives a Fourier-embedded timestep $\Delta t$ and an ensemble noise vector $z$ through separate shift, scale, and gate parameters, with $\Delta t$ applied first; at inference the user supplies $\Delta t$ and the same weights run a 6 h forecast, a 24 h forecast, or a mixed schedule. The authors show that within a roughly 3–24 hour range a timestep-generalist matches or nearly matches specialists, and that mixing timesteps during training reduces mean-state drift in fine-step rollouts at the price of a slowly accumulating excess of small-scale spectral power. They find the regime is bounded: a 1 h step drifts quickly, a 24 h generalist is slightly worse than its specialist in the first week, and adding a 48 h step degrades the whole model. The claim, stated in the authors' terms, is that one weight set can cover both short-range and extended-range deployment by changing its inference schedule rather than its architecture.

Load-bearing premise

For the configurable-timestep claim to hold, a few per-channel shifts, scales, and gates applied to one shared set of transformer weights must be enough to represent the correct one-step weather transition at every supported step size; the paper's own ablations show this assumption biting by 24 h and failing at 48 h.

Editorial extensions

If this is right

  • The same weight set can run a 6 h schedule for the first two weeks and a 24 h schedule beyond, giving both diurnal detail and extended-range stability in one forecast.
  • Mixed-timestep training acts as a regularizer: 1 h and 6 h generalist rollouts accumulate error more slowly than matched specialists, even though their first-step scores are slightly worse.
  • The useful timestep range is roughly 3–24 h; 1 h steps drift to worse-than-climatology by about 10 days, while 24 h steps keep positive skill to about three months.
  • A coarse 24 h timestep alone nearly matches the skill of 6 h forecasts in the medium range, showing that rollout depth, not just step fidelity, dominates long-lead scores.
  • Selective anomaly-space modeling improves rollout stability across all timesteps, with the largest gains at 1 h and 6 h steps, without changing first-step skill.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One test the paper does not run is a held-out timestep: if a model trained only on 6 h and 24 h can forecast accurately at 12 h or 18 h, the conditioning has learned a continuum rather than a lookup table.
  • The same AdaLN pathway could plausibly condition on other global signals—climate scenario, initial-condition metadata, or user preference for sharpness versus calibration—since the mechanism is a general modulation of shared weights, not a timestep-specific layer.
  • The 24 h performance ceiling may reflect model capacity or local attention rather than an inherent limit of timestep modulation; scaling up could extend the range, but would erode the computational savings that make coarse steps attractive.
  • A fixed hybrid schedule is only one policy among many; the configurable step opens the door to adaptive schedules that pick a timestep per initialization, per lead, or per region, which the paper leaves unexplored.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces GEM-3, a probabilistic global weather forecasting model whose autoregressive timestep Δt is an explicit conditioning input, implemented through sequential AdaLN modulation of a neighborhood-attention transformer. The authors claim that a single set of trained weights can be configured at inference time to use different timesteps, balancing short-range detail against extended-range stability, and that mixed-timestep training improves rollout stability relative to timestep-specialist models. The model is evaluated as a 0.25° operational system trained on 6 h and 24 h steps against ECMWF ENS, AIFS-ENS, and FGN, plus 1° ablations over 1–24 h and 6–48 h timestep ranges, with additional diagnostics on anomaly-space modeling and spectral degradation.

Significance. If the central claim holds, the paper makes a practically valuable contribution: a single model that can serve both short-range and extended-range forecasting without retraining, with a public reforecast dataset and a lightweight 134M-parameter architecture. The evaluation is unusually thorough for a systems paper: it includes external baselines on the same initialization periods, matched specialist-generalist ablations, multiple stability diagnostics (climatology crossover, mean-state drift, spectral power), and candid discussion of failure modes. The paper is honest about the bounded timestep range and the trade-off between spectral fidelity and drift suppression. The main weakness is that the abstract and parts of the discussion state the configurable-timestep and stability claims more broadly than the evidence supports.

major comments (3)
  1. [Abstract and §5 (Figs. 7–8)] The abstract claim that mixed-timestep training 'consistently improves rollout stability relative to timestep-specialist models' is not supported by the displayed data: in Fig. 7 the 24 h generalist is worse than the 24 h specialist over the first ~7 days, and the paper's own text states that the 24 h specialist 'actually shows better performance on the first ~7 days.' The word 'consistently' should be qualified to refer to the fine-timestep (1 h and 6 h) regime, or the claim revised.
  2. [Abstract, §3.2, §5] The central claim of 'configurable timestep ... across a broad forecast horizon' is demonstrated only for timesteps present in training, and the paper's ablations bound the mechanism: Fig. 8 shows that adding 48 h degrades the generalist, and the text concedes that 'limitations of our timestep conditioning scheme begin to manifest around the 24 h mark.' No experiment evaluates an unseen timestep (e.g., 12 h interpolation) or an extrapolated step beyond the training set, so the inference-time configurability is not shown to generalize beyond the discrete trained Δt values. The abstract overreaches the evidence; either add an interpolation experiment or temper the claim.
  3. [§5, operational model evaluation] The operational 0.25° model is trained only on {6 h, 24 h} and is evaluated only at those timesteps; the broader 1–24 h and 6–48 h generalist evidence comes from 1° ablations with a different training recipe (heavier rollout weighting). The claim that the production model can be configured 'across a broad forecast horizon' at operational resolution therefore rests on an extrapolation from lower-resolution ablations. The paper should state this limitation explicitly in the abstract or provide at least one operational-resolution evaluation at an additional timestep.
minor comments (6)
  1. [Abstract] The phrase 'near-SOTA medium-range probabilistic skill' is accurate in Fig. 3 but should be reconciled with the conclusion's 'competitive with current leading ML models'; FGN outperforms GEM-3 on z500, mslp, and prate, so a reader may perceive a mismatch in emphasis.
  2. [Eq. (12)] The timestep-dependent loss scaling uses a square-root factor with Δt_base = 24 h, but no ablation or empirical justification is given for this particular functional form; a brief sensitivity check or citation would strengthen the claim that the scaling addresses gradient imbalance.
  3. [§3.1, Eq. (8)] The notation N(τ, Δt) for the number of rollout steps conflicts with the standard use of N for the normal distribution in Eq. (10); consider using a different symbol such as n_steps to avoid ambiguity.
  4. [§5, Fig. 6] The crossover-lead metric is informative, but the figure would benefit from error bars or a statement of how sensitive the crossover is to the climatological baseline definition, since the baseline is not otherwise described.
  5. [§5, Fig. 3 and Fig. 4] The verification period differs between the medium-range (242 initializations, 2025–2026) and extended-range (104 initializations, 2023–2025) figures; this should be noted in the captions or text to avoid the impression of a single unified evaluation period.
  6. [§4.1] The statement that temperature, geopotential, and pressure anomalies are chosen because 'the low-frequency seasonal cycle is closely associated with the external solar forcing' is a physical postulate; the paper does not quantify the validity of this separation, so it would help to cite or provide a diagnostic showing that the anomaly transform does not alias synoptic variability.

Circularity Check

0 steps flagged · score 2.0 of 10

No material circularity: GEM-3's configurable-timestep claim is empirically tested against matched specialists and external baselines; only a minor non-load-bearing self-citation to GEM-2 is present.

full rationale

This paper is an empirical system evaluation rather than a derivation. The central claim—that one weight set can be configured at inference time over multiple autoregressive timesteps—is established by the AdaLN conditioning mechanism (Eq. 11) and by empirical comparisons of the resulting generalist against matched timestep-specialist models (Figs. 5, 7, 8) and against external baselines (ECMWF ENS, AIFS-ENS, FGN, climatology). No fitted quantity is renamed as a prediction: CRPS values are computed on held-out initializations (242 for medium range, 104 for extended range, and 37 for ablations), and the configurable-timestep behavior is an architectural property verified by rollouts, not an identity. The only self-citation is to the authors' GEM-2 [23], and it is used for design conventions—shifted-window vs. neighborhood attention, native diagnostic outputs, spectral power CRPS loss, validation pipeline—none of which carries the load of the multi-timestep inference claim. The paper's own ablations bound the mechanism (48 h step degrades the generalist; 24 h generalist is worse than the 24 h specialist in the first week; 'limitations of our timestep conditioning scheme begin to manifest around the 24 h mark'), which is honest negative evidence rather than circular reasoning. Overstatements such as 'consistently improves rollout stability' (supported only for 1 h and 6 h in Fig. 7) and 'broad forecast horizon' (demonstrated range is ≤24 h) are empirical-evidence concerns, not circularity. Score 2 reflects the minor, non-load-bearing self-citation.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claims rest on a standard probabilistic forecasting setup (Markov factorization), an empirical architectural premise (AdaLN modulation is sufficient for multi-timestep operation), and validation conventions (ERA5 as truth). No new physical entities are introduced; anomaly-space modeling is a transform, not an entity. The main hand-chosen quantities are the loss-scaling rule, training stage lengths, and inference schedules, none of which is fitted to a benchmark but all of which affect the stability claims.

free parameters (5)
  • Timestep loss scaling exponent = sqrt(Δt_base/Δt) with Δt_base = 24 h
    Chosen to rebalance gradient magnitudes in the mixed-timestep objective (Eq. 12); the central claim that all timesteps train well depends on this hand-chosen scaling.
  • Rollout training schedule = operational: ~70k single-step steps, then ~7k rollout-4 steps; ablations: 50k base, 10k rollout-4, 2k rollout-12
    Hand-selected stage lengths; rollout-stability claims are specific to this schedule and could change with different lengths.
  • Timestep sampling weights in rollout training = preferentially weighted towards 6 h, exact distribution not reported
    Affects how much stability gain is attributed to mixed-timestep training; under-specified in Section 5.
  • Operational hybrid timestep schedule = 6 h through day 14, then 24 h
    Inference-time choice that defines the operational GEM-3 product; the medium-range versus extended-range tradeoff relies on this schedule.
  • Anomaly-space variable subset = temperature, geopotential, pressure; moisture, wind, precipitation excluded
    Selected by domain reasoning about solar forcing; the claimed stability benefit of anomaly-space modeling depends on this choice.
assumptions (5)
  • domain assumption The atmosphere induces a well-defined conditional distribution p(·|h) over future trajectories given initialization H=h (Eq. 1).
    Standard probabilistic forecasting setup; unproved but accepted for defining the learning target.
  • domain assumption First-order Markov approximation p(Xt|X<t,C)=p(Xt|Xt-1,C) (Eq. 4).
    Underlies the autoregressive rollout and the composition of per-step kernels in Eq. (8); if higher-order memory matters, the rollout may be misspecified.
  • ad hoc to paper A single low-dimensional AdaLN modulation can represent the transition operators across the supported timestep range.
    Empirical premise of the multi-timestep mechanism; the paper itself finds it begins to fail at 24 to 48 h (Fig. 8 discussion).
  • domain assumption ERA5 reanalysis is treated as ground truth for validation and as sole training target.
    Standard practice in ML weather forecasting, but any systematic ERA5 errors propagate into skill claims.
  • ad hoc to paper For temperature, geopotential, and pressure, the transition can be separated into a solar-forced quasi-static climatological part and active synoptic dynamics.
    Justifies anomaly-space modeling; stated in Discussion, not proven; moisture and wind are excluded for the same reason.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Timestep-Conditioned Transformers for Global Weather Forecasting." pith.science (2026). https://pith.science/paper/ZICRJPDC

@misc{pith2026260806241,
  author       = {Pith},
  title        = {Pith review of: Timestep-Conditioned Transformers for Global Weather Forecasting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZICRJPDC}},
  note         = {Machine review of arXiv:2608.06241}
}
read the original abstract

Existing machine-learning weather forecasting models rely on predetermined and fixed autoregressive timesteps. The choice of model timestep involves a fundamental trade-off: shorter timesteps (e.g. 1 to 6 hours) finely resolve atmospheric dynamics within the diurnal cycle but increase error accumulation for a given forecast horizon, while longer timesteps (e.g. 24 hours) reduce error accumulation but limit the usability of short-range forecasts where sub-daily predictability is high. In this work, we present GEM-3, a probabilistic global weather model that addresses this trade-off through explicit multi-timestep inference. With a single set of trained weights, the model timestep can be configured at inference time to balance predictability and usability across a broad forecast horizon. Additionally, we find that mixed-timestep training consistently improves rollout stability relative to timestep-specialist models. Under the hood, GEM-3 is a lightweight neighborhood-attention transformer with ~134M parameters on an equirectangular grid with a number of architectural advancements beyond its predecessor GEM-2. The result is a practical forecasting system that couples near-SOTA medium-range probabilistic skill, stable extended-range rollouts, efficient training and inference, and decision-relevant diagnostics.

Figures

Figures reproduced from arXiv: 2608.06241 by the authors.

Figure 1
Figure 1. Configurable timestep inference with a single model. (a) Finer timesteps of 6 h or less resolve diurnal structure but require a deeper rollout to reach a given lead, while a coarse 24 h schedule reaches it in fewer steps. The operational GEM-3 model uses a hybrid schedule, running at 6 h through day 14 and switching to 24 h for the extended range. (b) A single set of weights supports this via AdaLN conditioning of t… view at source ↗
Figure 2
Figure 2. Multi-timestep conditioning architecture. (a) The model is a stack of L transformer blocks, each receiving the same conditioning signals. (b) Inside each block, the normalized activations are modulated sequentially: first by the Fourier-embedded timestep ∆t (amber, shift/scale/gate), then by ensemble noise z (green, shift/scale/gate). This pattern applies identically to both the neighborhood self-attention and MLP b… view at source ↗
Figure 3
Figure 3. Operational model medium-range CRPS. Global CRPS over the first 15 forecast days for the operational GEM-3 model against ECMWF ENS, ECMWF AIFS-ENS, and Google FGN. Evaluated on 242 initializations from July 2025 to March 2026, the period for which AIFS-ENS forecasts are available. 1 2 3 4 5 6 0.4 0.6 0.8 CRPS t2m 1 2 3 4 5 6 0.4 0.6 0.8 mx2t 1 2 3 4 5 6 0.4 0.6 0.8 mn2t 1 2 3 4 5 6 0.4 0.6 0.8 avg_100ws 1 2 3 4 5 6 … view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Operational model extended-range CRPS. Global CRPS through 6 weeks for the operational GEM-3 model, ECMWF ENS-46, and a 30-year climatological baseline. Evaluated on 104 initializations from July 2023 to July 2025. On standard 15-day global CRPS benchmarks, GEM-3 consi…
Figure 5
Figure 5. Figure 5: Performance of 1–24 h timestep-generalist model. CRPS evaluation of a single timestep￾conditioned model evaluated at 1, 3, 6, 12, and 24 h forecast timesteps. A useful stability metric is the lead time at which a forecast becomes worse than a climatological baseline. I…
Figure 6
Figure 6. Figure 6: Degradation of the 1–24 h generalist by timestep. The lead time in days at which forecast CRPS first becomes worse than a climatological baseline, for each inference timestep of the generalist model. Faint guide lines indicate the equivalent number of autoregressive st…
Figure 7
Figure 7. Figure 7: Performance of timestep-generalist versus timestep-specialists. CRPS evaluation of forecasts run from the 1–24 h timestep-generalist model against matched timestep-specialists trained at 1, 6, and 24 h. Next, we attempt to push the upper limit of the timestep range by …
Figure 8
Figure 8. Figure 8: Performance of a 6–48 h timestep-generalist model. CRPS evaluation of forecasts run from a 6–48 h timestep-generalist model against matched 6, 24, and 48 h timestep-specialists. Anomaly-space modeling ablation. Finally, we perform a comparison of the selective anomaly-…
Figure 9
Figure 9. Figure 9: Anomaly versus state forecasting. Comparison on three core variables across the useful 1–24 h timestep range. Colors encode timestep, with solid lines for anomaly forecasting and dashed lines for state forecasting. Selective anomaly modeling most clearly improves the s…
Figure 10
Figure 10. Figure 10: Mechanisms of rollout degradation. Additional error diagnostics for the 1–24 h timestep-generalist and matched 1, 6, and 24 h specialists. (a–c) log10 ratio of forecast to ERA5 degree-averaged z500 spherical harmonic power spectra at 1, 5, and 15-day lead; values abov…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

26 extracted references · 11 canonical work pages

  1. [1]

    F. Alet, I. Price, A. El-Kadi, D. Masters, S. Markou, T. R. Andersson, J. Stott, R. Lam, M. Will- son, A. Sanchez-Gonzalez, and P. Battaglia. Skillful joint probabilistic weather forecasting from marginals.arXiv preprint arXiv:2506.10772, 2025

    arXiv 2025

  2. [2]

    Alexe, E

    M. Alexe, E. Boucher, P. Lean, E. Pinnington, P. Laloyaux, A. McNally, S. Lang, M. Chantry, C. Burrows, M. Chrust, et al. Graphdop: Towards skilful data-driven medium-range weather forecasts learnt and initialised directly from observations.arXiv preprint arXiv:2412.15687, 2024

    arXiv 2024

  3. [3]

    K. Bi, L. Xie, H. Zhang, X. Chen, X. Gu, and Q. Tian. Accurate medium-range global weather forecasting with 3d neural networks.Nature, 619:533–538, 2023

    work page 2023

  4. [4]

    Bonev, T

    B. Bonev, T. Kurth, A. Mahesh, M. Bisson, J. Kossaifi, K. Kashinath, A. Anandkumar, W. D. Collins, M. S. Pritchard, and A. Keller. Fourcastnet 3: A geometric approach to probabilistic machine-learning weather forecasting at scale.arXiv preprint arXiv:2507.12144, 2025

    arXiv 2025

  5. [5]

    K. Chen, T. Han, F. Ling, J. Gong, L. Bai, X. Wang, J.-J. Luo, B. Fei, W. Zhang, X. Chen, et al. The operational medium-range deterministic weather forecasting can be extended beyond a 10-day lead time.Communications Earth & Environment, 6(1):518, 2025

    work page 2025

  6. [6]

    L. Chen, X. Zhong, F. Zhang, Y . Cheng, Y . Xu, Y . Qi, and H. Li. Fuxi: a cascade machine learning forecasting system for 15-day global weather forecast.npj climate and atmospheric science, 6(1):190, 2023

    work page 2023

  7. [7]

    Couairon, R

    G. Couairon, R. Singh, A. Charantonis, C. Lessig, and C. Monteleoni. Archesweather & archesweathergen: a deterministic and generative model for efficient ml weather forecasting. arXiv preprint arXiv:2412.12971, 2024

    arXiv 2024

  8. [8]

    Hassani, S

    A. Hassani, S. Walton, J. Li, S. Li, and H. Shi. Neighborhood attention transformer. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6185–6194, 2023

    work page 2023

Show all 26 references
  1. [9]

    Jordan, Y

    K. Jordan, Y . Jin, V . Boza, Y . Jiacheng, F. Cesista, L. Newhouse, and J. Bernstein. Muon: An optimizer for hidden layers in neural networks, 2024. URL https://kellerjordan. github.io/posts/muon

    2024

  2. [10]

    R. Keisler. Forecasting global weather with graph neural networks.arXiv preprint arXiv:2202.07575, 2022

    2022 arXiv

  3. [11]

    Kochkov, J

    D. Kochkov, J. Yuval, I. Langmore, P. Norgaard, J. Smith, G. Mooers, M. Klöwer, J. Lottes, S. Rasp, P. Düben, S. Hatfield, P. Battaglia, A. Sanchez-Gonzalez, M. Willson, M. P. Brenner, and S. Hoyer. Neural general circulation models for weather and climate.Nature, 632(8027): 1...

    2024

  4. [12]

    Kossaifi, N

    J. Kossaifi, N. Kovachki, M. Mardani, D. Leibovici, S. Ravuri, I. Shokar, E. Calvello, M. S. Abbas, P. Harrington, A. Subramaniam, N. Brenowitz, B. Bonev, W. Byeon, K. Kreis, D. Durran, A. Vahdat, M. Pritchard, and J. Kautz. Demystifying data-driven probabilistic medium-range ...

    2026

  5. [13]

    R. Lam, A. Sanchez-Gonzalez, M. Willson, P. Wirnsberger, M. Fortunato, F. Alet, S. Ravuri, T. Ewalds, Z. Eaton-Rosen, W. Hu, et al. Learning skillful medium-range global weather forecasting.Science, 382(6677):1416–1421, 2023

    2023

  6. [14]

    S. Lang, S. Ravuri, A. Botev, E. de Bézenac, I. Ayed, K. Lenc, R. Clark, M. Khan, C. Yang, R. Franklin, et al. Aifs: Ecmwf’s data-driven forecasting system.arXiv preprint arXiv:2406.01465, 2024

    2024 arXiv

  7. [15]

    Lehmann, F

    F. Lehmann, F. Ozdemir, Y . Cheng, T. Hoefler, S. Schemm, B. Soja, and S. Mishra. Can AI weather models predict beyond two weeks? a quantitative benchmark and analysis of long rollouts.arXiv preprint arXiv:2605.30184, 2026. 15

    2026 arXiv

  8. [16]

    J. Liu, J. Su, X. Yao, Z. Jiang, G. Lai, Y . Du, Y . Qin, W. Xu, E. Lu, J. Yan, et al. Muon is scalable for llm training.arXiv preprint arXiv:2502.16982, 2025

    2025 arXiv

  9. [17]

    Nguyen, J

    T. Nguyen, J. K. Gupta, S. Chen, K. Bi, W. Xie, K. Bi, X. Li, R. Y . Chen, P. Hassanzadeh, J. Song, et al. Climax: A foundation model for weather and climate. InProceedings of the 40th International Conference on Machine Learning (ICML), volume 202, pages 11502–11521, 2023

    2023

  10. [18]

    Nguyen, T

    T. Nguyen, T. Pham, T. Arcomano, V . Kotamarthi, I. Foster, S. Madireddy, and A. Grover. Omnicast: A masked latent diffusion model for weather forecasting across time scales.arXiv preprint arXiv:2510.18707, 2025

    2025

  11. [19]

    Pathak, S

    J. Pathak, S. Subramanian, P. Harrington, S. Raja, A. Chattopadhyay, M. Mardani, T. Kurth, D. Hall, Z. Li, K. Azizzadenesheli, et al. Fourcastnet: A global data-driven high-resolution weather model using adaptive fourier neural operators.arXiv preprint arXiv:2202.11214, 2022

    2022 arXiv

  12. [20]

    Peebles and S

    W. Peebles and S. Xie. Scalable diffusion models with transformers. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 4195–4205, 2023

    2023

  13. [21]

    Price, A

    I. Price, A. Sanchez-Gonzalez, F. Alet, T. R. Andersson, A. El-Kadi, D. Masters, T. Ewalds, J. Stott, S. Mohamed, P. Battaglia, et al. Gencast: Diffusion-based ensemble forecasting for medium-range weather.arXiv preprint arXiv:2312.15796, 2023

    2023 arXiv

  14. [22]

    Price, A

    I. Price, A. Sanchez-Gonzalez, F. Alet, T. R. Andersson, A. El-Kadi, D. Masters, T. Ewalds, J. Stott, S. Mohamed, P. Battaglia, R. Lam, and M. Willson. Probabilistic weather forecasting with machine learning.Nature, 637:84–90, 2024

    2024

  15. [23]

    Rauba, V

    P. Rauba, V . Cikojevic, F. Bartolic, S. Levang, T. Dickinson, and C. Dwelle. Probabilistic transformers for joint modeling of global weather dynamics and decision-centric variables. arXiv preprint arXiv:2601.03753, 2026

    2026

  16. [24]

    Stock, T

    J. Stock, T. Arcomano, and R. Kotamarthi. Swift: An autoregressive consistency model for efficient weather forecasting.arXiv preprint arXiv:2509.25631, 2025

    2025

  17. [25]

    X. Sun, X. Zhong, X. Xu, Y . Huang, H. Li, J. D. Neelin, D. Chen, J. Feng, W. Han, L. Wu, and Y . Qi. A data-to-forecast machine learning system for global weather.Nature Communications, 16(6658):1–12, 2025

    2025

  18. [26]

    J. A. Weyn, D. R. Durran, R. Caruana, and N. Cresswell-Clay. Sub-seasonal forecasting with a large ensemble of deep-learning weather prediction models.Journal of Advances in Modeling Earth Systems, 13(2):e2021MS002502, 2021. 16

    2021

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.