AIFS-CRPS: Ensemble forecasting using a model trained with a loss function based on the Continuous Ranked Probability Score
Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel pith:UE2F3S5Hrecord.jsonopen to challenge →
read the original abstract
Over the last three decades, ensemble forecasts have become an integral part of forecasting the weather. They provide users with more complete information than single forecasts as they permit to estimate the probability of weather events by representing the sources of uncertainties and accounting for the day-to-day variability of error growth in the atmosphere. This paper presents a novel approach to obtain a weather forecast model for ensemble forecasting with machine-learning. AIFS-CRPS is a variant of the Artificial Intelligence Forecasting System (AIFS) developed at ECMWF. Its loss function is based on a proper score, the Continuous Ranked Probability Score (CRPS). For the loss, the almost fair CRPS is introduced because it approximately removes the bias in the score due to finite ensemble size yet avoids a degeneracy of the fair CRPS. The trained model is stochastic and can generate as many exchangeable members as desired and computationally feasible in inference. For medium-range forecasts AIFS-CRPS outperforms the physics-based Integrated Forecasting System (IFS) ensemble for the majority of variables and lead times. For subseasonal forecasts, AIFS-CRPS outperforms the IFS ensemble before calibration and is competitive with the IFS ensemble when forecasts are evaluated as anomalies to remove the influence of model biases.
This paper has not been read by Pith yet.
Forward citations
Cited by 16 Pith papers
-
Disentangling the effects of sea surface temperature and CO$_2$ in global machine learned weather-climate emulators
Trains ACE emulator on independent SST-CO2 variations plus energy constraint to improve accuracy in decoupled climate forcing scenarios.
-
The physics of AI weather models
AI weather models may simulate the atmosphere via particle positions in latent space whose updates follow gradient flow on a learned free energy functional rather than conventional physical equations.
-
Uncertainty-aware Machine Learning Interatomic Potentials via Learned Functional Perturbations
Learned functional perturbations plus CRPS training convert deterministic ML interatomic potentials into probabilistic ones, improving CRPS by 19-32% on N-body benchmarks and uncertainty-error correlation from 0.75 to...
-
Examining Fast Radiatively Driven Responses Using Machine-Learning Weather Emulators
Historically trained ML weather emulators quantify fast precipitation changes from CO2 perturbations and produce results that agree with Earth System Models.
-
Decision-Aware Training for Sample-Based Generative Models
Augments the energy score objective for sample-based generative models with a differentiable decision loss that is itself a proper scoring rule, yielding targeted improvements on cost-sensitive regions in synthetic an...
-
Probabilistic storyline attribution using machine learning
Distributional autoencoders trained on climate model simulations model full conditional distributions of European temperature fields to enable probabilistic storyline attribution, illustrated by higher intensities and...
-
AdaWeather: Adaptively Mixing Probabilistic Weather Forecasts with Logarithmic Regret
AdaWeather adaptively mixes probabilistic weather forecasts and achieves logarithmic regret relative to the best static mixture of experts in hindsight.
-
RealBench: Benchmarking Data-Driven Numerical Weather Forecasting Under Operational Conditions and Extreme Event Challenges
RealBench is a benchmark for data-driven weather forecasting that enforces operational conditions via a 2025 OOD test set, operational analysis, in-situ observations, and event-specific extreme metrics to expose gaps ...
-
Uncertainty-aware Machine Learning Interatomic Potentials via Learned Functional Perturbations
Learned functional perturbations convert deterministic ML interatomic potentials to probabilistic models trained with CRPS, improving uncertainty calibration over Bayesian baselines on N-body and silica benchmarks.
-
Enhancing AI and Dynamical Subseasonal Forecasts with Probabilistic Bias Correction
Probabilistic bias correction doubles AI subseasonal forecast skill and wins a 2025 international competition by correcting biases in ECMWF models for pressure, temperature, and precipitation.
-
U-Cast: A Surprisingly Simple and Efficient Frontier Probabilistic AI Weather Forecaster
A standard U-Net with MAE pre-training followed by short CRPS fine-tuning via Monte Carlo Dropout matches or exceeds GenCast and IFS ENS probabilistic skill at 1.5° resolution while cutting training compute and infere...
-
Generative 3D Gaussian Splatting for Arbitrary-ResolutionAtmospheric Downscaling and Forecasting
A generative 3D Gaussian splatting model with scale-aware attention enables unified arbitrary-resolution forecasting and downscaling of 87 atmospheric variables.
-
EnScale: Temporally-consistent multivariate generative downscaling via proper scoring rules
EnScale emulates high-resolution regional climate model outputs from global circulation models for multiple variables using a two-step generative process with sparse local stochastic layers and energy score optimizati...
-
Reliability of Probabilistic Emulation of Physical Systems
CRPS-trained ensembles achieve better uncertainty reliability and speed than latent generative models for probabilistic emulation of 2D physical systems.
-
Pointwise is Pointless? A Multimodal Ablation Study for Precipitation Nowcasting with Graph Neural Networks
A multimodal GNN ablation for Nordic precipitation nowcasting shows sparse point observations improve station and onset scores while NWP and CRPS losses improve radar-grid performance, indicating local and field skill...
-
Performance Evaluation of GraphCast for Medium-Range Weather Forecasting over Brazil
GraphCast shows regime-dependent skill versus ECMWF HRES in Brazil, underperforming on winter baroclinic systems in medium range but gaining in extended range and summer moisture transport.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.