Pith. sign in

REVIEW 3 major objections 5 minor 13 references

Stop using root-mean-square error as a precipitation target!

T0 review · 3 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Pith's one-line read This paper argues that the standard training loss for precipitation models, root-mean-square error, should be replaced by the Tweedie deviance, a likelihood-based loss that matches rainfall's zero-inflated, heavy-tailed distribution, and sh

desk verdict A genuinely useful proposal to replace MSE with Tweedie deviance for precipitation ML, let down by nowcasting claims with no error bars on a single test month. read the letter →

arxiv 2509.08369 v1 pith:7CQ5WH7J submitted 2025-09-10 physics.ao-ph

classification physics.ao-ph
keywords TweediedistributioncompoundPoisson–Gammaprecipitationlossfunctionroot-mean-squareerrornowcastingdownscalingvariance–meanpowerlaw
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Rain is not Gaussian: it is a mix of many dry periods and rare, heavy downpours, yet most machine-learning models of precipitation are trained by minimizing mean-squared error, which assumes Gaussian errors. This paper argues that the training objective itself should match the data-generating process, and proposes the Tweedie deviance, a likelihood-based loss that naturally handles zero-inflated, non-negative, heavy-tailed rainfall. The author shows that the Tweedie power parameter p can be estimated from the observed variance–mean power law and that precipitation at every practical accumulation length sits between the Poisson (p=1) and Gamma (p=2) limits, never near Gaussian. On two standard tasks, diffusion downscaling over Beijing and ConvLSTM nowcasting over Kolkata, swapping RMSE for Tweedie deviance leaves overall metrics unchanged or better while improving extreme-rainfall recall and the compounding skill of autoregressive nowcasts. The point is practical: a one-line change to the loss function aligns the model with rainfall statistics and improves exactly where Gaussian loss fails.

What carries the argument

Tweedie deviance: a likelihood-based loss from the exponential–dispersion family with variance function V(µ)=µ^p. For 1<p<2 it is the compound Poisson–Gamma distribution, and its unit deviance has a closed form. The power p is fit once from the variance–mean power law; because p is continuous, the same loss interpolates from Poisson-like (p→1) to Gamma-like (p→2) behavior. The paper uses this deviance as the training loss in place of MSE, with a softplus output to keep predictions non-negative and dispersion set to 1.

What would settle it

Estimate p separately for stratified storm regimes (monsoon versus frontal versus orographic) at one location; if the differences exceed the ordinary least-squares confidence intervals, a single p is misspecified and the deviance would need a mixture or spatially varying p. A second check: retrain the same models with p obtained from profile likelihood instead of block-OLS estimation and compare extreme recall and wet-pixel MAE.

Watch

Extended reading notes

Core claim

At the center is the unit Tweedie deviance, a closed-form likelihood-based loss from the exponential–dispersion family whose variance function is V(µ)=µ^p. For 1<p<2 it corresponds to a compound Poisson–Gamma distribution, with a point mass at zero and continuous positive amounts, which matches observed precipitation. The paper derives that RMSE is the p→0 Gaussian limit of this same family, so using RMSE imposes a Gaussian likelihood on a non-Gaussian target. The author estimates p from log Var(Y)=c+p log E[Y] over non-overlapping blocks and finds p in [1.18, ~2] across gauge, reanalysis, and satellite data, increasing with accumulation period toward the Gamma limit. Training identical down

Load-bearing premise

The paper assumes that a single spatially averaged Tweedie power p, estimated by least-squares fitting of the variance–mean relationship, correctly describes the rainfall distribution for every location and case; if the true p varies by storm type, terrain, or intensity, or if that fit is biased, the Tweedie deviance is misspecified as a likelihood and the statistical justification for the observed gains weakens.

Editorial extensions

If this is right

  • Downscaling models trained with Tweedie deviance match RMSE-trained models on overall distributional fit, wet-pixel MAE, and dry hit rate, while recalling about 60% versus 50% of 99th-percentile extremes; operational downscaling can recover extreme events without sacrificing average skill.
  • Nowcasting models trained with Tweedie deviance improve wet-pixel MAE and dry-pixel hit rate at every lead time, and the improvements grow with autoregressive rollout—about 2% MAE at t+1 growing to about 16% at t+4—so the loss choice compounds over the forecast horizon.
  • The estimated Tweedie power p rises from near Poisson at 5-minute accumulation to near Gamma for monthly totals, so the same continuous loss family adapts to any accumulation period once p is estimated from the training data.
  • Because p is not known a priori and differs between locations and datasets, users should estimate p on the target data at the target accumulation period rather than fixing it.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the author leaves implicit: because the Tweedie deviance is a proper scoring rule for the conditional mean, it could replace MSE in other zero-inflated, heavy-tailed geophysical targets such as wind gusts, cloud water, or air-quality concentrations, with p estimated per variable.
  • A testable refinement: replacing the single global p with a spatially or seasonally indexed p, or parameterizing p as part of the network, could improve conditional tails further; the paper's own single-p design leaves this open.
  • The unexplained 'weekly dip' in p may reflect real meteorological structure rather than sampling noise; if it persists, p itself could serve as a diagnostic of rainfall regime, not just a tuning parameter.
  • The same likelihood framing extends naturally to probabilistic outputs: training a distribution head with Tweedie deviance would give a calibrated predictive distribution and explicit uncertainty, which a point-loss RMSE baseline cannot provide.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper argues that RMSE is a misspecified training objective for precipitation and proposes the Tweedie deviance as a differentiable, likelihood-based replacement. Section 2 derives the unit deviance for the exponential dispersion family and recovers the Gaussian, Poisson, and Gamma limits. Section 4.1 estimates the Tweedie power p from the variance–mean power law for a UK rain gauge, ERA5, and GPM-IMERG data, showing p increases with accumulation length. Section 4.2 compares RMSE and Tweedie training of a conditional diffusion downscaler over Beijing; Section 4.3 compares RMSE and Tweedie training of a ConvLSTM nowcaster over Kolkata. The paper claims improved extreme recall in downscaling and improved wet-pixel MAE and dry hit rates in nowcasting, with gains that compound with lead time.

Significance. If the empirical gains are robust, the paper provides a practical and well-motivated alternative to RMSE, with a clean derivation and a simple recipe for estimating p. The use of identical architectures for both loss comparisons is a strength, as is the multi-dataset demonstration that p is far from zero and varies with aggregation. The paper is honest about the non-significance of some downscaling differences. The principal concern is that the central nowcasting claim rests on a one-month evaluation without uncertainty quantification, and the 'statistically justified' framing is not tested against misspecification of p.

major comments (3)
  1. [§4.3, Fig. 6] The headline claim of ~2%→16% MAE improvement and ~2%→13% dry-hit-rate improvement rests entirely on one month (August 2018) with no error bars, confidence intervals, or significance tests, even though App. A.2 defines a three-month test period (Jun–Aug 2018). Given stochastic training and strong space–time autocorrelation in nowcast errors, the t+4 differences could be sampling noise. This is the paper's strongest claim and needs a full-test-period evaluation with block-bootstrap CIs and paired significance tests (or an explicit justification for the August-only choice).
  2. [§4.2, Fig. 4; Abstract] The abstract concludes that Tweedie loss 'improves wet-pixel MAE and extreme recall' in downscaling, and the introduction repeats this. Fig. 4b shows a non-significant MAE difference (0.415 vs 0.410) and the text states this is not significant; only extreme recall (0.504 vs 0.602, Fig. 4d) reaches significance, and only for a single month and a 99th-percentile threshold. The claim should be qualified and, if possible, the analysis extended to the full test year specified in App. A.3.
  3. [§4.1, App. A.4, §5] The Tweedie power p is estimated once from the marginal variance–mean power law (blockwise OLS, averaged over pixels) and then fixed for training. The paper itself defers p-sensitivity to future work. If p is biased or varies with intensity, storm type, or orography, the training objective is not the Tweedie likelihood the authors invoke as justification for the skill gains. Please add sensitivity tests (e.g., p ± 95% CI or profile-likelihood estimate) and report the exact p used for each experiment; otherwise the 'statistically justified' framing is not supported.
minor comments (5)
  1. [§2.1, Eq. (11)] The Taylor expansion for θ is not correct as written: for p=ε, θ = μ^{1-ε}/(1-ε) = μ[1 + ε(1 - ln μ)] + O(ε²), not μ[1 - (ε/2) ln μ] + O(ε²). The leading term is unaffected, but the displayed expression should be fixed.
  2. [§4.3 and Fig. 6b] The '~16%' improvement at t+4 is ambiguous: (0.669−0.578)/0.669 ≈ 14%, while (0.669−0.578)/0.578 ≈ 16%. The standard relative-to-RMSE convention gives ~14%; please align the abstract, main text, and figure caption on one definition and verify the numbers.
  3. [App. A.4] The actual values of p used for the Beijing hourly downscaling and Kolkata half-hourly nowcasting experiments are not reported. Since the method's reproducibility depends on this, please list the estimated p and its confidence interval for each training setup.
  4. [§4.2] The text says differences are 'bootstrap tested' but no details of the bootstrap procedure (resampling unit, number of replications, CI construction) are given. Please describe the procedure or cite a standard method.
  5. [§4.1, Fig. 2] The systematic 'weekly dip' in p is left unexplained. If it cannot be analyzed, at least label it as a possible finite-sample artifact and remove it from the bullet-point conclusions, or provide a targeted test.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the Tweedie loss is derived from first-principles distribution theory and the reported skill gains are measured on held-out data, not by construction.

full rationale

The paper's central derivation chain is self-contained. The Tweedie deviance is derived from the exponential dispersion family via the variance function V(μ)=μ^p (Eqs. 1–10), so the loss formula itself does not assume any empirical precipitation result. The index p is estimated from training data using the variance–mean power law (App. A.4) and then fixed for validation and testing, but the loss is used only as a training objective. The evaluation metrics that support the headline claims—wet-pixel MAE, dry hit rate, and extreme recall—are computed on held-out test periods (August 2018 for both downscaling and nowcasting) and are not algebraically equal to the Tweedie deviance or to the p-estimation procedure. Optimizing Tweedie deviance does not by construction improve MAE, hit rate, or recall; those are empirical outcomes. The extreme-recall improvement is bootstrap-tested, while the nowcasting results lack confidence intervals, but that is a statistical-evidence concern, not circularity. The only self-citation (Hunt and Turner, 2024) appears as an example of MSE being a default loss in AI weather models, and it is not load-bearing for any derived claim. No uniqueness theorem, no ansatz smuggled via citation, and no renaming of a known result as a new derivation are present. Therefore the paper does not exhibit significant circularity.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the Tweedie distributional assumption (where p is a fitted parameter), on the variance-mean power law as an estimator, and on a single-month evaluation. No new entities are introduced.

free parameters (4)
  • Tweedie power p = 1.18 (Reading 5-min) to ~1.85 (Beijing daily); exact values for downscaling and nowcasting tasks are estimated from trai
    Fitted via OLS to log Var vs log E on non-overlapping temporal blocks (App. A.4). The choice of block length (30 days for hourly/daily, etc.) is ad hoc, and the paper averages p across pixels, making it a free parameter rather than a derived constant.
  • Intercept c of the variance-mean power law = Not reported
    Part of the OLS fit; not used directly in the loss except through p, but it is a fitted parameter.
  • Scaling factor s = 99th percentile of training target pixels
    Inputs and targets are divided by s; this is a normalization choice, not fit to the loss, but it interacts with the deviance scaling (multiplying deviance by a constant).
  • Evaluation threshold tau = 0.1 mm
    Wet-pixel and dry hit rate use tau=0.1 mm; extreme recall uses 99th/99.9th percentile of truth. These choices affect the reported skill differences.
assumptions (5)
  • standard math Tweedie family is an exponential dispersion family with variance function V(mu)=mu^p and unit deviance given by Eq. 10
    Used to derive the loss; standard result from exponential dispersion family theory (e.g., Dunn, 2004).
  • standard math The cumulant function identities in Eqs. 7-9 (E[Y]=kappa'(theta), theta'=1/V) hold
    Standard properties of exponential dispersion families; used to derive the deviance integral.
  • domain assumption Precipitation at the target accumulation follows a Tweedie distribution with a single p, or at least has a variance-mean power law with that p
    The loss is justified as likelihood-based deviance for a Tweedie distribution. The paper estimates p from the variance-mean power law but does not test whether the full distribution (zero-inflation proportion, shape of the continuous part) matches Tweedie, nor whether p is spatially and conditionally constant.
  • domain assumption The variance-mean power law holds on the block-averaged statistics, and OLS on log-transformed values gives an unbiased estimate of p
    App. A.4 fits log Var = c + p log E via OLS across blocks. This ignores measurement error in both log E and log Var, and depends on the choice of block length (30 days etc.).
  • ad hoc to paper Evaluation on a single test month (August 2018) is representative of the models' skill
    Both skill experiments are evaluated only on August 2018 (the monsoon month), despite longer test periods in the appendix. The paper does not show the skill difference in other seasons, so the claim of 'consistent skill gains' is supported by one month.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Stop using root-mean-square error as a precipitation target!." pith.science (2026). https://pith.science/paper/7CQ5WH7J

@misc{pith2026250908369,
  author       = {Pith},
  title        = {Pith review of: Stop using root-mean-square error as a precipitation target!},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7CQ5WH7J}},
  note         = {Machine review of arXiv:2509.08369}
}
abstract

Root-mean-square error (RMSE) remains the default training loss for data-driven precipitation models, despite precipitation being semi-continuous, zero-inflated, strictly non-negative, and heavy-tailed. This Gaussian-implied objective misspecifies the data-generating process because it tolerates negative predictions, underpenalises rare heavy events, and ignores the mass at zero. We propose replacing RMSE with the Tweedie deviance, a likelihood-based and differentiable loss from the exponential--dispersion family with variance function $V(\mu)=\mu^p$. For $1<p<2$ it yields a compound Poisson--Gamma distribution with a point mass at zero and a continuous density for $y>0$, matching observed precipitation characteristics. We (i) estimate $p$ from the variance--mean power law and show that precipitation across temporal aggregations is far from Gaussian, with the Tweedie power $p$ increasing with accumulation length towards a Gamma limit; and (ii) demonstrate consistent skill gains when training deep data-driven models with Tweedie deviance in place of RMSE. In diffusion-model downscaling over Beijing, Tweedie loss improves wet-pixel MAE and extreme recall ($\sim0.60$ vs $0.50$ at the 99th percentile). In ConvLSTM nowcasting over Kolkata, Tweedie loss yields improved wet-pixel MAE and dry-pixel hit rates, with improvements that compound autoregressively with lead time (for MAE, $\sim2%$ at $t{+}1$ growing to $\sim16%$ at $t{+}4$). Because the Tweedie deviance is continuous in $p$, it adapts smoothly across scales, offering a statistically justified, practical replacement for RMSE in precipitation-based learning tasks.

Figures

Figures reproduced from arXiv: 2509.08369 by the authors.

Figure 1
Figure 1. Tweedie family PDFs for fixed mean and dispersion. Curves show fY (y) for µ = 1 and ϕ = 0.5 across powers p ∈ {0, 1, 1.25, 1.5, 1.75, 2, 3, 4}. Special cases: p = 0 is N (µ, ϕ) (solid line), p = 1 is Poisson(µ) (probability mass function shown as stems), p = 2 is Gamma, p = 3 is inverse-Gaussian. For 1 < p < 2 the distribution is compound Poisson–Gamma with a point mass at zero (stem height exp{−µ 2−p /[ϕ(2 − p)]}) … view at source ↗
Figure 2
Figure 2. Estimated Tweedie power parameter p as a function of accumulation period for four datasets: Reading gauge (red), ERA5 at Beijing (grey), ERA5 at Kolkata (dashed grey) and IMERG at Kolkata (blue). For each dataset and accumulation period, totals are formed by summing raw values over the period and p is estimated from the power–law relation log Var(Y ) = c + p log E[Y ] using ordinary least squares on non-overlapping … view at source ↗
Figure 3
Figure 3. Performance of Tweedie vs RMSE diffusion downscaling for three randomly selected test cases. Column 1 shows the actual precipitation in ERA5; column 2 shows the ERA5 precipitation coarsened to 1°, i.e., the model input; and column 3 shows their difference. Column 4 and 5 show the predictions from the Tweedie and RMSE models respectively; and and column 6 shows the difference in the mean absolute error between the mo… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Comparison of Tweedie vs RMSE diffusion downscaling over the August 2018 Beijing test period. (a) Distribu￾tion of per-timestep Kullback–Leibler divergence DKL(Truth ∥ Model) computed from histogrammed pixels (with ERA5 as truth). (b) mean absolute error over wet pixel…
Figure 5
Figure 5. Figure 5: For both timesteps, and across all three cases, [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 5
Figure 5. Figure 5: Performance of Tweedie vs RMSE ConvLSTM Kolkata nowcasts for three randomly selected test initialisations. Column 1 shows the last observed IMERG frame (t=0) that the models see; columns 2–3 show the subsequent half-hourly observations (t+1, t+2). Columns 4–5 show Twee…
Figure 6
Figure 6. Figure 6: Comparison of Tweedie vs RMSE ConvLSTM nowcasts over the August 2018 Kolkata test period. (a) Distribution of t+1 Kullback–Leibler divergence DKL(Truth ∥ Model) with IMERG as truth. (b–d) Mean scores vs lead time (t+1 to t+4) from autoregressive rollouts (each step fee…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

13 extracted references · 11 canonical work pages

  1. [1]

    Nasrollahi, 2010: Semi- parametric and parametric inference of extreme value models for rainfall data.Water resources manage- ment,24 (6), 1229–1249

    AghaKouchak, A., and N. Nasrollahi, 2010: Semi- parametric and parametric inference of extreme value models for rainfall data.Water resources manage- ment,24 (6), 1229–1249. Agrawal, S., L. Barrington, C. Bromberg, J. Burge, C. Gazen, and J. Hickey, 2019: Machine learning for precipitation nowcasting from radar images.arXiv preprint arXiv:1912.12132. Ambu...

  2. [2]

    Xiang, Y., J

    A comparison of distribu- tions of amounts.Water resources research,18 (5), 1461–1468. Xiang, Y., J. Ma, and X. Wu, 2020: A precipitation nowcasting mechanism for real-world data based on machine learning.Mathematical Problems in Engi- neering,2020 (1), 8408

  3. [17]

    Cao, Y., L. Chen, J. Wu, and J. Feng, 2025: Enhancing nowcasting with multi-resolution inputs using deep learning: Exploring model decision mechanisms.Geo- physical Research Letters,52 (4), e2024GL113

  4. [35]

    Svensson, C., J

    Suhaila, J., 2023: Tweedie models for Malaysia rain- fall simulations with seasonal variabilities.Journal of Water and Climate Change,14 (10), 3648–3670. Svensson, C., J. Hannaford, and I. Prosdocimi, 2017: Statistical distributions for monthly aggregations of precipitation and streamflow in drought indicator ap- plications.Water Resources Research,53 (2), 999–

  5. [308]

    Paulat, M

    Wernli, H., M. Paulat, M. Hagen, and C. Frei, 2008: SAL—A novel quality measure for the verification of quantitative precipitation forecasts.Monthly Weather Review,136 (11), 4470–4487. Wilks, D. S., 1989: Rainfall intensity, the weibull distri- bution, and estimation of daily surface runoff.Jour- nal of Applied Meteorology and Climatology,28 (1), 52–58. W...

  6. [529]

    Kostinski, A

    Kochkov, D., and Coauthors, 2024: Neural general circulation models for weather and climate.Nature, 632 (8027), 1060–1066. Kostinski, A. B., and A. R. Jameson, 1997: Fluctuation properties of precipitation. Part I: On deviations of single-size drop counts from the Poisson distribution. Journal of the atmospheric sciences,54 (17), 2174–

  7. [647]

    equitable threat score

    Ebert, E. E., L. J. Wilson, B. G. Brown, P. Nurmi, H. E. Brooks, J. Bally, and M. Jaeneke, 2004: Ver- ification of nowcasts from the WWRP Sydney 2000 forecast demonstration project.Weather and forecast- ing,19 (1), 73–96. Gupta, V. K., and E. Waymire, 1998: Some mathemat- ical aspects of rainfall, landforms, and floods.Ad- vanced series in statistical sci...

  8. [699]

    Climate Downscaling: A Deep-Learning Based Super-resolution Model of Precipitation Data with Attention Block and Skip Connections

    Chiang, C.-H., and Coauthors, 2024: Climate down- scaling: a deep-learning based super-resolution model of precipitation data with attention block and skip connections.arXiv preprint arXiv:2403.17847. Cho, H.-K., K. P. Bowman, and G. R. North, 2004: A comparison of gamma and lognormal distributions for characterizing satellite rain rates from the tropical...

Show all 13 references
  1. [931]

    Yan, C.-W., S. Q. Foo, V. H. Trinh, D.-Y. Yeung, K.- H. Wong, and W.-K. Wong, 2024: Fourier amplitude and correlation loss: Beyond using l2 loss for skillful precipitation nowcasting.Advances in Neural Infor- mation Processing Systems,37, 100 007–100

  2. [1018]

    Thom, H. C. S., 1958: A note on the gamma distribu- tion.Monthly weather review,86 (4), 117–122. Vandal, T., E. Kodra, S. Ganguly, A. Michaelis, R. Ne- mani, and A. R. Ganguly, 2017: DeepSD: Generating high resolution climate change projections through single image super-resol...

  3. [2186]

    Kumar, B., K

    Kozu, T., and Coauthors, 2001: Development of pre- cipitation radar onboard the Tropical Rainfall Mea- suring Mission (TRMM) satellite.IEEE transactions on geoscience and remote sensing,39, 102–116, doi: 10.1109/36.898669, URL http://dx.doi.org/10.1109/ 36.898669. Kumar, B., K...

  4. [2281]

    Martinez-Villalobos, C., and J. D. Neelin, 2019: Why do precipitation intensities tend to follow gamma dis- tributions?Journal of the Atmospheric Sciences, 76 (11), 3611–3631. Mittermaier, M. P., 2021: A “meta” analysis of the frac- tions skill score: The limiting case and imp...

  5. [3278]

    Murata, A., S.-i. I. Watanabe, H. Sasaki, H. Kawase, and M. Nosaka, 2020: Assessing goodness of fit to a gamma distribution and estimating future projec- tion on daily precipitation frequency using regional climate model simulations over Japan with and with- out the influence ...

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.