{"id":"157e7b16-64e4-41f1-9bde-8bce64469364","arxiv_id":"2509.08369","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Training precipitation models with Tweedie deviance instead of root-mean-square error improves extreme-rainfall recall in downscaling and improves nowcasting skill, with gains compounding over autoregressive lead times.","lead":"A weather AI paper argues that the standard error metric used to train rain-predicting models treats rainfall as if it were a smooth bell curve, which it is not. It shows that a statistical loss called Tweedie deviance predicts heavy rain better in two test models, downscaling and nowcasting.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Nowcasting improvements lack error bars and rest on a single test month; the headline compounding gains may be sampling noise.","rationale":"The paper's Tweedie deviance derivation is standard and correct, and the controlled same-architecture comparisons are a credit. The reader's conditional verdict is appropriate. I focus on nowcasting uncertainty rather than the p-estimation concern because even if the global p is imperfect, Tweedie deviance remains a proper scoring rule for the conditional mean; the empirical superiority is the decisive evidence for the practical recommendation 'replace RMSE'. The missing error bars and single-month evaluation make the strongest claim unverified, but not disproven, so the verdict remains conditional rather than moving to reject or accept.","tokens_in":16643,"tokens_out":11897,"duration_ms":147056,"concrete_test":"Re-evaluate on the full 2018 monsoon season already in the test split (June–August). For each month, compute wet-pixel MAE and dry hit rate at t+1..t+4 for both models, and form 95% bootstrap CIs over non-overlapping 5-day blocks (to respect autocorrelation) for the Tweedie-minus-RMSE difference. If the t+4 MAE difference is not positive in all three months and its CI excludes zero, the compounding claim is not established. If code/data are released, also retrain with three seeds to check seed sensitivity.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest empirical claim—Tweedie wet-pixel MAE improves by ~2% at t+1 growing to ~16% at t+4—is based on one month (August 2018, Fig. 6) with no confidence intervals or significance tests. The downscaling experiment (Fig. 4) shows that where the authors do bootstrap, the MAE and dry-hit-rate differences are not significant; only extreme recall is. For nowcasting, no such checks are reported. Differences between two trained networks are subject to initialization, early stopping, and spatiotemporal autocorrelation, so the t+4 difference could easily be within sampling noise. The compound-growth language implies a robust effect, but the evidence is a single realization. The paper itself (Sec. 5) defers p-sensitivity to future work, and code/data are not provided, so this headline result cannot be independently checked.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper argues that RMSE is a misspecified training objective for precipitation and proposes the Tweedie deviance as a differentiable, likelihood-based replacement. Section 2 derives the unit deviance for the exponential dispersion family and recovers the Gaussian, Poisson, and Gamma limits. Section 4.1 estimates the Tweedie power p from the variance–mean power law for a UK rain gauge, ERA5, and GPM-IMERG data, showing p increases with accumulation length. Section 4.2 compares RMSE and Tweedie training of a conditional diffusion downscaler over Beijing; Section 4.3 compares RMSE and Tweedie training of a ConvLSTM nowcaster over Kolkata. The paper claims improved extreme recall in downscaling and improved wet-pixel MAE and dry hit rates in nowcasting, with gains that compound with lead time.","tokens_in":16905,"tokens_out":9967,"duration_ms":108689,"significance":"If the empirical gains are robust, the paper provides a practical and well-motivated alternative to RMSE, with a clean derivation and a simple recipe for estimating p. The use of identical architectures for both loss comparisons is a strength, as is the multi-dataset demonstration that p is far from zero and varies with aggregation. The paper is honest about the non-significance of some downscaling differences. The principal concern is that the central nowcasting claim rests on a one-month evaluation without uncertainty quantification, and the 'statistically justified' framing is not tested against misspecification of p.","major_comments":[{"comment":"The headline claim of ~2%→16% MAE improvement and ~2%→13% dry-hit-rate improvement rests entirely on one month (August 2018) with no error bars, confidence intervals, or significance tests, even though App. A.2 defines a three-month test period (Jun–Aug 2018). Given stochastic training and strong space–time autocorrelation in nowcast errors, the t+4 differences could be sampling noise. This is the paper's strongest claim and needs a full-test-period evaluation with block-bootstrap CIs and paired significance tests (or an explicit justification for the August-only choice).","section":"§4.3, Fig. 6"},{"comment":"The abstract concludes that Tweedie loss 'improves wet-pixel MAE and extreme recall' in downscaling, and the introduction repeats this. Fig. 4b shows a non-significant MAE difference (0.415 vs 0.410) and the text states this is not significant; only extreme recall (0.504 vs 0.602, Fig. 4d) reaches significance, and only for a single month and a 99th-percentile threshold. The claim should be qualified and, if possible, the analysis extended to the full test year specified in App. A.3.","section":"§4.2, Fig. 4; Abstract"},{"comment":"The Tweedie power p is estimated once from the marginal variance–mean power law (blockwise OLS, averaged over pixels) and then fixed for training. The paper itself defers p-sensitivity to future work. If p is biased or varies with intensity, storm type, or orography, the training objective is not the Tweedie likelihood the authors invoke as justification for the skill gains. Please add sensitivity tests (e.g., p ± 95% CI or profile-likelihood estimate) and report the exact p used for each experiment; otherwise the 'statistically justified' framing is not supported.","section":"§4.1, App. A.4, §5"}],"minor_comments":[{"comment":"The Taylor expansion for θ is not correct as written: for p=ε, θ = μ^{1-ε}/(1-ε) = μ[1 + ε(1 - ln μ)] + O(ε²), not μ[1 - (ε/2) ln μ] + O(ε²). The leading term is unaffected, but the displayed expression should be fixed.","section":"§2.1, Eq. (11)"},{"comment":"The '~16%' improvement at t+4 is ambiguous: (0.669−0.578)/0.669 ≈ 14%, while (0.669−0.578)/0.578 ≈ 16%. The standard relative-to-RMSE convention gives ~14%; please align the abstract, main text, and figure caption on one definition and verify the numbers.","section":"§4.3 and Fig. 6b"},{"comment":"The actual values of p used for the Beijing hourly downscaling and Kolkata half-hourly nowcasting experiments are not reported. Since the method's reproducibility depends on this, please list the estimated p and its confidence interval for each training setup.","section":"App. A.4"},{"comment":"The text says differences are 'bootstrap tested' but no details of the bootstrap procedure (resampling unit, number of replications, CI construction) are given. Please describe the procedure or cite a standard method.","section":"§4.2"},{"comment":"The systematic 'weekly dip' in p is left unexplained. If it cannot be analyzed, at least label it as a possible finite-sample artifact and remove it from the bullet-point conclusions, or provide a targeted test.","section":"§4.1, Fig. 2"}],"recommendation":"major_revision","confidential_remarks":"The paper is likely to be of interest to the journal's readership; the derivation is clean and the practical recipe is useful. The main barrier is verification: the central nowcasting claim needs an uncertainty-aware, multi-month evaluation, and the p-sensitivity analysis should be added. I would not reject on the basis of the derivation or the concept."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a look if you do precipitation machine learning. The paper argues that RMSE is a poor training loss for precipitation and proposes the Tweedie deviance instead. The math is standard exponential-dispersion-family material, but it is cleanly derived and the practical case is made well. The genuinely new bits are the empirical estimates of the Tweedie power p across accumulation periods and datasets (showing a Poisson-to-Gamma progression) and two controlled deep-learning experiments that compare the same architecture trained with RMSE versus Tweedie deviance.\n\nCredit where it is due: the downscaling experiment is reported honestly. The authors bootstrap their metrics and explicitly say that MAE and dry-hit-rate differences are not significant; only extreme recall improves significantly (0.60 vs 0.50 at the 99th percentile). That is exactly how these comparisons should be presented, and it gives the claim credibility.\n\nThe soft spot is the nowcasting experiment. The headline result—wet-pixel MAE improving by ~2% at t+1 growing to ~16% at t+4—comes from a single test month (August 2018) with no confidence intervals, no significance tests, and no multiple seeds. Given that the downscaling experiment found most differences non-significant after bootstrapping, it is quite possible the nowcasting gains are partly sampling noise. The authors also fit one global p per dataset via a block-average OLS power law and then apply it everywhere; they acknowledge in Section 5 that p-sensitivity is future work, but that means the “statistically justified” framing rests on an untested assumption. No code or data are provided, so the headline numbers cannot be independently checked.\n\nThese are addressable issues rather than fatal flaws. The central argument—that a likelihood-based loss matching the zero-inflated, heavy-tailed character of precipitation is a better training target than MSE—survives the empirical weaknesses. The paper is clearly written, cites the relevant Tweedie literature, and does not overclaim beyond what the downscaling experiment supports.\n\nA serious editor should send this to peer review. A good referee can ask for error bars on the nowcasting metrics, a longer evaluation period, and ideally code and data. The contribution is solid enough to deserve that engagement.\n\nFor my own work, I would cite it as a reference for Tweedie deviance in precipitation ML, and I might bring it to a reading group focused on loss functions in Earth-system ML.","headline":"A genuinely useful proposal to replace MSE with Tweedie deviance for precipitation ML, let down by nowcasting claims with no error bars on a single test month.","tokens_in":17322,"tokens_out":1477,"would_cite":true,"duration_ms":17363,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper argues that the standard training loss for precipitation models, root-mean-square error, should be replaced by the Tweedie deviance, a likelihood-based loss that matches rainfall's zero-inflated, heavy-tailed distribution, and sh","keywords":["Tweedie distribution","compound Poisson–Gamma","precipitation","loss function","root-mean-square error","nowcasting","downscaling","variance–mean power law"],"falsifier":"Estimate p separately for stratified storm regimes (monsoon versus frontal versus orographic) at one location; if the differences exceed the ordinary least-squares confidence intervals, a single p is misspecified and the deviance would need a mixture or spatially varying p. A second check: retrain the same models with p obtained from profile likelihood instead of block-OLS estimation and compare extreme recall and wet-pixel MAE.","tokens_in":16578,"feed_emoji":"🌧️","tokens_out":5563,"duration_ms":53199,"temperature":0.7,"pith_summary":"Rain is not Gaussian: it is a mix of many dry periods and rare, heavy downpours, yet most machine-learning models of precipitation are trained by minimizing mean-squared error, which assumes Gaussian errors. This paper argues that the training objective itself should match the data-generating process, and proposes the Tweedie deviance, a likelihood-based loss that naturally handles zero-inflated, non-negative, heavy-tailed rainfall. The author shows that the Tweedie power parameter p can be estimated from the observed variance–mean power law and that precipitation at every practical accumulation length sits between the Poisson (p=1) and Gamma (p=2) limits, never near Gaussian. On two standard tasks, diffusion downscaling over Beijing and ConvLSTM nowcasting over Kolkata, swapping RMSE for Tweedie deviance leaves overall metrics unchanged or better while improving extreme-rainfall recall and the compounding skill of autoregressive nowcasts. The point is practical: a one-line change to the loss function aligns the model with rainfall statistics and improves exactly where Gaussian loss fails.","feed_headline":"Swap RMSE for Tweedie loss and rainfall models catch more extremes","feed_subtitle":"A one-line training-objective change improves extreme downscaling recall and compounds nowcast skill at longer lead times.","key_machinery":"Tweedie deviance: a likelihood-based loss from the exponential–dispersion family with variance function V(µ)=µ^p. For 1<p<2 it is the compound Poisson–Gamma distribution, and its unit deviance has a closed form. The power p is fit once from the variance–mean power law; because p is continuous, the same loss interpolates from Poisson-like (p→1) to Gamma-like (p→2) behavior. The paper uses this deviance as the training loss in place of MSE, with a softplus output to keep predictions non-negative and dispersion set to 1.","core_discovery":"At the center is the unit Tweedie deviance, a closed-form likelihood-based loss from the exponential–dispersion family whose variance function is V(µ)=µ^p. For 1<p<2 it corresponds to a compound Poisson–Gamma distribution, with a point mass at zero and continuous positive amounts, which matches observed precipitation. The paper derives that RMSE is the p→0 Gaussian limit of this same family, so using RMSE imposes a Gaussian likelihood on a non-Gaussian target. The author estimates p from log Var(Y)=c+p log E[Y] over non-overlapping blocks and finds p in [1.18, ~2] across gauge, reanalysis, and satellite data, increasing with accumulation period toward the Gamma limit. Training identical down","pith_inferences":["A natural extension the author leaves implicit: because the Tweedie deviance is a proper scoring rule for the conditional mean, it could replace MSE in other zero-inflated, heavy-tailed geophysical targets such as wind gusts, cloud water, or air-quality concentrations, with p estimated per variable.","A testable refinement: replacing the single global p with a spatially or seasonally indexed p, or parameterizing p as part of the network, could improve conditional tails further; the paper's own single-p design leaves this open.","The unexplained 'weekly dip' in p may reflect real meteorological structure rather than sampling noise; if it persists, p itself could serve as a diagnostic of rainfall regime, not just a tuning parameter.","The same likelihood framing extends naturally to probabilistic outputs: training a distribution head with Tweedie deviance would give a calibrated predictive distribution and explicit uncertainty, which a point-loss RMSE baseline cannot provide."],"forward_implications":["Downscaling models trained with Tweedie deviance match RMSE-trained models on overall distributional fit, wet-pixel MAE, and dry hit rate, while recalling about 60% versus 50% of 99th-percentile extremes; operational downscaling can recover extreme events without sacrificing average skill.","Nowcasting models trained with Tweedie deviance improve wet-pixel MAE and dry-pixel hit rate at every lead time, and the improvements grow with autoregressive rollout—about 2% MAE at t+1 growing to about 16% at t+4—so the loss choice compounds over the forecast horizon.","The estimated Tweedie power p rises from near Poisson at 5-minute accumulation to near Gamma for monthly totals, so the same continuous loss family adapts to any accumulation period once p is estimated from the training data.","Because p is not known a priori and differs between locations and datasets, users should estimate p on the target data at the target accumulation period rather than fixing it."],"supporting_citations":[{"why":"Supplies the practical likelihood-based fitting machinery for Tweedie models that the paper builds its loss on.","marker":"Dunn, 2004"},{"why":"Demonstrates the Tweedie/compound Poisson–Gamma as a simultaneous model of rainfall occurrence and amount, the statistical precedent for applying it to precipitation.","marker":"Hasan and Dunn, 2010"},{"why":"Gives the physical rationale for Gamma-like rainfall distributions under temporal averaging, supporting the p→2 limit as accumulation length grows.","marker":"Martinez-Villalobos and Neelin, 2019"},{"why":"Provides the ConvLSTM architecture and benchmark used for the Kolkata nowcasting experiment.","marker":"Shi et al., 2017"},{"why":"Supplies the deterministic DDIM sampling schedule used by the diffusion downscaling model.","marker":"Song et al., 2020"},{"why":"Defines GPM-IMERG, the half-hourly satellite precipitation dataset used to train and evaluate the nowcasting model.","marker":"Huffman et al., 2020"},{"why":"Defines ERA5, the reanalysis dataset used for the Beijing downscaling experiment and part of the p-estimation analysis.","marker":"Hersbach et al., 2020"}],"fun_headline_variants":["Tweedie loss beats RMSE for extreme rainfall prediction","Why RMSE fails for precipitation and what to use instead","RMSE underestimates heavy rain; Tweedie loss does not","For rainfall models, swap RMSE for Tweedie deviance","Train rain models with Tweedie loss, not RMSE"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The paper assumes that a single spatially averaged Tweedie power p, estimated by least-squares fitting of the variance–mean relationship, correctly describes the rainfall distribution for every location and case; if the true p varies by storm type, terrain, or intensity, or if that fit is biased, the Tweedie deviance is misspecified as a likelihood and the statistical justification for the observed gains weakens.","fun_headline_variants_meta":{"raw":{"variants":["Tweedie loss beats RMSE for extreme rainfall prediction","Why RMSE fails for precipitation and what to use instead","RMSE underestimates heavy rain; Tweedie loss does not","For rainfall models, swap RMSE for Tweedie deviance","Train rain models with Tweedie loss, not RMSE"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000232,"raw_usage":{"total_tokens":1398,"prompt_tokens":887,"completion_tokens":511,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":631,"completion_tokens_details":{"reasoning_tokens":424}},"tokens_in":631,"tokens_out":511,"duration_ms":5696,"temperature":1.0,"reasoning_tokens":424,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T20:38:44.346153+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Estimate p separately for stratified storm regimes (monsoon versus frontal versus orographic) at one location; if the differences exceed the ordinary least-squares confidence intervals, a single p is misspecified and the deviance would need a mixture or spatially varying p. A second check: retrain the same models with p obtained from profile likelihood instead of block-OLS estimation and compare extreme recall and wet-pixel MAE.","supporting_citations":[],"review_version":1}