{"id":"902bae95-8e30-4e59-b330-77f31bcbe6d9","arxiv_id":"2411.08843","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"An AFNO-based neural network estimates global 6-hourly solar irradiance from weather model inputs, with satellite fine-tuning improving accuracy against ground stations.","lead":"This paper trains a global neural network model to estimate 6-hourly surface solar irradiance from standard weather-model variables, and shows that fine-tuning it on satellite data improves accuracy. The approach could support long-range solar power forecasts without relying on real-time satellite or ground measurements.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Fine-tuning gains are credible, but the headline forecast capability rests on an untested assumption: the diagnostic model is never run on actual SFNO/NWP forecast fields, only on ERA5 reanalysis, so distribution shift could invalidate the long-term SSI forecast claim.","rationale":"I agree with the reader's weakest-assumption diagnosis. The fine-tuning comparison is an internally consistent, held-out-year result with a direct ground-truth check (BSRN); I see no internal arithmetic or leakage problem in the 31.5%/7.8% claims. The unsupported step is the extension from 'diagnostic model on ERA5' to 'SSI forecast model on SFNO/NWP outputs.' The paper's own Section 2.4 says substituting xti with a forecast state generates a forecast, but Section 3 never performs that substitution. This matters because the diagnostic model is trained on ERA5's specific cloud/radiation relationship; SFNO is trained to reproduce atmospheric states, not SSI, and its forecast distributions differ systematically from reanalysis. Without an evaluation on forecast inputs, the abstract/conclusion claim of 'long-term SSI forecasts' is an extrapolation rather than a measured result. A single forecasting experiment with SFNO inputs would settle this. If it shows comparable accuracy, the forecast claim is validated; if not, the paper's contribution is narrower: an SSI diagnostic/emulator with improved fine-tuning. Thus the reader's CONDITIONAL verdict is the right level; I would not escalate to REJECT because the measured estimation claims are coherent and the forecasting architecture is plausible.","tokens_in":11647,"tokens_out":8558,"duration_ms":84834,"concrete_test":"Compute 2018 forecasts with the SFNO checkpoint cited in [2], initialized daily at 00 UTC from ERA5; extract the 73-channel fields at 6-hourly lead times; feed them through the AFNOf model (reproduced from the paper or, if released, the checkpoint) and compute RMSE/MAE/bias against BSRN, SARAH3 (downsampled), and ERA5 SSI, exactly as in Table 2. If the forecast-input RMSE is within ~10% of the ERA5-input RMSE and beats a simple persistence/climatology baseline, the transfer assumption is validated; if it degrades by more, the long-term forecasting claim is unsupported and the paper should be revised.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central forecasting claim (abstract: 'utilizes variables forecast by these NWP and AI weather models to estimate 6-hourly SSI'; conclusion: 'able to provide stable medium-term SSI forecasts') depends on the mapping f learned from ERA5 reanalysis states transferring to SFNO forecast states. This transfer is never tested. All validation in Section 3 — the 2018 RMSE/MAE/bias comparisons against ERA5, SARAH3, and BSRN — feeds ERA5 analysis fields into the diagnostic model. Figure 1 shows the SFNO pipeline only schematically; no table reports metrics for SFNO-forecast inputs. Because f predicts the 6-hour accumulated SSI ending at the input state time, the forecast chain relies on the model correctly interpreting forecast states that carry SFNO-specific biases, especially in cloud and humidity fields (q at 1000–850 hPa and tcwv are primary SSI controls). Reanalysis-to-forecast distribution shift is a known failure mode for data-driven emulators, and this paper provides no evidence about its magnitude. The fine-tuning result itself (31.5% and 7.8% RMSE improvements over AFNOERA5/AFNOSARAH3 on SARAH3, 8.2% on BSRN) is supported by held-out-year evaluation; the missing piece is specifically the forecast-input leg. The manuscript does not state this as a limitation; it is an omitted evaluation rather than an acknowledged caveat.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a global, 0.25-degree-resolution neural diagnostic model that takes an atmospheric state at time ti and outputs the 6-hourly averaged surface solar irradiance (SSI) accumulated over [ti−Δ, ti]. The architecture is an AFNO with a symmetry-aware patch embedder, trained on 37 years of ERA5 reanalysis (1980–2016) and evaluated on 2018. The authors compare AFNO against a Pix2Pix U-Net and a pointwise MLP, all trained to emulate ERA5 SSI. They then fine-tune the ERA5-trained model on the SARAH3 satellite product (AFNOf) and compare it with a model trained directly on SARAH3 (AFNOSARAH3). Reported results are that AFNOf improves RMSE by 31.5% over AFNOERA5 and by 7.8% over AFNOSARAH3 on SARAH3 validation, and by 8.2% over AFNOERA5 at BSRN stations within the SARAH3 domain. The abstract and conclusion further claim that the model can be used with NWP and AI weather models such as SFNO to produce stable medium-term global SSI forecasts.","tokens_in":11959,"tokens_out":8582,"duration_ms":84850,"significance":"The central fine-tuning result is an empirical finding that is credible given the held-out 2018 evaluation and the use of independent BSRN ground observations. If the authors add a proper evaluation of forecast-model inputs, the paper would make a useful contribution: it demonstrates a practical way to obtain a global SSI field from a small set of atmospheric prognostic variables without requiring satellite or ground observations at run time, and it shows that regional satellite fine-tuning can improve an ERA5-pretrained model rather than degrade global behavior. The comparison with a UNet and an MLP also gives evidence for the importance of a large receptive field. These strengths should be retained in revision. The current manuscript, however, makes a forecasting claim that is not backed by any experiment using forecast-model outputs, and some aggregate metrics in Table 2 do not support the strong wording used in Section 3.3.","major_comments":[{"comment":"The paper's abstract and conclusion claim that the model 'utilizes variables forecast by these NWP and AI weather models to estimate 6-hourly SSI' and 'is able to provide stable medium-term SSI forecasts.' However, every quantitative evaluation in Section 3 runs the diagnostic model on 2018 ERA5 reanalysis fields; no experiment feeds SFNO or any other forecast model's output into the trained model. Because Eq. (1) maps the atmospheric state at time ti to SSI accumulated over [ti−Δ, ti], the forecast chain depends on f generalizing from reanalysis states to forecast states that carry model-specific biases, especially in cloud and humidity fields (q at 1000–850 hPa and tcwv). This distribution shift is unquantified. I request an experiment that injects actual SFNO (or other NWP) forecast fields for 2018 into AFNOERA5/AFNOf and evaluates against BSRN/SARAH3; if that is outside the scope of the paper, the abstract and conclusion must be revised to state that forecast-model integration is proposed but not yet evaluated.","section":"Section 3 and Eq. (1)"},{"comment":"The claim that the fine-tuned model 'demonstrates higher accuracy compared to SARAH3' and shows 'a significant improvement on ground stations: 8.2% lower RMSE compared to AFNOERA5' is metric- and subset-dependent. The 8.2% figure corresponds to the SARAH3-station subset (RMSE 50.90→46.72 W m−2); on all stations the RMSE improvement is 4.2%, while MAE on all stations is worse for AFNOf than AFNOERA5 (26.86 vs 25.93 W m−2) and the bias is more negative (−5.35 vs 0.79 W m−2). Outside the SARAH3 domain (ROW), AFNOf has a worse MAE than AFNOERA5 (27.53 vs 25.19 W m−2). The paper should either state the claim with the exact subset and metric, or provide a multi-metric, multi-subsample analysis that accounts for these discrepancies.","section":"Table 2 and Section 3.3"}],"minor_comments":[{"comment":"The architecture description states that the input atmospheric state has c=31 channels, but the variables listed in Table 1 sum to 24 channels (z on 7 levels, t on 8 levels including 2 m, q on 7 levels, tcwv, and surface pressure). The paper should clarify how the remaining channels are formed from the orography, coordinate embeddings, and solar zenith angle, since this is required for reproducibility.","section":"Section 2.4 and Table 1"},{"comment":"There is a typo in the fine-tuning paragraph: 'with a a cosine annealing scheduler' should read 'with a cosine annealing scheduler.'","section":"Section 2.4"},{"comment":"The unit 'Wm2' should be typeset as W m−2, and the station abbreviations in Table A.1 should be separated by spaces or line breaks for readability.","section":"Table 2 and Table A.1"},{"comment":"The paper states that only BSRN stations with less than 10% missing data are kept but does not report the number of stations in each subset (All, SARAH3, ROW). This number is needed to interpret the aggregated RMSE/MAE/bias values and to assess the robustness of the fine-tuning conclusion.","section":"Section 3.3"},{"comment":"The percentage improvements of AFNOf over AFNOERA5 and AFNOSARAH3 are reported in the text but the underlying absolute RMSE values are not tabulated. A small table with the numerical values shown in Figure 4 would make the comparison reproducible.","section":"Section 3.2 and Figure 4"},{"comment":"The availability section links to the datasets and to the Modulus repository but does not provide trained model weights, training scripts, or the exact preprocessing code. For a data-driven paper, one of these should be made available to support the reported numbers.","section":"Code & Data Availability"}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern about reanalysis-to-forecast distribution shift is the main reason for major revision: the paper's headline forecasting claim is untested, but it is testable within the manuscript's scope by running SFNO forecast fields through the diagnostic model. I also note that the 'Code & Data Availability' section currently provides no code or model weights beyond pointers to public datasets and the Modulus framework, which may be an issue for reproducibility depending on the journal's policy. The fine-tuning result itself is a solid empirical contribution and should be preserved."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version: the fine-tuning result is real and worth publishing; the paper's headline claim that the model delivers SSI forecasts from NWP/AI weather models is not tested anywhere in the manuscript.\n\nFine-tuning an ERA5-trained AFNO model on SARAH3 does seem to help. On 2018 SARAH3 validation, AFNOf beats AFNOERA5 by 31.5% RMSE and AFNOSARAH3 by 7.8%. On BSRN stations, it is 8.2% better in RMSE than AFNOERA5 and, importantly, it stays accurate outside the SARAH3 domain while AFNOSARAH3 falls apart (RMSE 97.78 vs 49.84). That is a genuine, non-obvious transfer result, and the held-out-year evaluation plus the independent BSRN check give me confidence it is not a train/test leak. The variable selection is physically sensible, and the downsampling validation in Appendix A is a nice touch.\n\nThe load-bearing problem is the forecast framing. The abstract and conclusion say the model uses variables forecast by NWP/AI models to estimate SSI, and the conclusion claims stable medium-term forecasts. Every experiment in Section 3 feeds ERA5 reanalysis fields into the diagnostic model. The SFNO leg appears only in a schematic. Reanalysis-to-forecast distribution shift is a known killer for emulators, especially with q, tcwv and cloud fields, so until they run actual SFNO outputs through the model, the forecasting claim is just an assumption. The paper does not acknowledge this; it is an omitted evaluation, not a listed limitation.\n\nAlso, the fine-tuning gain is RMSE-specific. On all BSRN stations, AFNOf has slightly worse MAE than AFNOERA5 (26.86 vs 25.93) and a more negative bias (-5.35 vs 0.79). So the 8.2% RMSE improvement is not a uniform accuracy gain. Minor, but worth knowing. No code or weights are released; pointing to the Modulus repo is not enough to reproduce the specific model. And there is no comparison against existing global SSI forecast systems (e.g. EPT-1.5) or satellite retrieval models, which makes practical significance hard to judge.\n\nFor peer review: yes, send it out. The fine-tuning result deserves referee time. But the authors should be pushed to either run the diagnostic model on real forecast outputs or re-scope the claims. If the forecast leg cannot be tested, a paper about estimation with fine-tuning would still be solid.","headline":"Fine-tuning an ERA5-trained SSI model on SARAH3 is a real, publishable result; the forecasting claim in the abstract is not supported by any experiment in the paper.","tokens_in":12482,"tokens_out":2838,"would_cite":false,"duration_ms":26141,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Pre-training a neural operator on reanalysis then fine-tuning it on satellite data gives more accurate global 6-hourly surface solar irradiance estimates than training on either data source alone, with a 31.5% RMSE improvement on the…","keywords":["surface solar irradiance","solar forecasting","neural operators","AFNO","fine-tuning","ERA5","SARAH3","global weather models"],"falsifier":"Feed actual forecast fields from SFNO or another weather model into the trained diagnostic model and compare the resulting 6-hourly SSI against ground-station and satellite data across several lead times; if RMSE grows sharply with lead time or far exceeds the errors seen when the model is fed reanalysis fields, the transfer assumption fails.","tokens_in":11452,"feed_emoji":"☀️","tokens_out":9586,"duration_ms":201011,"temperature":0.7,"pith_summary":"This paper tries to establish that a single global machine-learning model can estimate 6-hourly surface solar irradiance—the sunlight reaching the ground—from the atmospheric variables that numerical weather prediction and AI weather models routinely forecast, without needing real-time satellite or ground observations. The model is an Adaptive Fourier Neural Operator, a neural network that processes global fields in Fourier space, trained on 37 years of ERA5 reanalysis data and then fine-tuned on the satellite-derived SARAH3 product. The central measured finding is that fine-tuning the reanalysis-trained model on satellite data outperforms both training on reanalysis alone and training on satellite data alone: on the SARAH3 validation domain the fine-tuned model reduces RMSE by 31.5% relative to the reanalysis-trained version and by 7.8% relative to the satellite-only version, and at ground stations it beats the reanalysis-trained model by 8.2% RMSE. If the result holds, solar-power forecasting can piggyback on advances in weather prediction, producing global multi-day irradiance forecasts from forecast-model output alone.","feed_headline":"Satellite fine-tuning cuts solar irradiance forecast RMSE by 31.5%","feed_subtitle":"Reanalysis pre-training plus satellite fine-tuning beats either data source alone for global solar forecasts.","key_machinery":"The load-bearing component is the Adaptive Fourier Neural Operator (AFNO), a neural network architecture that mixes spatial information globally by transforming latents with the discrete Fourier transform, applying an MLP, and transforming back; this gives every output pixel access to the whole globe in one pass. Around this, the model uses a symmetric convolutional patch embedder that wraps around longitude to avoid grid-edge artifacts, sinusoidal latitude/longitude embeddings instead of learned positional embeddings, and a solar-zenith-angle conditioning field. The training target is 6-hourly averaged ERA5 surface solar radiation downwards; the loss is latitude-weighted L2, and fine-tuning uses a masked L2 loss restricted to the SARAH3 domain. The argument's mechanism is that the global receptive field encodes the previous six hours of cloud advection from the current atmospheric state, which is what a 6-hour average irradiance estimate requires.","core_discovery":"On the paper's own terms, the discovery is that a diagnostic neural operator can learn the mapping from atmospheric state to accumulated solar irradiance well enough that pre-training on reanalysis and fine-tuning on satellite observations is a better strategy than training on either data source alone. The paper reports that AFNOf—the ERA5-trained model fine-tuned on SARAH3—achieves a 31.5% lower RMSE than AFNOERA5 and a 7.8% lower RMSE than AFNOSARAH3 when validated against SARAH3 6-hourly fields, and an 8.2% lower RMSE than AFNOERA5 at BSRN ground stations. It further argues that because the model's inputs are only atmospheric variables forecast by weather models, the same diagnostic model can be chained to a prognostic weather model to produce global, 6-hourly SSI forecasts at 0.25° resolution over multi-day lead times, independent of real-time satellite or ground observations. The receptive-field analysis is part of the same argument: the global receptive field of AFNO lets it capture cloud advection over the previous six hours, which a point-wise MLP cannot do and a convolutional U-Net only partially does.","pith_inferences":["If the transfer from reanalysis to forecast-model inputs is the real bottleneck, the reported 31.5% improvement from fine-tuning may understate the benefit in an operational forecast setting, where satellite fine-tuning could also partially correct forecast-model systematic biases.","The receptive-field result suggests a testable extension: with a global-receptive-field model, forecast skill should degrade more slowly with lead time than with local models, because cloud advection over the aggregation window is captured rather than approximated.","The same architecture could be fine-tuned on other satellite products or on regional high-resolution irradiance networks to build a global model that is locally accurate everywhere, not just where SARAH3 covers."],"forward_implications":["Global 6-hourly SSI forecasts can be produced by chaining the diagnostic model to a weather forecast model, using only variables those models already output, with no real-time satellite or ground data.","Fine-tuning on satellite data improves accuracy inside the satellite domain while preserving skill outside it, so regional high-quality data can be folded into a global model.","The AFNO-based model outperforms a convolutional U-Net baseline by 10.7% in RMSE and 76% in bias on ERA5 validation, indicating that global receptive field is the relevant architectural property.","The same training-and-fine-tuning recipe can be applied to any future weather model's variable set, so SSI forecasting inherits progress in numerical weather prediction and AI weather prediction automatically."],"supporting_citations":[{"why":"Defines the 73 output variables used as model inputs and provides the spherical Fourier neural operator weather model the framework is designed to couple with.","marker":"[1]"},{"why":"Shows satellite-based irradiance estimates are more accurate than ERA5 at ground stations, motivating fine-tuning on SARAH3 and the ground-station validation design.","marker":"[6]"},{"why":"Supplies the 37 years of global reanalysis fields and the 6-hourly accumulated SSI target used for pre-training.","marker":"[19]"},{"why":"Provides the satellite-derived SSI dataset used for fine-tuning and as the high-accuracy validation reference.","marker":"[20]"},{"why":"Provides the ground-station pyranometer measurements used for the final evaluation against observations.","marker":"[22]"},{"why":"Contributes the AFNO-based architecture and training scheme adapted from FourCastNet for the diagnostic model.","marker":"[25]"},{"why":"Supplies the Adaptive Fourier Neural Operator layer, the global token-mixing mechanism at the center of the model.","marker":"[26]"},{"why":"Provides the Pix2Pix U-Net architecture used as a convolutional baseline in the comparison.","marker":"[29]"}],"fun_headline_variants":["Global solar irradiance via neural operators fine-tuned on satellite data","Fine-tuning neural solar forecasts on satellite data cuts RMSE by 31.5%","Blend reanalysis and satellite data to forecast solar irradiance globally","Chained weather models and neural operators give multi-day solar forecasts"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The model is trained and validated only on past weather maps from the ERA5 dataset, so its ability to forecast from live weather-model predictions assumes those predictions resemble ERA5 maps closely enough that the learned mapping still holds.","fun_headline_variants_meta":{"raw":{"variants":["Global solar irradiance via neural operators fine-tuned on satellite data","Fine-tuning neural solar forecasts on satellite data cuts RMSE by 31.5%","Blend reanalysis and satellite data to forecast solar irradiance globally","Chained weather models and neural operators give multi-day solar forecasts"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000704,"raw_usage":{"total_tokens":3181,"prompt_tokens":960,"completion_tokens":2221,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":576,"completion_tokens_details":{"reasoning_tokens":2143}},"tokens_in":576,"tokens_out":2221,"duration_ms":14378,"temperature":1.0,"reasoning_tokens":2143,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T21:15:38.923215+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Feed actual forecast fields from SFNO or another weather model into the trained diagnostic model and compare the resulting 6-hourly SSI against ground-station and satellite data across several lead times; if RMSE grows sharply with lead time or far exceeds the errors seen when the model is fed reanalysis fields, the transfer assumption fails.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the 73 output variables used as model inputs and provides the spherical Fourier neural operator weather model the framework is designed to couple with."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Shows satellite-based irradiance estimates are more accurate than ERA5 at ground stations, motivating fine-tuning on SARAH3 and the ground-station validation design."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the satellite-derived SSI dataset used for fine-tuning and as the high-accuracy validation reference."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the ground-station pyranometer measurements used for the final evaluation against observations."},{"cited_title":"-Y ., Zhou, T","cited_arxiv_id":null,"evidence_quote":"Provides the Pix2Pix U-Net architecture used as a convolutional baseline in the comparison."}],"review_version":1}