{"id":"ba06d800-e2ee-456b-bebc-6c0e0c55c916","arxiv_id":"2505.10281","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"MFogHub is a new open benchmark with 68,000 multi-region, multi-satellite marine fog samples and experiments showing regional and satellite generalization differences.","lead":"MFogHub combines satellite images and expert labels from 15 fog-prone coastal regions and six weather satellites into a dataset of more than 68,000 samples for detecting and forecasting marine fog. The paper also benchmarks 16 deep learning models to reveal how performance shifts across regions and satellites.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Forecasting benchmark tables report impossible MSE/MAE pairs (e.g., MSE 3315, MAE 17461), so the forecasting evaluations cannot be valid; the claim that MFogHub supports marine fog forecasting rests on these numbers.","rationale":"The reader's stated weakest assumption is annotation quality, which is a valid concern for the detection benchmarks. However, the forecasting results contain an even more direct, checkable internal inconsistency: every reported MSE/MAE pair in Tables 3 and 5 violates the inequality sqrt(MSE) >= MAE. This is not a matter of interpretation or consensus; it is a mathematical impossibility under the standard definitions the paper claims to use. Because the abstract and introduction emphasize forecasting as a core capability of MFogHub, and because the experiments are the primary evidence for that capability, the invalid metrics are load-bearing. The dataset itself may still be valuable, and the detection experiments are not directly affected, so the appropriate response is to require the authors to correct the metric definitions/units and regenerate all forecasting tables, or to soften the forecasting claims until that is done. Since the reader's verdict is already CONDITIONAL and this concern reinforces that conditionality rather than overturning it, the verdict should remain unchanged.","tokens_in":15389,"tokens_out":4286,"duration_ms":43385,"concrete_test":"Take the raw prediction and ground-truth tensors for any forecasting row in Table 3 (e.g., ConvLSTM on N.S.), and recompute MSE and MAE with the definitions stated in §4.2. If sqrt(MSE) < MAE, the published pair is impossible and the benchmarking pipeline has a metric or scaling bug; then recompute all rows of Tables 3 and 5. Ideally, release the exact metric script with the repository.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that MFogHub supports both detection and forecasting is supported in part by Tables 3 and 5, which report MSE and MAE for eight forecasting baselines. These two metrics are mathematically linked: for any error vector e, sqrt(MSE) = ||e||_2 / sqrt(N) >= ||e||_1 / N = MAE (by Cauchy-Schwarz or Jensen's inequality). Yet Table 3 lists for ConvLSTM on N.S.: MSE=3315.46, MAE=17461.5, where sqrt(3315.46) ≈ 57.6 << 17461.5. The same impossible pattern repeats across every row in Tables 3 and 5 (e.g., Table 5 H8/9 ConvLSTM: MSE=697.12, MAE=7540.91). Unless the columns are mislabeled or the errors are computed on different quantities (e.g., MSE on normalized pixels and MAE on raw counts), at least one metric is not what the paper claims. No definition in §4.2 or the text explains such a discrepancy. Since all forecasting conclusions—regional generalization, satellite variation, model rankings—are derived from these tables, the forecasting half of the benchmark results is not trustworthy. The dataset may still contain valid temporal sequences, but the paper's demonstration that MFogHub enables marine fog forecasting is currently unsupported. This is an internal consistency failure, not a disagreement with the community.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents MFogHub, a large-scale marine fog dataset that integrates multi-regional and multi-satellite observations. The dataset includes over 68,000 samples from 15 coastal fog-prone regions observed by six geostationary satellites (FY4A, FY4B, GOES-16, GOES-17, H8/9, MeteoSat), organized into 21 cube-streams with dimensions of timestamp-spectral band-latitude-longitude. More than 11,600 samples are pixel-level annotated by meteorological experts. The paper describes the data collection and organization in detail, provides exploratory analysis of regional, satellite, and spectral-band variations, and reports benchmark experiments for eight detection models and eight forecasting models. The central claims are that MFogHub is the first global multi-regional, multi-satellite marine fog benchmark, and that it enables rigorous evaluation of model generalization across regions and satellites.","tokens_in":15670,"tokens_out":2864,"duration_ms":28692,"significance":"If the dataset is as described and the benchmarks are valid, MFogHub would be a substantial community resource: it is the first to combine 15 regions, six satellites, and both detection and forecasting tasks in a single open dataset. The cube-stream organization, built on the Mesogeos concept, is a sensible way to handle spatiotemporally aligned multi-spectral data, and the paper gives credit to that prior work. The data collection pipeline using ICOADS records and a 12.8-degree sliding window is clearly described, and the paper performs useful exploratory analysis on regional discrepancy and satellite variation. The detection benchmark tables report plausible metric values and the authors make an explicit recommendation on the positive-to-negative sample ratio based on experiments, which is falsifiable. However, the forecasting benchmark tables contain internally inconsistent MSE/MAE/PSNR values, and all experiments are reported as single-run point estimates. These issues undermine the forecasting half of the paper's central claim, so the manuscript requires major revision.","major_comments":[{"comment":"The reported forecasting metrics are mutually inconsistent. For any error vector e, the root-mean-square error satisfies sqrt(MSE) = ||e||_2 / sqrt(N) >= ||e||_1 / N = MAE by the Cauchy-Schwarz inequality. Yet Table 3 lists for ConvLSTM on the N.S. sub-dataset MSE=3315.46 and MAE=17461.5, so sqrt(MSE) ≈ 57.6 < 17461.5. The same reversed ordering appears in every row of Tables 3 and 5, for example Table 5 H8/9 ConvLSTM: MSE=697.12, MAE=7540.91. The PSNR values also do not correspond to the reported MSE under a standard 0-255 or 0-1 pixel range. Unless the columns are mislabeled or the metrics are computed on different quantities (e.g., MSE on normalized pixels and MAE on raw counts), at least one of the metrics is not what the paper claims. No such explanation is given in Section 4.2. Because all forecasting conclusions and model rankings are derived from these tables, the forecasting benchmark results are not currently trustworthy and must be recomputed or clarified.","section":"Section 4.3, Tables 3 and 5"},{"comment":"All experimental results are reported as single-run point estimates without standard deviations, confidence intervals, or the number of random seeds. With eight baseline models and multiple sub-datasets, differences between models (e.g., Table 4's DlinkViT vs. ViT on H8/9, or Table 3's TAU vs. MIM on M.W.) could be within run-to-run noise. The paper should state the number of seeds, report variance, and describe the train/validation/test splits and hyperparameter selection procedure. Without this, the generalization conclusions based on small metric differences are not robust.","section":"Section 4.2 and Tables 2-6"},{"comment":"The 11,600 pixel-level annotation masks are described as 'meticulously annotated by meteorological experts' with procedures deferred to the Supplementary Materials. No inter-annotator agreement, annotation protocol summary, or independent validation is provided in the main text. Since all detection benchmarks and the detection-based generalization analysis depend on these labels, the paper should at least summarize the annotation protocol, quality control procedure, and estimated label uncertainty in the main text. This is a load-bearing point for the detection half of the dataset.","section":"Section 2.3 and detection experiments"}],"minor_comments":[{"comment":"The reference for SimVP-v2 is given as [6] in the table, but [6] is PhyDNet; the correct reference appears to be [29]. Please fix the citation.","section":"Table 5"},{"comment":"The region list contains a typo: 'South Sae' should be 'South Sea'. Also, 'costal' in Section 2.1 should be 'coastal'.","section":"Table 1"},{"comment":"The phrase 'we prioritiz coverage' should be 'we prioritize coverage'. Also, the distinction between 'L1 data' and 'L1.5 data from MeteoSat' is unclear and would benefit from a brief explanation or reference.","section":"Section 2.2"},{"comment":"In the text, the model name 'Deeplabv3' is used while the table and reference list say 'Deeplabv3p'; please standardize the name. Also 'V AN' appears with a space in Table 3; it should be 'VAN'.","section":"Section 4.3"},{"comment":"The recommendation of a 2:1 positive-to-negative sample ratio is based on two regions (B.C. and G.A.) only. This is an interesting empirical observation, but the phrasing 'for future dataset construction' is too broad without testing on additional regions or tasks.","section":"Figure 9 and Section 4.6"}],"recommendation":"major_revision","confidential_remarks":"The internally inconsistent MSE/MAE values in Tables 3 and 5 are concerning; they suggest either a unit/scale mismatch in the metrics or an error in table generation. The authors should be asked to provide the raw prediction-error histograms or a clear definition of how each metric was computed. The dataset itself may still be valuable, and the detection benchmarks appear plausible, but the forecasting benchmark as presented cannot be accepted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: the dataset artifact is real and worth having. MFogHub is the first marine fog benchmark spanning 15 regions, 6 geostationary satellites, 21 cube-streams, and 68k samples, with 11,600 pixel-level masks. If the repo and supplementary material deliver what the paper promises, this gives the subfield a common testbed for cross-region and cross-satellite generalization that did not exist before. The cube-stream organization is a sensible adaptation of Mesogeos, and the ICOADS-based region selection plus multi-satellite collection is a genuine piece of work. Credit where due: this is more than a routine dataset extension, and the detection experiments (Tables 2 and 4) are at least plausible in shape.\n\nNow the soft spots, in proportion. I checked the stress-test note's metric arithmetic and it lands. The forecasting half of the paper does not hold up as reported. Tables 3 and 5 list MSE and MAE values that violate the inequality sqrt(MSE) >= MAE. For example, ConvLSTM on N.S. has MSE=3315.46 and MAE=17461.5; sqrt(MSE) is about 57.6, which is less than the MAE by a factor of 300. The same pattern repeats in nearly every row of both tables. No scaling or normalization is described that would explain this. That means the numerical forecasting results, and every conclusion drawn from them about regional or satellite generalization, are not trustworthy as published. This is not a stylistic quibble; it is an internal inconsistency in the core evidence for the forecasting claim.\n\nThere are other issues. The 11,600 annotation masks get one sentence in the main text; no protocol, no inter-annotator agreement, no independent validation. The experiments are single-run point estimates without error bars, so small differences between models could be noise. A few citation slips (e.g., SimVP-v2 cited as [6] in Table 5) suggest a hasty final pass. None of these are fatal to the dataset concept, but they matter for the benchmark numbers.\n\nWho this is for: anyone working on sea fog detection or spatiotemporal predictive learning with satellite data. The dataset itself is the contribution; the forecasting evaluation needs to be redone or corrected before the numbers can be used.\n\nMy recommendation: send it to peer review, but the referee should require the authors to reconcile or relabel the MSE/MAE columns, add error bars or multiple seeds, and describe the annotation protocol. If the tables cannot be fixed, the forecasting sections should be replaced with a correct evaluation or removed. The dataset deserves a serious referee; the forecasting numbers do not deserve to be taken at face value.\n\nBest,","headline":"Valuable new multi-region, multi-satellite marine fog dataset, but the forecasting benchmark tables report internally impossible MSE/MAE pairs, so that half of the evaluation is not currently credible.","tokens_in":16258,"tokens_out":2476,"would_cite":true,"duration_ms":23380,"reading_group":"yes","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper introduces MFogHub, a dataset combining 15 coastal fog-prone regions and six geostationary satellites to benchmark and improve marine fog detection and forecasting models.","keywords":["marine fog detection","marine fog forecasting","multi-regional dataset","multi-satellite dataset","geostationary satellites","cube-stream structure","domain generalization","remote sensing benchmark"],"falsifier":"Take a random sample of the 11,600 pixel-level masks and have them re-annotated by an independent team of marine-fog meteorologists; if agreement is low, the benchmark rankings and generalization conclusions are unreliable. A complementary check is to compare fog/no-fog labels against lidar ceilometer or visibility station records at the same times and locations.","tokens_in":15174,"feed_emoji":"🌫️","tokens_out":8351,"duration_ms":70540,"temperature":0.7,"pith_summary":"Marine fog studies have been held back by small datasets drawn from one coastline and one satellite, making it hard to tell whether a deep learning model has truly learned fog or just memorized a local scene. This paper introduces MFogHub, a dataset that combines annotated marine fog observations from 15 coastal fog-prone regions and six geostationary satellites into over 68,000 samples, organized into 21 'cube-streams' (timestamp–spectral band–latitude–longitude). The paper argues that this breadth is the point: by benchmarking sixteen baseline models, MFogHub shows that a model's fog-detection and forecasting skill fluctuates with the region and the satellite, and that training on multiple regions improves generalization. If the dataset is sound, it gives the community a shared testbed for building and evaluating fog models that work globally rather than locally.","feed_headline":"First global marine-fog dataset spans 15 regions and 6 satellites","feed_subtitle":"68,000 satellite samples reveal how fog models generalize across oceans and sensors.","key_machinery":"The load-bearing object is the cube-stream, a four-dimensional data structure $R^{T\\times C\\times H\\times W}$ (timestamp, spectral band, latitude, longitude) that packages each region-satellite pair into a spatiotemporally aligned stream. The cube-stream does two jobs: it makes the 21 streams sliceable along any axis, so users can assemble custom sub-datasets by region, satellite, time, or spectral band; and it links detection (a single timestamp) and forecasting (a sequence of timestamps) from the same underlying data. The dataset's selection pipeline is the second piece of machinery: millions of shipboard weather observations are gridded and tallied to locate the 15 fog-prone coastal regions, and multi-spectral Level-1 data from the six satellites are projected and resampled to a common 1-km, 1024×1024 grid.","core_discovery":"The central claim is that MFogHub is the first dataset that deliberately spans both multiple marine-fog regions and multiple geostationary satellites, and that this joint coverage lets researchers measure, rather than assume, how well fog models generalize. The paper substantiates this with 68,000 samples from 15 regions and six satellites, 21 cube-streams, 693 fog events, and 11,600 pixel-level masks annotated by meteorological experts. Benchmarking eight detection and eight forecasting baselines, it finds that model rankings shift across regions (e.g., DlinkViT and ABCNet are strong in the California Current and Gulf of Alaska but weaker in Baja California) and across satellites (e.g., SimVP-v2 jumps to first place on Fengyun-4A data while PredRNN dominates on Himawari-8/9), and that models trained on combined regions consistently beat single-region training. The paper interprets these patterns as evidence that single-region, single-satellite benchmarks can misrepresent a method's true capability.","pith_inferences":["The cube-stream format can be reused for other spatiotemporal geophysical phenomena (aerosols, low clouds, sea ice) with minimal adaptation, since the slicing and multi-sensor alignment logic is generic.","By pairing overlapping satellites over the Yellow and Bohai Seas, MFogHub accidentally creates a ready-made domain-adaptation benchmark for remote sensing.","The 2:1 positive-to-negative ratio recommendation likely transfers to other segmentation tasks with rare foreground classes, but this is an untested extrapolation.","The regional splits could be used to test whether identical spectral signatures correspond to the same fog labels across regions, linking the dataset to physical fog-formation studies."],"forward_implications":["Single-region fog models cannot be trusted to generalize; MFogHub provides the first shared testbed for measuring that gap.","Pooling data across regions raises cross-region performance for most architectures, making multi-region training the expected default for future fog models.","Forecasting models generalize across regions more consistently than detection models, but satellite choice still changes model rankings.","Natural-color band combinations (0.64, 0.86, 3.9 µm) outperform true-color three-band inputs, giving a concrete band-selection rule for constrained deployments.","A positive-to-negative sample ratio of 2:1 is recommended for fog detection training sets, as it balances missed detections and false alarms."],"supporting_citations":[{"why":"Shipboard weather observations that are gridded and tallied to select the 15 fog-prone coastal regions.","marker":"[25]"},{"why":"Prior work that introduced the cube-stream data organization on which MFogHub's timestamp-spectral band-latitude-longitude structure is built.","marker":"[13]"},{"why":"Review of marine fog that grounds the geographic definition of fog-prone regions.","marker":"[14]"},{"why":"Companion review on marine fog formation and forecasting used to justify the phenomenon's importance.","marker":"[15]"},{"why":"Existing GOES-based single-region sea fog detection dataset that MFogHub extends across regions and satellites.","marker":"[20]"},{"why":"One of the existing single-region sea fog datasets against which MFogHub positions its multi-region coverage.","marker":"[2]"}],"fun_headline_variants":["First multi-region, multi-satellite fog dataset maps generalization","68,000 fog samples from 15 regions, 6 satellites reveal model shifts","Fog model rankings shift across oceans and sensors, new dataset shows","Marine fog dataset: 15 regions, 6 satellites, 68k samples","New fog dataset benchmarks 16 models across 15 regions and 6 satellites"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The weakest link is the 11,600 pixel-level annotation masks: the paper says they were made by meteorological experts, but reports no annotation protocol, inter-annotator agreement, or independent validation, so every detection and generalization result inherits the accuracy of those labels.","fun_headline_variants_meta":{"raw":{"variants":["First multi-region, multi-satellite fog dataset maps generalization","68,000 fog samples from 15 regions, 6 satellites reveal model shifts","Fog model rankings shift across oceans and sensors, new dataset shows","Marine fog dataset: 15 regions, 6 satellites, 68k samples","New fog dataset benchmarks 16 models across 15 regions and 6 satellites"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000983,"raw_usage":{"total_tokens":4188,"prompt_tokens":977,"completion_tokens":3211,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":593,"completion_tokens_details":{"reasoning_tokens":3111}},"tokens_in":593,"tokens_out":3211,"duration_ms":23078,"temperature":1.0,"reasoning_tokens":3111,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:12:27.826147+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a random sample of the 11,600 pixel-level masks and have them re-annotated by an independent team of marine-fog meteorologists; if agreement is low, the benchmark rankings and generalization conclusions are unreliable. A complementary check is to compare fog/no-fog labels against lidar ceilometer or visibility station records at the same times and locations.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Shipboard weather observations that are gridded and tallied to select the 15 fog-prone coastal regions."},{"cited_title":"Mesogeos: A multi-purpose dataset for data-driven wildfire modeling in the mediterranean","cited_arxiv_id":null,"evidence_quote":"Prior work that introduced the cube-stream data organization on which MFogHub's timestamp-spectral band-latitude-longitude structure is built."},{"cited_title":"Marine fog: challenges and advancements in observations, modeling, and forecast- ing","cited_arxiv_id":null,"evidence_quote":"Review of marine fog that grounds the geographic definition of fog-prone regions."},{"cited_title":"Marine fog: A review","cited_arxiv_id":null,"evidence_quote":"Companion review on marine fog formation and forecasting used to justify the phenomenon's importance."},{"cited_title":"A probability-based daytime algorithm for sea fog detection using goes-16 imagery","cited_arxiv_id":null,"evidence_quote":"Existing GOES-based single-region sea fog detection dataset that MFogHub extends across regions and satellites."},{"cited_title":"Sea fog monitoring method based on deep learning satellite multi-channel image fusion (in chi- nese)","cited_arxiv_id":null,"evidence_quote":"One of the existing single-region sea fog datasets against which MFogHub positions its multi-region coverage."}],"review_version":1}