{"id":"53e3d231-cd58-4cfa-96c0-8a8ef8221b45","arxiv_id":"2412.06835","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"APS-LSTM combines FFT-based multi-period division, periodic and spatial self-attention, and LSTM encoding to improve flood flow forecasts on two real-world watershed datasets.","lead":"APS-LSTM is a flood forecasting model that splits rainfall and flow data into multiple periods using Fourier analysis, then applies attention within each period to learn spatial relationships. On two Chinese watersheds it reports lower average error than six baselines, though the gains are modest and reported without error bars.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The empirical superiority claim rests on single-seed runs with no error bars or significance tests; the reported average gains may be within run-to-run variability.","rationale":"The reader's weakest_assumption—that the adjacency matrix A is never defined—is a genuine reproducibility concern: Section III-A introduces G=(V,E,A) and Section IV-A uses the Laplacian of A, but the paper never explains how A is built. However, the graph enters only through the fixed Laplacian embedding, while the spatial self-attention itself is a graph-agnostic attention over stations; and the code is publicly available, so A could in principle be recovered. The empirical claim is the more load-bearing issue. The paper's 'superiority' is based on a single seed, point estimates in Table I, no error bars, and no significance tests. The advantages are small at some horizons and absent at T+1, so run-to-run variance or different baseline tuning could change the ranking. There is also no explicit statement that the temporal split is chronological, which matters for a forecasting task. All of this is concrete and testable: rerun with multiple seeds and check the split. The reader already returned CONDITIONAL; this stress-test supports that verdict rather than changing it, hence UNCHANGED.","tokens_in":9020,"tokens_out":3472,"duration_ms":35366,"concrete_test":"Run all seven models on both basins with at least 10 random seeds under identical train/validation/test splits; report mean and standard deviation for RMSE, MAE, and MAPE, plus a paired significance test (e.g., Wilcoxon signed-rank over per-sample errors or a bootstrap over test windows). Also inspect the released code to verify whether the split is chronological and whether any training and test windows overlap in time. If the average RMSE advantage over the second-best baseline is not significant at p<0.05 or is not replicated across seeds, the 'superiority' claim in Section V-D should be downgraded.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (Section V-D) is that APS-LSTM 'performs best on average' against six baselines on two real-world basins. The evidence is Table I, but Section V-C reports only that 'The random seed is set to 2' and gives no standard deviations, confidence intervals, or repeated trials. The average RMSE gains are modest (4.1% on TunXi, 11.3% on ChangHua), and at T+1 APS-LSTM is not the best model on either dataset (e.g., TunXi T+1 RMSE 21.58 vs. DLinear's 15.31). With a single seed and no significance test, the ranking could be driven by initialization variance or by undocumented baseline hyperparameter choices. Moreover, the paper states the dataset is 'split into 80% training set, 5% validation set and 15% test set' but does not say the split is chronological; if it is random, overlapping sliding windows could leak future information into training and inflate all models' apparent skill. These evaluation gaps directly threaten the empirical claim, independently of the unspecified adjacency matrix construction.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes APS-LSTM, a flood-forecasting model that combines multi-periodicity discovery via Fast Fourier Transform (FFT), periodic and spatial self-attention modules, and adaptive aggregation of period-wise representations, followed by LSTM encoding and a linear decoder. The method is evaluated on two Chinese watershed datasets (TunXi and ChangHua) against six baselines (LSTM, GRU, DLinear, STA-LSTM, GraphWaveNet, TimesNet). The central claim, stated in Section V-D, is that APS-LSTM performs best on average across the 6-hour forecasting horizon, with reported average RMSE reductions of 4.1% (TunXi) and 11.3% (ChangHua) over the second-best model. The paper also includes an ablation study and qualitative visualizations of spatial and periodic attention.","tokens_in":9244,"tokens_out":3393,"duration_ms":30059,"significance":"If the empirical claim holds, APS-LSTM offers a novel combination of FFT-based multi-period division with spatial self-attention for hydrological forecasting, and the paper's code release (github.com/oopcmd/APS-LSTM) is a concrete reproducibility asset. I found no circularity: the FFT amplitude weights used for aggregation are computed from the input window, not from the test target, and the evaluation is on held-out data. The significance is nevertheless bounded by the current experimental rigor: single-seed results, no error bars, an unspecified graph construction, and an unspecified split protocol could all affect the reported ranking. The architectural ideas are interesting and the paper is clearly written, but the evidence needs strengthening before the superiority claim can be accepted.","major_comments":[{"comment":"The central claim that APS-LSTM 'performs best on average' is supported only by single-seed results (Section V-C states 'The random seed is set to 2') with no standard deviations, confidence intervals, or significance tests. The reported average RMSE gains are modest (4.1% on TunXi, 11.3% on ChangHua), and at T+1 on TunXi APS-LSTM is substantially worse than DLinear (RMSE 21.58 vs. 15.31). Without repeated trials, the average ranking could be within run-to-run variability. Please report multiple seeds with error bars and, where feasible, statistical significance tests.","section":"Section V-C / Table I"},{"comment":"The graph G=(V,E,A) is introduced as a predefined input, and both the Laplacian embedding (Section IV-A) and the spatial self-attention (Section IV-B.2) depend on the adjacency matrix A, but the paper never states how A is constructed. This makes the method irreproducible and leaves open whether the reported gains transfer to basins without a reliable graph. Please specify the graph construction (e.g., distance thresholds, k-nearest neighbors, or basin topology) and any normalization applied.","section":"Sections III-A and IV-A"},{"comment":"The dataset split is described as '80% training set, 5% validation set and 15% test set' but it is not stated whether the split is chronological. For time series with overlapping sliding windows, a random split can leak future information into the training set and inflate the apparent skill of all models. Please state explicitly that the split is temporal and that no test-set statistics are used in preprocessing (the Min-Max normalization in Eq. (12) already uses only the training set, which is good).","section":"Section V-C"},{"comment":"The MAPE metric masks out flow values less than 1, but this threshold is introduced ad hoc and is not justified. Since MAPE is one of the three headline metrics and the reported ranking changes across metrics, please justify the threshold (e.g., by the physical range of the flow data) and discuss its influence on the MAPE comparisons.","section":"Section V-C / Eq. (11)"},{"comment":"No hyperparameter details are given for the six baselines, such as hidden dimensions, number of layers, learning rates, number of epochs, or early-stopping criteria. Without evidence that the baselines were tuned to a comparable degree, the claim of superiority over them is not yet established. Please provide the baseline configurations or explicitly state that the original published settings were used.","section":"Section V-C"}],"minor_comments":[{"comment":"The notation is inconsistent: Eq. (2) uses ⊙ between weight matrices and the concatenated input [h_{t-1}, x_t], while Eq. (3) defines ⊙ as element-wise multiplication. The LSTM equations should use matrix multiplication (or an explicit concatenated weight matrix notation) to avoid confusion.","section":"Eq. (2) and Eq. (3)"},{"comment":"The subscript in 'Wf1 , · · ·, Wf2' appears to be a typo; the last index should be k, giving 'W_f1, ..., W_fk'.","section":"Eq. (10)"},{"comment":"The word 'Ubantu' should be 'Ubuntu'.","section":"Section V-C"},{"comment":"The notation X_t ∈ R^{1×N} and X_t^i ∈ R^{1×1} is confusing, especially because the flow station is indicated by i=N but the target Y_{t+1} is a scalar. Please clarify the station indexing and the relationship between X and the target.","section":"Section III-A"},{"comment":"The set of edges E in G=(V,E,A) is defined but never used in the paper; either use it or remove it from the notation to avoid dead conceptual weight.","section":"Section III-A"}],"recommendation":"major_revision","confidential_remarks":"The code release is a strong asset, but the manuscript should confirm in the repository that the exact adjacency matrix and all hyperparameter settings for the baselines are available. The editor may also want to check whether the journal's scope is best served by the application framing (flood forecasting) given that the method is a general spatio-temporal sequence model; this does not affect my technical assessment."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read APS-LSTM. It adapts TimesNet's FFT-based period division to spatio-temporal flood forecasting, adding per-period spatial self-attention and an LSTM encoder. That combination is new for this application, and the paper does a few things right: two real watershed datasets, an ablation that separates the periodic and spatial attention modules, and code on GitHub. The reported average gains over six baselines are modest (about 4% RMSE on TunXi, 11% on ChangHua), and the authors are transparent that the model is not best at T+1 on either dataset. The architecture is a plausible composition of known pieces.\n\nThe soft spots are in the evaluation, not in the design. Single seed, no error bars, no significance tests. With gains that small, run-to-run variance could easily flip the ranking. The paper never explains how the adjacency matrix A is constructed, even though the graph Laplacian embedding and the spatial attention both depend on it. And the data split is described only as 80/5/15; it does not say the split is chronological. If it is random, overlapping sliding windows can leak future information into training and inflate all models' skill. These are fixable, but they need to be fixed before the central claim is solid.\n\nI also want to flag the MAPE mask: the paper masks out flows below 1 when computing MAPE. That is disclosed, which is good, but it makes the MAPE numbers not directly comparable with the baselines' reported MAPE if those baselines did not use the same mask. That is a smaller issue, but worth a footnote.\n\nThe T+1 weakness deserves more discussion. On TunXi, APS-LSTM's T+1 RMSE is 21.58 versus DLinear's 15.31, so the \"best on average\" claim is carried by T+3 and beyond. That does not invalidate the paper, but it dulls the \"superiority\" framing.\n\nShould a serious journal send this to reviewers? Yes. The empirical question is real, the method is clearly described, and the weaknesses are addressable with more careful experimentation. A reviewer should ask for multiple seeds, confidence intervals, a stated and justified adjacency matrix, and a clearly chronological split. This is an incremental but competent application paper that could be made solid.\n\nFor a reading group, it is a decent case study in evaluation pitfalls in applied time-series forecasting. I would not cite it in the next year unless the revised version closes the gap.","headline":"Competent incremental application of TimesNet-style period division to flood forecasting, with a real empirical question but evaluation gaps (single seed, unspecified graph, unclear split) that need fixing before the average gains are credible.","tokens_in":9749,"tokens_out":2152,"would_cite":false,"duration_ms":19896,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"APS-LSTM claims that splitting rainfall and flow into multiple FFT-detected period views, each with its own spatial attention, cuts average RMSE by 4.1% and 11.3% on two Chinese basins.","keywords":["flood forecasting","multi-periodicity","Fast Fourier Transform","spatial self-attention","LSTM","spatio-temporal prediction","hydrological data","adaptive aggregation"],"falsifier":"On a third basin with an independently validated station graph, run APS-LSTM against its second-best baseline and check the average RMSE gap; if the gap disappears or reverses, the advantage is dataset-specific. Alternatively, replace $A$ with a random or identity adjacency matrix and re-run: a large RMSE change would show the graph matters, while a negligible change would show spatial attention is not actually using it.","tokens_in":8839,"feed_emoji":"🌊","tokens_out":5920,"duration_ms":56243,"temperature":0.7,"pith_summary":"APS-LSTM is a flood-forecasting model that tries to show that hydrological time series should be treated as a mixture of several overlapping periods, not one fixed seasonality, and that rainfall-to-flow spatial relationships should be learned separately for each period view. The paper claims that using the Fast Fourier Transform to find the strongest frequencies, reshaping the series into period-versus-trend tensors, and applying periodic and spatial self-attention on each view, followed by amplitude-weighted aggregation, improves 1-to-6-hour-ahead flow prediction. On two real watersheds in China, it reports average RMSE reductions of 4.1% and 11.3% over the second-best baseline, with the advantage growing from the third hour onward. The wider point is that multi-periodicity and per-period spatial attention are complementary signals worth exploiting in flood forecasting.","feed_headline":"A multi-period attention LSTM cuts flood-forecast RMSE by up to 11.3%","feed_subtitle":"FFT-split periods sharpen 6-hour-ahead flood forecasts on two Chinese basins.","key_machinery":"The load-bearing object is the Adaptive Periodic and Spatial Self-Attention Block (APS-Block). It first uses FFT amplitudes to pick the top-$k$ frequencies and reshape the input into $k$ two-dimensional tensors whose axes are \"within a period\" and \"across periods\"; then it applies periodic self-attention with 2D convolution projections and spatial self-attention with 1D convolution projections inside each division; finally it sums the $k$ outputs with softmax weights derived from the FFT amplitudes. The block is stacked with residual connections, and its aggregated output is encoded by an LSTM and decoded by a linear layer, with a graph Laplacian embedding injected at the input.","core_discovery":"The paper's central claim, stated in Section V-D, is that APS-LSTM \"performs best on average compared to the other 6 baselines.\" Table I supports this with average RMSE, MAE, and MAPE on the TunXi and ChangHua basins, where APS-LSTM reduces average RMSE by 4.1% and 11.3% relative to the second-best model and leads all baselines from the third forecast hour onward. The ablation study reports that removing either the periodic or the spatial self-attention module raises average RMSE, and the case-study visualizations show attention weights shifting across period divisions and stations. The authors read these results as evidence that discovering multiple periods with FFT and attending to spatial structure within each period division captures information that single-period spatio-temporal models miss.","pith_inferences":["A test this paper does not run is applying APS-LSTM to a basin with weak periodicity, such as a flash-flood catchment dominated by single storm pulses; if the FFT division adds nothing there, the reported gains may be specific to basins with regular seasonal cycles.","Because the paper never specifies how the adjacency matrix $A$ is built, a concrete extension is to learn the graph from rainfall-to-flow cross-correlation or delay estimates, which would test whether the spatial gains are sensitive to graph quality.","The reported metrics are averaged over the whole test set, while the peak-flow case study is qualitative; evaluating peak error and time-to-peak across all flood events would sharpen the claim."],"forward_implications":["If the FFT period division is doing the work, models that assume a single fixed period should be systematically worse on basins with overlapping rainfall and flow cycles.","The slower RMSE growth from T+1 to T+6 implies that the advantage concentrates in longer-horizon predictions, where peak-flow timing errors normally grow.","Since both ablations hurt, periodic and spatial self-attention provide complementary gains, and a model with only one of them is expected to be inferior.","The approach produces interpretable attention maps, so it can be used to identify which rainfall stations matter at which period and lead time."],"supporting_citations":[{"why":"Supplies the FFT-based period division and amplitude-weighted aggregation that APS-LSTM adapts into multi-period spatial attention.","marker":"[16]"},{"why":"LSTM is the sequence encoder at the core of the model and also one of the baselines.","marker":"[2]"},{"why":"DLinear is a baseline representing trend and periodicity methods that the paper argues are insufficient.","marker":"[8]"},{"why":"Graph WaveNet is the spatial spatio-temporal baseline that APS-LSTM compares against.","marker":"[20]"},{"why":"STA-LSTM is the flood-specific attention baseline that the paper extends.","marker":"[18]"},{"why":"Laplacian eigenmaps provide the graph embedding used to inject spatial structure into the input.","marker":"[23]"},{"why":"Scaled dot-product attention is the mechanism behind the periodic and spatial self-attention modules.","marker":"[26]"}],"fun_headline_variants":["FFT-split periods sharpen flood forecasts by up to 11.3%","Multi-period LSTM beats baselines on flood prediction","Spatial attention trims flood forecast error by 11.3%","APS-LSTM: FFT-driven attention for better flood forecasts","Cut flood RMSE 11.3% with periodic attention"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the predefined station graph $G=(V,E,A)$ correctly encodes rainfall-to-flow spatial connections, yet the paper never explains how $A$ is constructed, so if that graph is wrong the spatial attention can amplify noise and the reported gains may not transfer.","fun_headline_variants_meta":{"raw":{"variants":["FFT-split periods sharpen flood forecasts by up to 11.3%","Multi-period LSTM beats baselines on flood prediction","Spatial attention trims flood forecast error by 11.3%","APS-LSTM: FFT-driven attention for better flood forecasts","Cut flood RMSE 11.3% with periodic attention"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000698,"raw_usage":{"total_tokens":3131,"prompt_tokens":901,"completion_tokens":2230,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":517,"completion_tokens_details":{"reasoning_tokens":2139}},"tokens_in":517,"tokens_out":2230,"duration_ms":14053,"temperature":1.0,"reasoning_tokens":2139,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T20:38:28.287230+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a third basin with an independently validated station graph, run APS-LSTM against its second-best baseline and check the average RMSE gap; if the gap disappears or reverses, the advantage is dataset-specific. Alternatively, replace $A$ with a random or identity adjacency matrix and re-run: a large RMSE change would show the graph matters, while a negligible change would show spatial attention is not actually using it.","supporting_citations":[{"cited_title":"Long short-term memory,","cited_arxiv_id":null,"evidence_quote":"LSTM is the sequence encoder at the core of the model and also one of the baselines."},{"cited_title":"Are transformers effective for time series forecasting?","cited_arxiv_id":null,"evidence_quote":"DLinear is a baseline representing trend and periodicity methods that the paper argues are insufficient."},{"cited_title":"Interpretable spatio- temporal attention lstm model for flood forecasting,","cited_arxiv_id":null,"evidence_quote":"STA-LSTM is the flood-specific attention baseline that the paper extends."}],"review_version":1}