{"id":"2ca245ad-2769-46fd-abe1-e61195d39578","arxiv_id":"2507.21155","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"For sparse and low-velocity retail demand series, SPADE-S reduces quantile-forecast bias and loss compared to SPADE and MQTransformer baselines on three large internal datasets.","lead":"SPADE-S, a forecasting model from Amazon's SCOT Forecasting team, adds a multi-head convolutional encoder and a separate parametric arm for sparse series to the earlier SPADE model. On three internal demand datasets, it reports P90 quantile-loss gains of 2.2% to 6.6% overall and up to 15% on some magnitude subgroups.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Sparse arm's exponential ICDF contradicts the stated P50-zero rule, leaving the zero-category gains (e.g., D1 P90 -10.05%) dependent on an unstated clipping the paper never defines.","rationale":"I read the paper as a serious engineering contribution: it identifies a real failure mode (quantile-loss magnitude bias and convolutional under-dispersion on sparse inputs), proposes a specific architecture, and validates it with extensive internal ablations on very large datasets. The direction of the reported bias corrections matches the stated mechanism, and the ablation table (Table 3) shows the main model dominating the variants, which is independent supporting evidence. I found no reason to doubt the authenticity of the experiments.\n\nThe most load-bearing concern is the ambiguous specification of the SparseQuantileNetwork. Equation (4) and the accompanying P50-zero statement cannot both hold unless an unstated clipping rule is used, and the paper never defines how the exponential ICDF is reconciled with the zero rule. Because the affected series are the majority in D1 and D2, the headline gains in the zero category are not reproducible from the text. This is not a cosmetic issue: the 10.05% P90 improvement on D1 zero series could be an artifact of a hard-coded zero for q<=0.5 rather than the claimed exponential tail model. The paper also does not report the empirical zero frequency in the forecast horizon, which is the quantity that would justify the P50-zero rule.\n\nThe reader's weakest assumption identifies the same region, and I agree with the conditional verdict. This concern is local and addressable: state the clipping rule, provide code or a precise forward pass, and re-confirm the zero-category numbers. I do not see grounds for rejection, since the overall claims are supported by consistent ablations and the concern does not affect the nonzero-category gains on D2/D3.","tokens_in":14894,"tokens_out":9200,"duration_ms":88582,"concrete_test":"Run the trained SPADE-S sparse arm on a sample of D1 zero-category series and record the raw outputs for q=0.5 and q=0.9. If q=0.5 exactly equals 0 while q=0.9 equals -h ln(0.1), an unstated clipping is applied to equation (4); the manuscript must define this rule and the D1 zero-category P90 improvement (-10.05%) should be recomputed under the documented rule. If q=0.5 equals -h ln(0.5), the text's 'always forecast quantiles no more than P50 to be zero' is false and the reported zero-category results are misdescribed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that SPADE-S reduces sparsity-based bias rests on the SparseQuantileNetwork (Section 2.2). Equation (4) defines its quantile forecast as the exponential ICDF y_hat(q) = -h(span, theta) ln(1-q), which is strictly positive for every q in (0,1), including q=0.5. The paragraph immediately after states 'we always forecast quantiles no more than P50 to be zero.' These two specifications are mutually exclusive. If the implementation clips q <= 0.5 to zero, the effective model is a zero-inflated exponential whose zero-inflation probability is fixed at 0.5 rather than estimated; the paper does not state this. If the ICDF is used unmodified, P50 is positive, contradicting the text. Either way, the forward pass is not well specified by the manuscript. This matters because 63.7% of D1 and 90% of D2 series are routed to this arm, and the zero-category P90 improvement on D1 is 10.05% (Table 2). Without a precise definition of the zero rule, that gain cannot be attributed to the claimed exponential tail fit versus an arbitrary clipping rule. The paper also asserts without proof that any continuous distribution 'would result in P50 over-bias'; this only holds if the empirical zero mass exceeds 0.5, which the paper never reports for the forecast horizon.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript introduces SPADE-S, an extension of the SPADE forecaster, consisting of (i) a masked multi-head dilated causal convolutional encoder and (ii) a sparse quantile network that routes series with zero trailing-year demand to a parametric exponential arm, with the text stating that quantiles no more than P50 are forecast as zero. The authors claim that this design reduces magnitude- and sparsity-based systematic biases and improves P50/P90 weighted quantile loss relative to SPADE on datasets D1 and D2 and relative to MQTransformer on D3, with overall P90 gains of 2.21%, 6.58%, and 4.28% respectively, and larger gains on zero and slow-moving categories. The paper also includes a formal argument that common losses weight series by magnitude under equal relative error, a simulation demonstrating convolutional predictive collapse under sparsity, and an ablation ladder (V9–V19) on D1 with partial ablations on D2.","tokens_in":15149,"tokens_out":7878,"duration_ms":73126,"significance":"If the results hold, SPADE-S would provide a practical solution to the well-recognized problem of systematic degradation on low-magnitude and sparse series in large-scale retail demand forecasting. The magnitude-weighting observation (Appendix A.4) is correct under its stated assumptions, and the ablation study is unusually extensive for a paper built on proprietary data, giving internal consistency to the main model choice. The multi-head encoder's variance-reduction-by-ensembling intuition is plausible and computationally cheap. However, the central architectural mechanism—the sparse quantile network—is not precisely defined (see major comments), and the empirical support for the descriptor 'state-of-the-art' rests on only two baselines, one of which is the authors' own SPADE model, with no uncertainty quantification. These issues currently prevent the paper's headline claims from being fully verified.","major_comments":[{"comment":"The definition of the SparseQuantileNetwork is internally inconsistent. Equation (4) sets y_hat(q) = -h(span, theta) ln(1-q), which is strictly positive for every q in (0,1), including q = 0.5. The paragraph immediately after states that 'we always forecast quantiles no more than P50 to be zero.' The manuscript does not specify how these two specifications are reconciled: is the ICDF output clipped at q <= 0.5? Is the exponential actually a zero-inflated mixture with a fixed or learned zero mass? If the implementation clips, the effective model is a zero-inflated exponential with the zero-inflation probability fixed at 0.5, which is neither stated nor estimated; if it does not clip, the P50 forecast is positive and contradicts the text. This is load-bearing because 63.7% of D1 and 90% of D2 series are routed to this arm, and the headline zero-category P90 gain of 10.05% on D1 (Table 2) depends on the exact form of the zero rule. Please specify the forward pass precisely and, if clipping is used, justify the fixed 0.5 threshold or include it as a tunable or learned parameter.","section":"Section 2.2, Eq. (4)"},{"comment":"The claim that 'any continuous distribution would result in P50 over-bias' is asserted without proof and is only correct if the conditional probability of zero demand at the forecast horizon exceeds 0.5. The paper reports the fraction of series with zero aggregate demand over the trailing 52 weeks (Table 1), which is a routing criterion, not the empirical zero frequency in the forecast target. For series whose horizon-level zero probability is below 0.5, a continuous distribution can produce a P50 forecast that is unbiased or even negatively biased. The authors should report the empirical zero-demand frequency per horizon for the routed series and either prove the claim under explicit assumptions or qualify it. Without this, the justification for the P50-zero rule is incomplete.","section":"Section 2.2, sparse quantile network paragraph"},{"comment":"The 'state-of-the-art' claim is supported by a comparison with only two baselines: SPADE for D1 and D2 (the authors' own model) and MQTransformer for D3. No error bars, confidence intervals, or significance tests are reported, despite the fact that the overall improvements are modest (P50 gains of 0.77%–1.95%; P90 gains of 2.21%–6.58%). While the dataset sizes are enormous, the lack of any uncertainty quantification makes it impossible to assess whether the differences are stable across training runs or backtest windows. The authors should report repeated-run variance, backtest-window stability, or at minimum standard errors, and ideally include additional strong baselines (e.g., DeepAR, a Transformer baseline, or a dedicated intermittent-demand method) to substantiate the 'state-of-the-art' descriptor.","section":"Section 3.2 and Table 2"},{"comment":"The zero-category results in Table 3 show that the P50-zero rule reduces over-bias dramatically (V19: P50 over-bias -32.91%, P90 over-bias -35.40% relative to baseline) but at the cost of increased under-bias (P50 under-bias +1.64%, P90 under-bias +9.74%). The paper's discussion in Section 3.2 describes the gains as 'primarily driven by significant reductions in over-bias of the forecast,' without mentioning the under-bias increase. Because zero series constitute the large majority of D1 and D2, the aggregate quantile-loss improvement may largely reflect this over-bias/under-bias trade-off rather than a genuine improvement in tail calibration on sparse series. The authors should report the bias decomposition for all categories and discuss whether the increased under-bias is acceptable for the application.","section":"Table 3 (D1 ablations)"}],"minor_comments":[{"comment":"The abstract contains a typo: 'respectfully' should be 'respectively' in the phrase 'P50 forecast accuracy gains of 0.92%, 0.77%, and 1.95% respectfully.'","section":"Abstract"},{"comment":"The expression for the exponential scale parameter, written as h(span, theta[t]) = span / sigma * theta[t] (with the symbol sigma appearing as an unexpanded character), is ambiguous because sigma is not defined. Please clarify the intended formula, e.g., whether the scale is span * theta[t] or span / sigma * theta[t], and define all symbols.","section":"Section 2.2, Eq. (4)"},{"comment":"Footnote 1 states that D3 does not observe extreme holiday- or promotion-related spikes, so 'PeakAttention is unnecessary.' This implies that the D3 configuration of SPADE-S omits a component that is present in the D1/D2 configurations. Please state explicitly which architecture components are used in each dataset and whether the same model family is being compared across all three use-cases.","section":"Section 3.2, footnote 1"},{"comment":"In Table 4, the baseline V9 column is missing values in the 'All' rows (e.g., 'All P50ql' and 'All P90ql'). Since these are baseline values, they should be shown explicitly as 0% for clarity, rather than left blank.","section":"Appendix B.2, Table 4"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is from Amazon SCOT Forecasting, and for two of the three datasets the only baseline is the authors' own SPADE model. The internal inconsistency in the sparse-arm definition (Eq. (4) vs. the P50-zero rule) is a clear blocker for acceptance. The title's term 'foundational forecaster' may also overstate the generality, since all evaluations are in retail demand forecasting, but that is a scope consideration rather than a technical fault. The paper does not provide external reproducibility assets, which is understandable given proprietary data, but the absence of any uncertainty quantification on the headline numbers is a concern the editor may wish to weigh."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a credible, large-scale industrial forecasting paper that identifies a real failure mode—sparse/low-magnitude series get over-biased by conv encoders—and offers a specific fix: route zeros to a parametric exponential arm and use identical parallel dilated conv heads. The multi-head encoder is a modest variation on existing parallel-conv ideas, but the sparse-routing combination is genuinely new. The ablations (V9–V19) are extensive and internally consistent; the direction of the gains matches the hypothesis. Credit where due: 700M/100M/3M series, decomposition by velocity category, and consistent P90/P50 improvements.\n\nSoft spots. First, the sparse arm is under-specified. Equation (4) gives the exponential ICDF, which is positive for all q; the next paragraph says quantiles ≤ P50 are forecast as zero. That can be read as a clipping rule, but the paper never says 'clip' or defines the effective distribution as a zero-inflated exponential with fixed 0.5 zero mass. The forward pass needs one sentence of clarity. The claim that 'any continuous distribution would result in P50 over-bias' only holds when zero mass exceeds 0.5; that is true for the sparse series as defined (zero in trailing 52 weeks), so it's correct but needs the qualifier. Second, the headline 'outperforms existing state-of-the-art' rests on only two baselines—SPADE (self-cited) and MQTransformer—with no error bars, repeated runs, or significance tests. For a paper making SOTA claims at this scale, external baselines (Croston, DeepAR, Chronos, Moirai) and some uncertainty quantification are needed. Third, no code or data release; internal datasets are fine, but hyperparameters and architecture details are sparse.\n\nNone of this sinks the central claim. The paper is worth serious refereeing because the problem is real, the solution is plausible, and the internal evidence is substantial. The authors just need to tighten the specification, broaden the baselines, and add error bars. This is a paper for forecasting practitioners and researchers working on sparse-demand or heterogeneous-magnitude series. I'd give it a conditional accept with major revisions.","headline":"Credible industrial sparsity fix with a real architectural contribution, but the sparse-arm zero rule needs clarification and the SOTA claim needs broader baselines.","tokens_in":15802,"tokens_out":3600,"would_cite":true,"duration_ms":35247,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SPADE-S reduces magnitude- and sparsity-based bias in large-scale demand forecasting, cutting P90 losses by 2.21%, 6.58%, and 4.28% on three retail datasets.","keywords":["time series forecasting","sparse demand","quantile regression","probabilistic forecasting","convolutional encoder","demand forecasting","foundational model"],"falsifier":"Compute the empirical median of one-period-ahead demand among series with zero aggregate demand in the trailing 52 weeks in D1; if a non-negligible share of those series has a positive median, SPADE-S's invariant rule that all quantiles at or below P50 are zero is contradicted by the data it is designed to forecast.","tokens_in":1793,"feed_emoji":"📦","tokens_out":2686,"duration_ms":109747,"temperature":0.7,"pith_summary":"SPADE-S is a deep-learning architecture for large-scale product demand forecasting that tries to fix a systematic failure: existing quantile forecasters such as SPADE and MQTransformer predict poorly on low-magnitude and sparse series, and their aggregate scores hide this because high-volume series dominate the loss. The paper identifies the causes—quantile loss weights series by absolute magnitude, training sampling skews toward fast movers, and convolutional encoders collapse their prediction intervals on sparse inputs—and claims that its two additions, a masked multi-head dilated convolutional encoder and a SparseQuantileNetwork that routes zero-demand series to an exponential model arm, substantially reduce those biases. On three retail datasets spanning roughly 3 million to 700 million series, SPADE-S reports P90 overall forecast accuracy gains of 2.21%, 6.58%, and 4.28%, P50 gains of 0.92%, 0.77%, and 1.95%, and subgroup gains up to 10–15% on sparse and fast-moving categories. If these results hold, a single forecaster can serve both fast-moving and near-zero demand without sacrificing either, which matters as inventory systems move to granular, sparse forecasting.","feed_headline":"Sparse-demand forecast errors cut up to 15 percent","feed_subtitle":"A two-branch forecaster routes zero-demand series to an exponential arm, cutting P90 losses 2.2-6.6% on retail data.","key_machinery":"The load-bearing mechanism is a two-branch routing architecture. The main branch is a masked multi-head dilated causal convolutional encoder: several identical dilated causal convolution stacks run in parallel and are linearly combined, which the paper argues reproduces the uncertainty benefit of an ensemble without training separate models. The sparse branch is the SparseQuantileNetwork, which classifies a series as sparse when its trailing-52-week aggregate demand is zero, estimates an exponential scale from patch embeddings through an MLP, and produces horizon-specific quantiles through the closed-form inverse CDF; forcing all quantiles at or below P50 to zero is what prevents distributional collapse and P50 over-bias on zero-demand series. A short proof that quantile loss factors as $g(y)f(r)$ with $g(y)=y$ under equal relative error explains why unmodified architectures over-weight high-magnitude series and why the two-branch design is needed.","core_discovery":"The paper's central claim is that magnitude- and sparsity-related systematic bias in deep-learning quantile forecasters is largely an architectural problem, not an irreducible property of heterogeneous data, and that SPADE-S solves it with two mechanisms. First, a masked multi-head dilated causal convolutional encoder runs several identical dilated convolution stacks in parallel and combines them linearly, capturing the robustness of an ensemble with shared structure. Second, a SparseQuantileNetwork routes any series with zero aggregate demand in the trailing 52 weeks, and not classified as a new product, away from the main encoder; a patched MLP estimates the scale of an exponential distribution, and quantile forecasts come from the exponential ICDF $\\hat y^{(q)} = -h(\\operatorname{span},\\theta)\\ln(1-q)$, with all quantiles at or below P50 set to zero. Against SPADE on D1 and D2 and MQTransformer on D3, the model reports P90 gains of 2.21%, 6.58%, and 4.28% overall, P50 gains of 0.92%, 0.77%, and 1.95%, and subgroup P90 gains up to 10.05%, 14.80%, and 6.10% on the zero-demand and fast-moving categories where bias was worst.","pith_inferences":["Editorial inference: a direct test of the exponential assumption is to train the same sparse arm with a zero-inflated negative-binomial ICDF on the same routed subset; if P90 losses stay flat, the exponential choice is incidental, and if they improve, the parametric form is load-bearing rather than the routing alone.","Editorial inference: the magnitude-bias proof suggests a simpler control experiment with the SPADE baseline plus loss reweighting proportional to $1/\\sum y_{i,t,h}$; comparing that control to SPADE-S would isolate how much of the gain is architectural versus a correction to the objective.","Editorial inference: the universal rule that quantiles at or below P50 are zero implicitly claims the conditional median of sparse demand is exactly zero; in neighboring zero-inflated domains with occasional small positive demand, a data-dependent threshold would be needed, so the architecture likely transfers less cleanly than the headline percentages suggest.","Editorial inference: the multi-head convolution ensemble principle could be tested as a drop-in encoder replacement in other quantile-regression settings where distributional collapse is observed, such as intermittent energy or retail demand at different seasonal scales."],"forward_implications":["If the reported gains hold, unified foundational forecasters no longer need to sacrifice tail accuracy on slow movers to keep fast movers accurate; SPADE-S claims simultaneous improvements in both subgroups on D1 and D2.","The exponential ICDF with zero-forced low quantiles means a full probabilistic sparse forecast can be produced in a single forward pass with no sample-path generation, which is what makes training on hundreds of millions of series tractable.","Because P90 gains exceed P50 gains on all three datasets, the paper treats tail estimation as the main beneficiary of removing magnitude bias; inventory systems driven by service-level constraints depend directly on those P90 quantiles.","The D1 ablation supports the paper's claim that the sparse arm, rather than the adjusted sampling cutoff or multi-head encoder alone, drives the zero-category improvement: the full model beats the ablation variants on zero-category P90.","The D2 results attribute faster-category P90 gains, such as 14.80% for fast-moving series, to sparse routing preventing zero series from biasing higher-velocity forecasts downward."],"supporting_citations":[{"why":"SPADE, the baseline on D1 and D2 and the architecture SPADE-S builds on; supplies the base encoder, peak filtering, and the comparison numbers the gains are measured against.","marker":"[21]"},{"why":"MQTransformer, the baseline on D3; supplies the attention-based architecture and comparison dataset for the moderate-velocity use case.","marker":"[6]"},{"why":"MQCNN, the multi-horizon quantile forecaster whose convolutional encoder family underlies the collapse simulation and the main encoder design.","marker":"[20]"},{"why":"Multi-head attention paper; motivates the parallel multi-head convolution encoder as a cheaper way to obtain complementary input representations.","marker":"[19]"},{"why":"Bagging paper; supplies the variance-reduction rationale for combining multiple identical convolution heads in one encoder.","marker":"[4]"},{"why":"RevIN normalization paper; cited as poorly suited to distributional forecasting, motivating the paper's encoder design choice.","marker":"[8]"}],"fun_headline_variants":["SPADE-S cuts sparse-forecast errors up to 15%","Sparsity-robust forecaster SPADE-S lifts accuracy up to 15%","New forecaster targets sparse data, lifts demand forecasts up to 15%","Two-branch SPADE-S reduces sparse-bias, boosting forecast accuracy 15%"],"cache_read_input_tokens":17792,"weakest_assumption_plain":"The load-bearing premise is that sparse demand is exponential with a median of exactly zero, so the SparseQuantileNetwork's zero-forced P50 and closed-form tail quantiles describe real zero-demand series; if the true conditional distribution has a positive median or a heavier tail, the reported zero-category gains are largely an artifact of that parametric assumption.","fun_headline_variants_meta":{"raw":{"variants":["SPADE-S cuts sparse-forecast errors up to 15%","Sparsity-robust forecaster SPADE-S lifts accuracy up to 15%","New forecaster targets sparse data, lifts demand forecasts up to 15%","Two-branch SPADE-S reduces sparse-bias, boosting forecast accuracy 15%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000698,"raw_usage":{"total_tokens":3198,"prompt_tokens":1034,"completion_tokens":2164,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":650,"completion_tokens_details":{"reasoning_tokens":2076}},"tokens_in":650,"tokens_out":2164,"duration_ms":16745,"temperature":1.0,"reasoning_tokens":2076,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:08:37.818459+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the empirical median of one-period-ahead demand among series with zero aggregate demand in the trailing 52 weeks in D1; if a non-negligible share of those series has a positive median, SPADE-S's invariant rule that all quantiles at or below P50 are zero is contradicted by the data it is designed to forecast.","supporting_citations":[{"cited_title":"Olivares, Boris Oreshkin, Sunny Ruan, Sitan Yang, Abhi- nav Katoch, Shankar Ramasubramanian, Youxin Zhang, Michael W","cited_arxiv_id":null,"evidence_quote":"SPADE, the baseline on D1 and D2 and the architecture SPADE-S builds on; supplies the base encoder, peak filtering, and the comparison numbers the gains are measured against."},{"cited_title":"MQTransformer: Multi- Horizon Forecasts with Context Dependent and Feedback-Aware Attention","cited_arxiv_id":null,"evidence_quote":"MQTransformer, the baseline on D3; supplies the attention-based architecture and comparison dataset for the moderate-velocity use case."},{"cited_title":"A Multi-horizon Quantile Recurrent Forecaster","cited_arxiv_id":null,"evidence_quote":"MQCNN, the multi-horizon quantile forecaster whose convolutional encoder family underlies the collapse simulation and the main encoder design."},{"cited_title":"Reversible instance normalization for accurate time-series forecasting","cited_arxiv_id":null,"evidence_quote":"RevIN normalization paper; cited as poorly suited to distributional forecasting, motivating the paper's encoder design choice."}],"review_version":2}