STAN: Smooth Transition Autoregressive Networks

Hugo Inzirillo; Remi Genet

REVIEW 3 major objections 5 minor 31 references

STAN: Smooth Transition Autoregressive Networks

T0 review · 3 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper proposes STAN, a neural network with STAR-style logistic gates, and reports it beats MLP, GRU, and LSTM baselines on 1-hour-ahead forecasts across most energy regions.

desk verdict A gated MLP presented as a neural STAR model, with the gate input undefined and the STAR baseline missing from the benchmark. read the letter →

arxiv 2501.18699 v1 pith:W2FW5EEE submitted 2025-01-30 cs.LG

Hugo Inzirillo , Remi Genet This is my paper · ORCID

classification cs.LG MSC 62M1068T07

keywords smoothtransitionautoregressiveregimeswitchingtimeseriesforecastingneuralnetworkslogisticfunctionenergyloaddeeplearningSTAR

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Traditional Smooth Transition Autoregressive (STAR) models capture regime changes through smooth transitions driven by a threshold variable, but they require the transition function and variables to be specified in advance. This paper proposes STAN, a deep neural network whose layers each contain a learned logistic gate, mimicking STAR's gradual switching between regimes. On a 12-region hourly energy consumption dataset, STAN achieves the best 1-hour-ahead RMSE in 8 of 12 regions, with gains over linear regression of up to roughly 37 percent, and consistently beats MLP baselines at all horizons. The authors argue this makes STAN a scalable, data-driven alternative to STAR for economic and financial forecasting, with transition behaviour that is learned rather than pre-specified.

What carries the argument

The load-bearing object is the STAN layer: a linear autoregressive path $\phi^{(l)}_i \tilde{y}^{(l)}_i$ plus a nonlinear path $\theta^{(l)}_i \mathrm{ReLU}(\tilde{y}^{(l)}_i)$ scaled by the learned logistic transition function $G(z_i; \gamma^{(l)}_i, c^{(l)}_i) = 1/(1+\exp(-\gamma^{(l)}_i(z_i-c^{(l)}_i)))$. Each unit in each layer carries its own transition speed $\gamma$ and threshold location $c$, so the network can discover multiple smooth switches across layers. This is the proposed neural analogue of the multiple transition functions in MRSTAR.

What would settle it

Generate data from a known two-regime LSTAR model with a specified transition variable, fit STAN to it, and check whether the learned thresholds $\hat{c}^{(l)}_i$ and speeds $\hat{\gamma}^{(l)}_i$ recover the generating transition function; alternatively, replace $z_i$ in the gate with random noise and measure test RMSE. If accuracy does not degrade or the thresholds do not track the true regime indicator, the STAR-like mechanism is not doing the work.

Watch

Extended reading notes

Core claim

The central claim is that a STAN layer of the form $\hat{y}^{(l)}_i = \phi^{(l)}_i \tilde{y}^{(l)}_i + \theta^{(l)}_i \mathrm{ReLU}(\tilde{y}^{(l)}_i) G(z_i; \gamma^{(l)}_i, c^{(l)}_i)$, where $G$ is the logistic function $1/(1+\exp(-\gamma_i(z_i-c_i)))$, reproduces the smooth regime-switching behaviour of STAR when stacked into a network. The paper reports that on an hourly energy benchmark, both STAN variants beat their MLP counterparts at every horizon and produce the best 1-hour-ahead results in 8 of 12 regions. The authors interpret this as evidence that a neural architecture can serve as a flexible replacement for STAR, retaining regime-switching structure while learning transitions from data.

Load-bearing premise

The load-bearing premise is that stacking layers with learned logistic gates reproduces the regime-switching behaviour of STAR models, but the paper never defines the transition variable $z_i$ that every gate depends on; if $z_i$ has no meaningful interpretation, the architecture is just a gated MLP and the claimed link to STAR is not established.

Editorial extensions

If this is right

STAN beats all tested baselines at the 1-hour horizon, achieving the best RMSE in 8 of 12 regions of the energy dataset and cutting RMSE by up to roughly 37 percent relative to linear regression.
Both STAN-3000-3 and STAN-3000-4 outperform their same-depth MLP counterparts at every horizon (1, 6, and 12 hours), indicating the transition gate, not just model width, drives the improvement.
STAN trains much faster than GRU and LSTM models despite having about 18 million parameters, because its feed-forward structure avoids recurrent loops.
At the 12-hour horizon the GRU-300-3 baseline wins in more regions than STAN, so STAN's regime-switching advantage concentrates in short-term forecasting.
The success of the STAR analogy suggests that structure from classical econometric time-series models can be encoded directly into deep learning architectures.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper never defines the transition variable $z_i$ that every gate depends on; a direct test is to set $z_i$ to a lagged level of the series, as in STAR, and check whether the learned thresholds $\hat{c}_i^{(l)}$ match economically meaningful levels, which would confirm the gate is doing regime detection rather than acting as a generic nonlinearity.
Because the comparison is against plain MLPs that lack the gate entirely, an ablation that fixes $G=1$ (removing the gate) would isolate exactly how much of the gain comes from the STAR-style mechanism versus the extra parameters in the gating path.
A quantitative bridge to the STAR literature would be to simulate data from a known two-regime LSTAR model and verify that STAN recovers the generating transition function's smoothness and threshold parameters; the paper's current experiments only measure predictive RMSE.
The univariate design and the absence of covariates limit the comparison to a single energy benchmark; extending STAN to multivariate inputs with a context variable, which the conclusion lists as future work, would test whether the claimed advantage survives in richer forecasting settings.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things you should know. The paper's central claim—that STAN is a powerful neural alternative to STAR—is not backed by the evidence, because the benchmark never includes a STAR model and the transition variable z_i in Eqs. 9–10 is never defined. Second, the stress-test note is right: without z_i, the architecture cannot be implemented or falsified, and the STAR analogy is cosmetic.

Credit where due: the related-work section is a competent survey of STAR, MRSTAR, and related extensions; the experimental appendix reports training time, parameter counts, and run-to-run standard deviations; and Section 5 plainly admits that the theoretical connection is future work. The empirical observation that a wide feed-forward network with gated ReLU units beats plain MLPs at the one-hour horizon on this energy dataset is plausible and mildly useful.

The problems are not minor. The 80/20 split is random, not temporal, which leaks information in a forecasting task. RMSE differences are often in the third decimal, no significance tests are given, and the standard deviations in Table 4 are large enough to erase several claimed wins. The absent STAR baseline means the stated headline comparison never happens. And Eq. 9 is a gated linear unit; the STAR vocabulary is interpretation, not mechanism. If z_i were defined as a lagged level, the link to STAR would be tangible, but as written it is a placeholder.

Who the paper is for: an applied forecaster might take away a small tip about gated MLPs; an econometrician who works on regime-switching will not be convinced. I would not send this to peer review as is. The authors could make a publishable applied paper by defining z_i, using a temporal split, adding a real STAR baseline and a significance test, and releasing code. If they do that, I would look again. On the current submission, desk rejection is the proportionate outcome.

Referee Report

3 major / 5 minor

Summary. The paper proposes STAN (Smooth Transition Autoregressive Networks), a feed-forward neural network whose layers are designed as an analogy to smooth transition autoregressive (STAR) models. Each layer combines a linear autoregressive term with a nonlinear term gated by a logistic transition function. The authors evaluate two STAN variants against linear regression, MLP, GRU, and LSTM baselines on the PJM hourly energy consumption dataset for 1-, 6-, and 12-hour-ahead forecasting, reporting RMSE, training time, and parameter counts. The paper concludes that STAN excels at short-horizon forecasting and is computationally efficient, and it suggests that neural networks can be a powerful alternative to STAR models. However, the transition variable in the defining equations is never specified, the empirical evaluation uses a random rather than temporal split, and no STAR model is included as a baseline.

Significance. If the architecture were fully specified and rigorously evaluated, the idea of embedding learnable smooth-transition mechanisms into deep networks could be a useful bridge between classical econometric STAR models and modern deep learning, particularly for regime-dependent time series. The paper currently provides no formal theory, no code, and no STAR baseline, so the claimed contribution is not yet established. The most valuable aspect is the conceptual direction, but the manuscript as written does not substantiate its central claims.

major comments (3)

[Section 3.1, Eqs. (9)-(10)] The transition variable z_i is never defined. It is not specified as a lagged value of the series, a linear combination of inputs, a learnable parameter, or any other quantity. In a STAR model, z_{t-d} is an observable lagged variable that gives the regimes economic meaning; without an analogous definition, Equations (9) and (10) cannot be implemented, and the gate could be a constant, a free parameter, or a function of nothing. The claimed analogy to STAR is therefore unverifiable, and Section 5 confirms that the theoretical connection is not formalized. This is load-bearing because the model's novelty and identity depend entirely on being a neural STAR analog.
[Section 4.1] The experimental protocol states that "we randomly split the data into 80% training and 20% testing sets." For multi-horizon time series forecasting with overlapping lookback windows, a random split is invalid: it creates temporally interleaved training and test observations, so future information leaks into training through overlapping windows. This makes the RMSE comparisons in Table 1 unreliable. The authors should use a temporal split (e.g., first 80% of the timeline for training and the last 20% for testing) or a rolling-origin evaluation.
[Section 4.2 and Table 1] No STAR model is included among the baselines, despite the abstract claiming that STAN is "a powerful alternative to STAR models." Without estimating a standard STAR (e.g., LSTAR or MRSTAR) on the same data and horizons, the central claim of the paper is unsupported. Additionally, the performance differences between STAN and the MLP/GRU baselines in Table 1 are often at the third decimal place (e.g., 0.089 vs. 0.090 for AEP at the 1-hour horizon), while the standard deviations reported in Table 4 (e.g., 3.8e-2 for the same entry) are of the same order of magnitude. No significance tests, confidence intervals, or paired comparisons are reported, so the claim that STAN "consistently outperforms" is not statistically substantiated.

minor comments (5)

[Section 2, Eq. (1)] The STAR equation contains indexing errors: the summation should be over the lag index with terms φ_i y_{t-i} and θ_i y_{t-i} G(z_{t-d}; γ; c), and the notation for the transition-function parameters is inconsistent ("γ; c" vs. "γ, c").
[Section 3.1, Eq. (11)] Equation (11) has a missing closing bracket after the G(...) term, and the quantities w_{\tilde{y}_\tau} and b_{\tilde{y}_\tau} are not defined.
[Section 4.1] The lookback window notation "max(45, 5 × n ahead)" is ambiguous because "n ahead" is not defined as a variable; the paper should use a consistent symbol such as H or τ for the forecast horizon.
[Table 1] Table 1 is extremely wide and difficult to read; splitting it into separate panels per horizon or transposing the layout would substantially improve readability.
[General] The manuscript contains numerous typos, missing accents, and formatting artifacts (e.g., "mode ls", "R´ emi", "oﬀer", and the nonstandard radical notation in Eq. (12)). A thorough copyedit is needed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; STAN's empirical comparisons are held-out and its STAR analogy, though underspecified, is not derived from its own outputs.

full rationale

The paper's predictive claims rest on RMSE computed on held-out test splits (Section 4.1 and Table 1) for trained weights phi, theta, gamma, and c. Those parameters are fitted only on training data, so the reported 'best performance in 8 out of 12 regions' is not a fitted input renamed as a prediction. The self-citations ([2], [3], [9]) appear only as background in the introduction and related work, e.g., 'Recently we also proposed a Temporal KANs (TKANs) to forecast time series [3]', and they do not carry the STAN argument; no uniqueness theorem or ansatz is imported from those papers. The main substantive weakness is different from circularity: the threshold variable z_i in Eqs. (9)-(10) is never defined or estimated, and the conclusion itself admits that 'formalizing the theoretical connections between STAN's transition mechanism and traditional STAR models' remains future work. That means the STAR analogy is not formally established, but the absence of a definition is an underspecification and reproducibility defect, not a reduction of the output to the input by construction. Consequently, no circular step meeting the quoted-equation standard is present.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The model is a standard gated feed-forward network whose learned weights are fitted by gradient descent. The novel parts are the STAR analogy and the gate structure; neither is derived from first principles. The main hidden assumptions are the existence of a meaningful transition variable z_i, the equivalence of stacked gated layers to STAR, and the validity of a random temporal split for forecasting evaluation.

free parameters (2)

STAN hidden layer width = 3000
Chosen by experimentation as optimal in Section 4.1; not derived from STAR structure.
STAN depth = 3 and 4 layers
Two variants are tested without a principled selection rule beyond empirical tuning.

assumptions (5)

standard math Universal approximation of multilayer feedforward networks justifies using a neural network for nonlinear time series.
Invoked in Section 1 via Hornik [8].
domain assumption A logistic function G(z; gamma, c) with learnable gamma and c reproduces the smooth regime switching of STAR.
Stated in Section 3.1; no theorem connects Eq. 10 to Eq. 1.
ad hoc to paper Each STAN layer has a well-defined threshold variable z_i.
Eq. 10 requires z_i, but Section 3.1 never defines it.
ad hoc to paper Stacking gated layers of Eq. 9 preserves or improves STAR-like regime structure.
Asserted in Section 3.1 and the Conclusion; formalization is deferred to future work in Section 5.
ad hoc to paper Random 80/20 split provides a valid evaluation of a forecasting model.
Used in Section 4.1; violates temporal ordering of the series and is load-bearing for the reported RMSE.

how reviews work

0 comments

Cite this review

Pith. "Pith review of STAN: Smooth Transition Autoregressive Networks." pith.science (2026). https://pith.science/paper/W2FW5EEE

@misc{pith2026250118699,
  author       = {Pith},
  title        = {Pith review of: STAN: Smooth Transition Autoregressive Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/W2FW5EEE}},
  note         = {Machine review of arXiv:2501.18699}
}

read the original abstract

Traditional Smooth Transition Autoregressive (STAR) models offer an effective way to model these dynamics through smooth regime changes based on specific transition variables. In this paper, we propose a novel approach by drawing an analogy between STAR models and a multilayer neural network architecture. Our proposed neural network architecture mimics the STAR framework, employing multiple layers to simulate the smooth transition between regimes and capturing complex, nonlinear relationships. The network's hidden layers and activation functions are structured to replicate the gradual switching behavior typical of STAR models, allowing for a more flexible and scalable approach to regime-dependent modeling. This research suggests that neural networks can provide a powerful alternative to STAR models, with the potential to enhance predictive accuracy in economic and financial forecasting.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

31 extracted references · 27 canonical work pages

[1]

Binkowski, G

M. Binkowski, G. Marti, and P. Donnat. Autoregressive convolut ional neural networks for asynchronous time series. In International Conference on Machine Learning , pages 580–589. PMLR, 2018

work page 2018
[2]

Genet and H

R. Genet and H. Inzirillo. A temporal kolmogorov-arnold transfo rmer for time series forecasting. arXiv preprint arXiv:2406.02486 , 2024

arXiv 2024
[3]

Genet and H

R. Genet and H. Inzirillo. Tkan: Temporal kolmogorov-arnold net works. arXiv preprint arXiv:2405.07344 , 2024

arXiv 2024
[4]

C. W. Granger and N. R. Swanson. An introduction to stochastic unit-root processes. Journal of Econometrics , 80(1):35–62, 1997

work page 1997
[5]

J. D. Hamilton. State-space models. Handbook of econometrics , 4:3039– 3080, 1994

work page 1994
[6]

Hatanaka

M. Hatanaka. Time-series-based econometrics: unit roots and co- integrations. OUP Oxford, 1996

work page 1996
[7]

Hochreiter and J

S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural com- putation, 9(8):1735–1780, 1997

work page 1997
[8]

K. Hornik. Approximation capabilities of multilayer feedforward ne tworks. Neural Networks, 4(2):251–257, 1991. ISSN 0893-6080. 11

work page 1991

Show all 31 references

[9]

Inzirillo

H. Inzirillo. Deep state space recurrent neural networks for t ime series forecasting. arXiv preprint arXiv:2407.15236 , 2024

2024 arXiv
[10]

Johansen

S. Johansen. Estimation and hypothesis testing of cointegrat ion vectors in gaussian vector autoregressive models. Econometrica: journal of the Econometric Society, pages 1551–1580, 1991

1991
[11]

C.-J. Kim. Dynamic linear models with markov-switching. Journal of econometrics, 60(1-2):1–22, 1994

1994
[12]

G. Koop, M. H. Pesaran, and S. M. Potter. Impulse response a nalysis in nonlinear multivariate models. Journal of econometrics , 74(1):119–147, 1996

1996
[13]

H.-M. Krolzig. Markov-switching vector autoregressions: Modelling, sta - tistical inference, and application to business cycle anal ysis, volume 454. Springer Science & Business Media, 2013

2013
[14]

L. Li, J. Yan, X. Yang, and Y. Jin. Learning interpretable deep s tate space model for probabilistic time series forecasting. arXiv preprint arXiv:2102.00397, 2021

2021 arXiv
[15]

B. Lim, S. ¨O. Arık, N. Loeﬀ, and T. Pﬁster. Temporal fusion transform- ers for interpretable multi-horizon time series forecasting. International Journal of Forecasting, 37(4):1748–1764, 2021

2021
[16]

Z. Liu, Y. Wang, S. Vaidya, F. Ruehle, J. Halverson, M. Soljaˇ ci ´ c, T. Y. Hou, and M. Tegmark. Kan: Kolmogorov-arnold networks. arXiv preprint arXiv:2404.19756, 2024

2024 arXiv
[17]

Lundbergh, T

S. Lundbergh, T. Ter¨ asvirta, and D. Van Dijk. Time-varying smooth tran- sition autoregressive models. Journal of Business & Economic Statistics , 21(1):104–121, 2003

2003
[18]

L¨ utkepohl

H. L¨ utkepohl. Vector autoregressive models. InHandbook of research meth- ods and applications in empirical macroeconomics , pages 139–164. Edward Elgar Publishing, 2013

2013
[19]

Ma and H

K. Ma and H. Leung. A novel lstm approach for asynchronous m ultivariate time series prediction. In 2019 International Joint Conference on Neural Networks (IJCNN) , pages 1–7. IEEE, 2019

2019
[20]

B. N. Oreshkin, D. Carpov, N. Chapados, and Y. Bengio. N-bea ts: Neural basis expansion analysis for interpretable time series forecasting. arXiv preprint arXiv:1905.10437, 2019

1905 arXiv
[21]

S. S. Rangapuram, M. W. Seeger, J. Gasthaus, L. Stella, Y. Wa ng, and T. Januschowski. Deep state space models for time series forecas ting. Ad- vances in neural information processing systems , 31, 2018. 12

2018
[22]

Salinas, V

D. Salinas, V. Flunkert, J. Gasthaus, and T. Januschowski. De epar: Prob- abilistic forecasting with autoregressive recurrent networks. International journal of forecasting , 36(3):1181–1191, 2020

2020
[23]

Z. Tang, C. De Almeida, and P. A. Fishwick. Time series forecastin g using neural networks vs. box-jenkins methodology. Simulation, 57(5):303–310, 1991

1991
[24]

Ter¨ asvirta

T. Ter¨ asvirta. Speciﬁcation, estimation, and evaluation of sm ooth transi- tion autoregressive models. Journal of the american Statistical association , 89(425):208–218, 1994

1994
[25]

Terasvirta and H

T. Terasvirta and H. M. Anderson. Characterizing nonlinearities in business cycles using smooth transition autoregressive models. Journal of applied econometrics, 7(S1):S119–S136, 1992

1992
[26]

Ter¨ asvirta, D

T. Ter¨ asvirta, D. Van Dijk, and M. C. Medeiros. Linear models, smooth transition autoregressions, and neural networks for forecast ing macroeco- nomic time series: A re-examination. International Journal of Forecasting , 21(4):755–774, 2005

2005
[27]

H. Tong. Non-linear Time Series: A Dynamical System Approach . Oxford University Press, 08 1990. ISBN 9780198522249

1990
[28]

Van Dijk and P

D. Van Dijk and P. H. Franses. Modeling multiple regimes in the busin ess cycle. Macroeconomic dynamics, 3(3):311–340, 1999

1999
[29]

Vaswani, N

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A . N. Gomez, /suppress L. Kaiser, and I. Polosukhin. Attention is all you need.Advances in neural information processing systems , 30, 2017

2017
[30]

Zhang, B

G. Zhang, B. Eddy Patuwo, and M. Y. Hu. Forecasting with artiﬁcial neural networks:: The state of the art. International Journal of Forecasting, 14(1): 35–62, 1998. ISSN 0169-2070. doi: https://doi.org/10.1016/S0169-2070(97) 00044-7

1998 doi
[31]

Zivot and J

E. Zivot and J. Wang. Vector autoregressive models for multiva riate time series. Modeling ﬁnancial time series with S-PLUS ®, pages 385–429, 2006. 13 Appendices model LinearRegression STAN-3000-3 STAN-3000-4 Linear MLP-3 000-3 MLP-3000-4 GRU-300-3 LSTM-300-3 steps ahead datas...

2006

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.

[1] [1]

Binkowski, G

M. Binkowski, G. Marti, and P. Donnat. Autoregressive convolut ional neural networks for asynchronous time series. In International Conference on Machine Learning , pages 580–589. PMLR, 2018

work page 2018

[2] [2]

Genet and H

R. Genet and H. Inzirillo. A temporal kolmogorov-arnold transfo rmer for time series forecasting. arXiv preprint arXiv:2406.02486 , 2024

arXiv 2024

[3] [3]

Genet and H

R. Genet and H. Inzirillo. Tkan: Temporal kolmogorov-arnold net works. arXiv preprint arXiv:2405.07344 , 2024

arXiv 2024

[4] [4]

C. W. Granger and N. R. Swanson. An introduction to stochastic unit-root processes. Journal of Econometrics , 80(1):35–62, 1997

work page 1997

[5] [5]

J. D. Hamilton. State-space models. Handbook of econometrics , 4:3039– 3080, 1994

work page 1994

[6] [6]

Hatanaka

M. Hatanaka. Time-series-based econometrics: unit roots and co- integrations. OUP Oxford, 1996

work page 1996

[7] [7]

Hochreiter and J

S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural com- putation, 9(8):1735–1780, 1997

work page 1997

[8] [8]

K. Hornik. Approximation capabilities of multilayer feedforward ne tworks. Neural Networks, 4(2):251–257, 1991. ISSN 0893-6080. 11

work page 1991