REVIEW 2 major objections 4 minor 63 references
A tensor network approach for chaotic time series prediction
T0 review · 2 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read A tensor-network representation of the truncated Volterra series predicts chaotic time series more accurately and trains at least ten times faster than a conventional echo state network on a 70-task benchmark.
desk verdict Solid, reproducible application of a known tensor-network Volterra model to chaotic forecasting; the result is plausible, with one moderate numerical caveat. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the matrix product operator (MPO) decomposition of the Volterra design matrix $U$ and the coefficient matrix $H$, constructed from the repeated row-wise Khatri-Rao product $U = \tilde{U}^{\odot D}$, where $\tilde{U}$ is the $N \times (PM+1)$ matrix of delayed extended inputs. This exact representation avoids materializing the exponentially large $U$. Kernel symmetry bounds the rank of $U$ by $R = \binom{PM+D}{PM}$, and the least-squares solution is obtained through an SVD-based pseudoinverse computed in MPO form, which is what makes training feasible with only two hyperparameters, the maximum delay $M$ and maximum monomial degree $D$.
What would settle it
Take one chaotic task where the TN model underperforms the ESN, compute the symmetry error $S$ defined in the paper's appendix for the fitted kernel, then re-symmetrize the kernel and retest; if the gap closes, the central claim depends on SVD thresholding for correlated inputs rather than on the Volterra representation itself.
Extended reading notes
Core claim
The paper's central claim is that the tensor-network Volterra model outperforms a conventional echo state network for chaotic time-series prediction, in accuracy and in training cost. The argument is that the truncated Volterra series, which next-generation reservoir computing uses as its state space, has a symmetric kernel that can be stored and solved as a matrix product operator: each row of the design matrix is the vectorization of a symmetric tensor, so the full matrix is a repeated row-wise Khatri-Rao product and never needs to be materialized. The minimum-norm least-squares solution is symmetric in exact arithmetic regardless of the rank of the design matrix, which the paper proves in an appendix; numerically, however, highly correlated inputs can make the pseudoinverse break that symmetry, and the paper reports that some chaotic tasks show exactly this rank-deficiency behavior. On the 70-task benchmark the model still wins most short-term and long-term forecasting comparisons against the tuned ESN, with training times at least one order of magnitude smaller.
Load-bearing premise
The load-bearing premise is numerical: the pseudoinverse solution must keep the Volterra kernel symmetric in floating-point arithmetic for the correlated inputs that real chaotic systems produce, because the MPO compression is justified by that symmetry.
Editorial extensions
If this is right
- On the paper's 70-task benchmark, the tensor-network model improves over the tuned ESN on almost all tasks for short-term prediction (sMAPE and valid prediction time) and on most tasks for the long-term climate metric.
- Training with the tensor-network model is at least an order of magnitude faster: the full TN optimization runs in about an hour on two threads, while the ESN grid search takes over 60 hours.
- Hyperparameter optimization reduces to two integers, $M$ and $D$, eliminating the reservoir-size, spectral-radius, input-strength, leak-rate, and regularization knobs of the ESN.
- Because echo state networks can be approximated by truncated Volterra series, the paper argues any ESN performance should be matchable by a TN model with much simpler tuning.
Reading between the lines
- The appendix's numerical symmetry loss suggests a cheap, testable fix: re-symmetrize the fitted kernel after the pseudoinverse and check whether accuracy on rank-deficient chaotic tasks improves; the paper does not do this.
- The same MPO construction should carry over to spatiotemporal systems if the SVD step is swapped for a randomized or low-rank SVD, an extension the paper lists as future work rather than a result.
- If the benchmark claims hold, tensor-network Volterra models become a credible default baseline for low-dimensional chaos forecasting, with the additional benefit that the learned MPO cores are inspectable Volterra kernels rather than an opaque reservoir.
- A direct comparison with sparse system identification and polynomial kernel methods on the same 70 systems would reveal whether the advantage comes from the Volterra representation itself or from the MPO compression; the paper only lists this as future work.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a tensor-network approach, specifically a matrix-product-operator (MPO) representation of a truncated Volterra series, for predicting chaotic time series. The model is benchmarked against a leaky echo state network (ESN) on 70 low-dimensional chaotic systems from the dysts database. The authors report that the tensor-network model achieves better performance on most tasks according to short-term metrics (sMAPE and valid prediction time) and a climate metric (Wasserstein distance between spectra), while requiring at least one order of magnitude less training time. The model depends on only two hyperparameters (maximum delay M and maximum monomial degree D). An appendix establishes theoretically that the minimal-norm least-squares solution is symmetric regardless of the rank of the design matrix, but also presents numerical examples showing that correlated inputs can break this symmetry in practice.
Significance. If the central claim is robust, the paper offers a practical, low-hyperparameter alternative to echo state networks for low-dimensional chaotic forecasting, with a clear speed advantage. The availability of source code and the use of a standardized benchmark (dysts) are strengths. However, the numerical symmetry issue documented in the appendix directly affects the model actually evaluated, and the ESN baseline is optimized under a restricted search budget; both issues must be resolved before the accuracy claim can be taken at face value.
major comments (2)
- [Appendix, Table 1 and closing paragraph] The appendix shows that for correlated inputs, the SVD-based pseudoinverse breaks the symmetry of the Volterra kernel, with the symmetry metric S (Eq. 31) reaching 0.13 even when rank(U)=R (Table 1, N=100 row). The manuscript then states that 'some of the chaotic tasks fulfill rank(U) < min(N, R)' and that larger rank discrepancies correlate with higher S, but it never reports S for the 70 benchmark tasks or demonstrates that the symmetry loss does not affect the prediction metrics. Since the MPO construction in 'MPOs enforcing symmetry' is justified by the exact symmetry of H, the model actually evaluated on those tasks may not be the symmetric Volterra kernel described in the theory. The central empirical claim could therefore be sensitive to SVD truncation thresholds rather than to the tensor-network structure. Please quantify S or an equivalent symmetry-loss measure for all 70 tasks, and either show that the reported results are insensitive to the truncation threshold or provide a mitigation (e.g., projecting H onto the symmetric subspace).
- [Validation section] The ESN baseline is optimized with a fixed reservoir size Nr=500 and a coarse grid over rho, g, epsilon, and lambda, using a single validation trajectory. The paper acknowledges the limitation and argues that a more exhaustive search would not change the conclusion, but no supporting evidence is provided. Since the abstract claims the TN model is 'better' than a 'conventional' ESN, the restricted ESN tuning makes the accuracy comparison potentially unfair. The training-time advantage (at least one order of magnitude) is independent of this issue, but the accuracy claim needs either a more extensive ESN hyperparameter search, a sensitivity analysis showing robustness to ESN tuning, or a rephrasing that restricts the claim to the explored ESN configuration.
minor comments (4)
- [Discussion] The introduction motivates the TN model as a remedy for NGRC's exponential parameter growth, but the benchmark only compares with ESN; a comparison with NGRC or a standard truncated Volterra model would clarify the practical benefit and is currently missing.
- [Validation section] The sentence 'we expect that since ESN models can be approximated by truncated Volterra series, we could always find a TN model that performs as well as the ESN' is presented as a justification, but it is only an expectation. Consider deriving it or softening the claim to avoid giving the impression of an established result.
- [Appendix, Eq. (31)] The claimed correlation between the rank discrepancy and the symmetry metric S is not quantified; reporting a correlation coefficient or a scatter plot would strengthen the appendix's argument.
- [Acknowledgements] There is a typo in the acknowledgements: 'manuscrip' should be 'manuscript'.
Circularity Check
No material circularity: the TN model is trained on a dedicated set and tested on held-out trajectories, the symmetry guarantee is an external mathematical result (Batselier et al.) rather than a self-citation, and the Appendix's symmetry-loss caveat is a numerical limitation, not a step that reduces the prediction to its inputs.
full rationale
The derivation chain is: define the Volterra feature matrix U (Eq. 13), solve the least-squares problem H = U^+Y (Eq. 18), and implement H via MPOs from refs. 34/35. The claim that columns of H are symmetric follows from the row-space containment R(U) subset of S_D (Appendix Eqs. 28-29) and was proved in ref. 33 under a persistently-exciting assumption; the paper extends it in the Appendix without rank assumptions. This is independent linear algebra, not a definition of the target. The numerical results compare TN against ESN on held-out test trajectories using sMAPE, VPT, and Wasserstein metrics, so the 'predictions' are not fitted to the test data. The only self-citation is Orus's own tensor-network review (ref. 39), which is background and not load-bearing. The Appendix's Table 1 shows that for correlated inputs the numerical pseudoinverse can break symmetry (S up to 0.13), and the paper admits some chaotic tasks have rank(U) < min(N,R); this is an honest numerical limitation affecting the exact equivalence between the implemented model and the symmetric Volterra kernel, but it does not make any prediction equivalent to its inputs by construction. The statement that ESNs can be approximated by truncated Volterra series is an expectation motivating the comparison, not an input to the TN training. Hence score 0.
Assumptions & free parameters
free parameters (3)
- Maximum delay M =
grid {1,2,3,4}, per-task selection not reported
- Maximum monomial degree D =
grid {2,3,4}, per-task selection not reported
- ESN hyperparameters (rho, g, epsilon, lambda, Nr=500) =
grid ranges: rho in [0.8,1.2], g in [0.1,1], epsilon in [0.1,1], lambda in [1e-13,1e-1]; per-task optima not reported
assumptions (5)
- standard math Truncated Volterra series have universal approximation properties for fading-memory systems (Boyd and Chua; Grigoryeva and Ortega).
- standard math The MPO construction of reference 35 exactly represents the Khatri-Rao product matrix U without truncation.
- standard math The minimal-norm least-squares solution produces symmetric Volterra kernels even for rank-deficient U.
- domain assumption ESN models can be approximated by truncated Volterra series, so a TN model can always be found that performs as well as the ESN.
- domain assumption Numerical SVD truncation preserves the symmetry of the Volterra kernel for chaotic correlated inputs.
Cite this review
Pith. "Pith review of A tensor network approach for chaotic time series prediction." pith.science (2026). https://pith.science/paper/R4HPI6B4
@misc{pith2026250517740,
author = {Pith},
title = {Pith review of: A tensor network approach for chaotic time series prediction},
year = {2026},
howpublished = {\url{https://pith.science/paper/R4HPI6B4}},
note = {Machine review of arXiv:2505.17740}
}
read the original abstract
Making accurate predictions of chaotic time series is a complex challenge. Reservoir computing, a neuromorphic-inspired approach, has emerged as a powerful tool for this task. It exploits the memory and nonlinearity of dynamical systems without requiring extensive parameter tuning. However, selecting and optimizing reservoir architectures remains an open problem. Next-generation reservoir computing simplifies this problem by employing nonlinear vector autoregression based on truncated Volterra series, thereby reducing hyperparameter complexity. Nevertheless, the latter suffers from exponential parameter growth in terms of the maximum monomial degree. Tensor networks offer a promising solution to this issue by decomposing multidimensional arrays into low-dimensional structures, thus mitigating the curse of dimensionality. This paper explores the application of a previously proposed tensor network model for predicting chaotic time series, demonstrating its advantages in terms of accuracy and computational efficiency compared to conventional echo state networks. Using a state-of-the-art tensor network approach enables us to bridge the gap between the tensor network and reservoir computing communities, fostering advances in both fields.
Figures
Reference graph
Works this paper leans on
-
[1]
Slingo, J. & Palmer, T. Uncertainty in weather and climate prediction. Philos. Transactions Royal Soc. A: Math. Phys. Eng. Sci. 369, 4751–4767 (2011)
work page 2011
-
[2]
May, R. & McLean, A. R. Theoretical ecology: principles and applications (Oxford University Press, 2007)
work page 2007
-
[3]
Argyris, A. et al. Chaos-based communications at high bit rates using commercial fibre-optic links. Nature 438, 343–346 (2005)
work page 2005
-
[4]
Reservoir kernels and Volterra series
Gonon, L., Grigoryeva, L. & Ortega, J.-P. Reservoir kernels and volterra series. arXiv preprint arXiv:2212.14641 (2022)
work page Pith review arXiv 2022
-
[5]
Grigoryeva, L., Ting, H. L. J. & Ortega, J.-P. Infinite-dimensional next-generation reservoir computing. arXiv preprint arXiv:2412.09800 (2024)
work page Pith review arXiv 2024
-
[6]
Vlachas, P. R., Byeon, W., Wan, Z. Y ., Sapsis, T. P. & Koumoutsakos, P. Data-driven forecasting of high-dimensional chaotic systems with long short-term memory networks. Proc. Royal Soc. A: Math. Phys. Eng. Sci. 474, 20170844 (2018)
work page 2018
-
[7]
Model scale versus domain knowledge in statistical forecasting of chaotic systems
Gilpin, W. Model scale versus domain knowledge in statistical forecasting of chaotic systems. Phys. Rev. Res. 5, 043252 (2023)
work page 2023
- [8]
Show all 63 references
-
[9]
& Gauthier, D
Griffith, A., Pomerance, A. & Gauthier, D. J. Forecasting chaotic systems with very low connectivity reservoir computers. Chaos: An Interdiscip. J. Nonlinear Sci. 29 (2019)
2019
-
[10]
Vlachas, P.-R. et al. Backpropagation algorithms and reservoir computing in recurrent neural networks for the forecasting of complex spatiotemporal dynamics. Neural Networks 126, 191–217 (2020)
2020
-
[11]
& Ortega, J.-P
Grigoryeva, L., Hart, A. & Ortega, J.-P. Chaos on compact manifolds: Differentiable synchronizations beyond the takens theorem. Phys. Rev. E 103, 062204 (2021)
2021
-
[12]
& Ortega, J.-P
Grigoryeva, L., Hart, A. & Ortega, J.-P. Learning strange attractors with reservoir systems. Nonlinearity 36, 4674 (2023)
2023
-
[13]
& Ortega, J.-P
Grigoryeva, L. & Ortega, J.-P. Universal discrete-time reservoir computers with stochastic inputs and linear readouts using non-homogeneous state-affine systems. J. Mach. Learn. Res. 19, 1–40 (2018)
2018
-
[14]
& Ortega, J.-P
Grigoryeva, L. & Ortega, J.-P. Echo state networks are universal. Neural Networks 108, 495–508 (2018)
2018
-
[15]
& Ortega, J.-P
Gonon, L. & Ortega, J.-P. Reservoir computing universality with stochastic inputs. IEEE transactions on neural networks learning systems 31, 100–112 (2019)
2019
-
[16]
& Ortega, J.-P
Gonon, L. & Ortega, J.-P. Fading memory echo state networks are universal. Neural Networks 138, 10–13 (2021)
2021
-
[17]
Tutorial on training recurrent neural networks, covering BPPT, RTRL, EKF and the echo state network approach, vol
Jaeger, H. Tutorial on training recurrent neural networks, covering BPPT, RTRL, EKF and the echo state network approach, vol. 5 (Citeseer, 2002)
2002
-
[18]
& Jaeger, H
Lukoˇseviˇcius, M. & Jaeger, H. Reservoir computing approaches to recurrent neural network training. Comput. science review 3, 127–149 (2009)
2009
-
[19]
Sun, C. et al. A systematic review of echo state networks from design to application. IEEE Transactions on Artif. Intell. 5, 23–37 (2022)
2022
-
[20]
A., Penny, S
Platt, J. A., Penny, S. G., Smith, T. A., Chen, T.-C. & Abarbanel, H. D. A systematic exploration of reservoir computing for forecasting complex spatiotemporal dynamics. Neural Networks 153, 530–552 (2022)
2022
-
[21]
& Ortega, J.-P
Gonon, L., Grigoryeva, L. & Ortega, J.-P. Approximation bounds for random neural networks and reservoir systems. The Annals Appl. Probab. 33, 28–69 (2023)
2023
-
[22]
Tanaka, G. et al. Recent advances in physical reservoir computing: A review. Neural Networks 115, 100–123 (2019)
2019
-
[23]
Physical reservoir computing—an introductory perspective
Nakajima, K. Physical reservoir computing—an introductory perspective. Jpn. J. Appl. Phys. 59, 060501 (2020)
2020
-
[24]
& Fischer, I
Nakajima, K. & Fischer, I. Reservoir computing (Springer, 2021)
2021
-
[25]
Yan, M. et al. Emerging opportunities and challenges for the future of reservoir computing. Nat. Commun. 15, 2056 (2024)
2024
-
[26]
J., Bollt, E., Griffith, A
Gauthier, D. J., Bollt, E., Griffith, A. & Barbosa, W. A. Next generation reservoir computing. Nat. communications 12, 1–8 (2021)
2021
-
[27]
Barbosa, W. A. & Gauthier, D. J. Learning spatiotemporal chaos using next-generation reservoir computing. Chaos: An Interdiscip. J. Nonlinear Sci. 32 (2022). 10/12
2022
-
[28]
J., Fischer, I
Gauthier, D. J., Fischer, I. & R¨ohm, A. Learning unseen coexisting attractors. Chaos: An Interdiscip. J. Nonlinear Sci. 32 (2022)
2022
-
[29]
M., Barbosa, W
Kent, R. M., Barbosa, W. A. & Gauthier, D. J. Controlling chaotic maps using next-generation reservoir computing.Chaos: An Interdiscip. J. Nonlinear Sci. 34 (2024)
2024
-
[30]
M., Barbosa, W
Kent, R. M., Barbosa, W. A. & Gauthier, D. J. Controlling chaos using edge computing hardware. Nat. Commun. 15, 3886 (2024)
2024
-
[31]
& Chua, L
Boyd, S. & Chua, L. Fading memory and the problem of approximating nonlinear operators with volterra series. IEEE Transactions on circuits systems 32, 1150–1161 (1985)
1985
-
[32]
& Ortega, J.-P
Grigoryeva, L. & Ortega, J.-P. Differentiable reservoir computing. J. Mach. Learn. Res. 20, 1–62 (2019)
2019
-
[33]
& Wong, N
Batselier, K., Chen, Z. & Wong, N. Tensor network alternating linear scheme for mimo volterra system identification. Automatica 84, 26–35 (2017)
2017
-
[34]
Enforcing symmetry in tensor network mimo volterra identification
Batselier, K. Enforcing symmetry in tensor network mimo volterra identification. IF AC-PapersOnLine54, 469–474 (2021)
2021
-
[35]
A khatri-rao product tensor network for efficient symmetric mimo volterra identification
Batselier, K. A khatri-rao product tensor network for efficient symmetric mimo volterra identification. IF AC-PapersOnLine 56, 7282–7287 (2023)
2023
-
[36]
& Favier, G
Khouaja, A. & Favier, G. Identification of parafac-volterra cubic models using an alternating recursive least squares algorithm. In 2004 12th European Signal Processing Conference, 1903–1906 (IEEE, 2004)
2004
-
[37]
& Bouilloc, T
Favier, G. & Bouilloc, T. Parametric complexity reduction of volterra models using tensor decompositions. In 2009 17th European Signal Processing Conference, 2288–2292 (IEEE, 2009)
2009
-
[38]
& Kibangou, A
Favier, G. & Kibangou, A. Tensor-based approaches for nonlinear and multilinear systems modeling and identification. Algorithms 16, 443 (2023)
2023
-
[39]
Tensor networks for complex quantum systems
Or´us, R. Tensor networks for complex quantum systems. Nat. Rev. Phys. 1, 538–550 (2019)
2019
-
[40]
Ba˜nuls, M. C. Tensor network algorithms: A route map. Annu. Rev. Condens. Matter Phys. 14, 173–191 (2023)
2023
-
[41]
& Schwab, D
Stoudenmire, E. & Schwab, D. J. Supervised learning with tensor networks. Adv. neural information processing systems 29 (2016)
2016
-
[42]
& Zhang, P
Cheng, S., Wang, L., Xiang, T. & Zhang, P. Tree tensor networks for generative modeling.Phys. Rev. B 99, 155131 (2019)
2019
-
[43]
Tomut, A. et al. Compactifai: extreme compression of large language models using quantum-inspired tensor networks. arXiv preprint arXiv:2401.14109 (2024)
2024 arXiv
-
[44]
Low-rank tensor decompositions for nonlinear system identification: A tutorial with examples
Batselier, K. Low-rank tensor decompositions for nonlinear system identification: A tutorial with examples. IEEE Control. Syst. Mag. 42, 54–74 (2022)
2022
-
[45]
Chaos as an interpretable benchmark for forecasting and data-driven modelling.arXiv preprint arXiv:2110.05266 (2021)
Gilpin, W. Chaos as an interpretable benchmark for forecasting and data-driven modelling.arXiv preprint arXiv:2110.05266 (2021)
2021 arXiv
-
[46]
Horn, R. A. & Johnson, C. R. Matrix analysis (Cambridge university press, 2012)
2012
-
[47]
A singularly valuable decomposition: the svd of a matrix
Kalman, D. A singularly valuable decomposition: the svd of a matrix. The college mathematics journal 27, 2–23 (1996)
1996
-
[48]
Oseledets, I. V . & Tyrtyshnikov, E. E. Breaking the curse of dimensionality, or how to use svd in many dimensions.SIAM J. on Sci. Comput. 31, 3744–3759 (2009)
2009
-
[49]
Kolda, T. G. & Bader, B. W. Tensor decompositions and applications. SIAM review 51, 455–500 (2009)
2009
-
[50]
Oseledets, I. V . Tensor-train decomposition.SIAM J. on Sci. Comput. 33, 2295–2317 (2011)
2011
-
[51]
Verstraete, F., Garcia-Ripoll, J. J. & Cirac, J. I. Matrix product density operators: Simulation of finite-temperature and dissipative systems. Phys. review letters 93, 207204 (2004)
2004
-
[52]
& Vidal, G
Zwolak, M. & Vidal, G. Mixed-state dynamics in one-dimensional quantum lattice systems: A time-dependent superopera- tor renormalization algorithm. Phys. review letters 93, 207205 (2004)
2004
-
[53]
Oseledets, I. V . Approximation of 2ˆd\times2ˆd matrices using tensor decomposition. SIAM J. on Matrix Analysis Appl. 31, 2130–2145 (2010)
2010
-
[54]
& Wong, N
Batselier, K., Ko, C.-Y . & Wong, N. Tensor network subspace identification of polynomial state space models.Automatica 95, 187–196 (2018)
2018
-
[55]
Favier, G., Kibangou, A. Y . & Bouilloc, T. Nonlinear system modeling and identification using volterra-parafac models. Int. J. Adapt. Control. Signal Process. 26, 30–53 (2012). 11/12
2012
-
[56]
& Favier, G
Bouilloc, T. & Favier, G. Nonlinear channel modeling and identification using baseband volterra–parafac models. Signal Process. 92, 1492–1498 (2012)
2012
-
[57]
& Batselier, K
Memmel, E., Menzen, C. & Batselier, K. Bayesian framework for a mimo volterra tensor network. IF AC-PapersOnLine 56, 7294–7299 (2023)
2023
-
[58]
echo state
Jaeger, H. The “echo state” approach to analysing and training recurrent neural networks-with an erratum note. Bonn, Ger. Ger. Natl. Res. Cent. for Inf. Technol. GMD Tech. Rep. 148, 13 (2001)
2001
-
[59]
A., Zhang, L., Nicolaou, Z
Kaptanoglu, A. A., Zhang, L., Nicolaou, Z. G., Fasel, U. & Brunton, S. L. Benchmarking sparse system identification with low-dimensional chaos. Nonlinear Dyn. 111, 13143–13164 (2023)
2023
-
[60]
& Chitre, M
Bagge Carlson, F. & Chitre, M. New metrics between rational spectra and their connection to optimal transport. arXiv e-prints arXiv–2004 (2020)
2020
-
[61]
Wikner, A. et al. Stabilizing machine learning prediction of dynamics: Novel noise-inspired regularization tested with reservoir computing. Neural Networks 170, 94–110 (2024)
2024
-
[62]
& Wong, N
Batselier, K., Yu, W., Daniel, L. & Wong, N. Computing low-rank approximations of large-scale matrices with the tensor network randomized svd. SIAM J. on Matrix Analysis Appl. 39, 1221–1244 (2018)
2018
-
[63]
L., Proctor, J
Brunton, S. L., Proctor, J. L. & Kutz, J. N. Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proc. national academy sciences 113, 3932–3937 (2016). Acknowledgements We thank Hannah Lim Jing Ting, Miguel C. Soriano, and Juan-Pa...
2016
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.