Recognition: unknown
Kolmogorov-Sinai entropies identify optimal observables for prediction and dynamics reconstruction in chaotic systems
Pith reviewed 2026-05-07 09:51 UTC · model grok-4.3
The pith
Kolmogorov-Sinai entropy of an observable predicts its reconstruction error for the underlying chaotic dynamics.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Under modest technical conditions the Kolmogorov-Sinai entropy of an observable bounds its reconstruction error of the underlying dynamics in chaotic, ergodic systems. The Oseledets Multiplicative Ergodic Theorem establishes an invariant filtration on the tangent bundles of delay-embedded manifolds that is diffeomorphically related across admissible observables, with Lyapunov exponents governing the forward propagation of perturbations. Reconstruction error is thereby bounded by a quantity monotonically related to the sum of positive Lyapunov exponents, which the Ruelle inequality shows is itself bounded by the Kolmogorov-Sinai entropy.
What carries the argument
Kolmogorov-Sinai entropy of an observable, which upper-bounds the sum of positive Lyapunov exponents and thereby sets the rate at which reconstruction errors grow in delay-embedded manifolds.
If this is right
- Observables with lower Kolmogorov-Sinai entropy produce lower reconstruction RMSE when their delay coordinates are used to recover the attractor.
- The entropy value supplies an a priori ranking that can be computed directly from data before any model is trained.
- The same ranking remains valid under added measurement noise, as shown by sharpened correlations on the tetracosane trajectory.
- Any data-driven modeling pipeline for chaotic flows can insert this entropy calculation as a data-selection filter.
Where Pith is reading between the lines
- The same entropy criterion could be used to choose which variables to retain in high-dimensional simulations such as turbulence or climate models.
- Minimal-entropy observables might also minimize long-term prediction error, not only one-step reconstruction error.
- The result suggests testing whether the ordering persists when the embedding dimension is varied or when the underlying flow is only weakly chaotic.
Load-bearing premise
The system must be ergodic and the Oseledets theorem must apply to the tangent bundles of delay-embedded manifolds formed from different observables so that an invariant filtration exists across them.
What would settle it
An observable whose Kolmogorov-Sinai entropy is high yet whose delay-coordinate reconstruction RMSE is low (or vice versa) on a known chaotic attractor would falsify the claimed monotonic relationship.
Figures
read the original abstract
Choosing the optimal observable to model dynamical systems for which we do not know the driving equations is nearly always an ad hoc art. Takens' Delay Embedding Theorem guarantees a diffeomorphism between delay-coordinate vectors built from generic scalar observables and the underlying invariant attractor, but is agnostic to optimal observable choice, and formal bounds on reconstruction quality across observables are not known. Here we prove that, under modest technical conditions, the Kolmogorov-Sinai entropy of an observable predicts its reconstruction error of the underlying dynamics in chaotic, ergodic systems. Using the Oseledets Multiplicative Ergodic Theorem, we show that the tangent bundles of reconstructed manifolds admit an invariant Oseledets filtration diffeomorphically related across admissible observables, with Lyapunov exponents controlling the propagation of perturbations. We bound reconstruction error by a quantity monotonically related to the sum of positive Lyapunov exponents and, by the Ruelle inequality, the Kolmogorov-Sinai entropy. We validate this empirically on the Lorenz-63 attractor, the Hastings-Powell food chain, and a tetracosane molecular-dynamics trajectory, recovering Spearman rank correlations between $h^{KS,UB}$ and reconstruction RMSE up to $\rho=+0.89$ ($p=5.5\times 10^{-8}$) for the realistic tetracosane case, sharpening to $\rho=+0.97$ under added measurement noise. This provides a rigorous foundation for observable selection in chaotic systems, applicable as an a priori data-selection criterion for any data-driven modeling pipeline.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper claims to prove that, under modest technical conditions, the Kolmogorov-Sinai entropy of a scalar observable predicts its reconstruction error when delay-embedding the underlying dynamics in chaotic, ergodic systems. It invokes the Oseledets Multiplicative Ergodic Theorem to obtain an invariant filtration on the tangent bundles of reconstructed manifolds that is diffeomorphically related across admissible observables; Lyapunov exponents then control perturbation growth, and reconstruction error is bounded by a quantity monotonically related to the sum of positive exponents (hence, via Ruelle's inequality, to the observable's KS entropy h^{KS,UB}). Empirical support consists of Spearman rank correlations between h^{KS,UB} and reconstruction RMSE reaching +0.89 (p=5.5e-8) on a tetracosane molecular-dynamics trajectory and +0.97 under added noise, with similar results on Lorenz-63 and Hastings-Powell.
Significance. If the central derivation holds, the result supplies a principled, a priori entropy-based criterion for choosing observables in data-driven modeling of chaotic systems, replacing ad hoc selection. The combination of standard ergodic-theory tools (Oseledets, Ruelle) with concrete, reproducible correlations on both low-dimensional maps and a realistic high-dimensional trajectory constitutes a concrete strength; the noise-robust sharpening of the correlation is particularly noteworthy.
major comments (2)
- [Abstract / central derivation] Abstract and proof outline: the claimed monotonic dependence of the reconstruction-error bound on the observable-specific h^{KS,UB} appears inconsistent with the invariance of the Lyapunov spectrum. Takens' theorem guarantees a diffeomorphism for generic observables, so the Oseledets filtration and the sum of positive Lyapunov exponents are identical across such observables; any bound derived solely from that sum must therefore be independent of the particular observable. The paper must clarify how h^{KS,UB} is defined (system entropy versus observable-induced entropy rate) and why the error bound nevertheless varies with the observable.
- [Technical conditions / Oseledets application] § on technical conditions (Oseledets applicability): the modest conditions required for the invariant filtration to exist on delay-embedded tangent bundles are stated but not verified for the concrete systems. In particular, ergodicity of the tetracosane trajectory and the precise relation between the observable-dependent cocycle and the original flow's filtration need explicit checks; without them the error bound cannot be asserted to hold for the reported observables.
minor comments (2)
- [Notation] Notation: the symbol h^{KS,UB} is used throughout but its precise computational definition (partition, binning, or upper-bound estimator) is not restated in the main text; a short explicit formula or reference to the supplementary algorithm would aid reproducibility.
- [Validation] Empirical section: the number of distinct observables tested and the exact procedure for computing RMSE on the reconstructed attractors should be stated in a table or methods paragraph so that the reported Spearman values can be independently verified.
Simulated Author's Rebuttal
We thank the referee for the careful reading and constructive comments on our manuscript. The points raised help us strengthen the clarity of the central derivation and the technical foundations. We address each major comment below and will revise the manuscript accordingly.
read point-by-point responses
-
Referee: [Abstract / central derivation] Abstract and proof outline: the claimed monotonic dependence of the reconstruction-error bound on the observable-specific h^{KS,UB} appears inconsistent with the invariance of the Lyapunov spectrum. Takens' theorem guarantees a diffeomorphism for generic observables, so the Oseledets filtration and the sum of positive Lyapunov exponents are identical across such observables; any bound derived solely from that sum must therefore be independent of the particular observable. The paper must clarify how h^{KS,UB} is defined (system entropy versus observable-induced entropy rate) and why the error bound nevertheless varies with the observable.
Authors: We agree that the underlying Lyapunov spectrum of the dynamical system is invariant under the diffeomorphisms provided by Takens' theorem for generic observables, and that the Oseledets filtration is correspondingly preserved. However, h^{KS,UB} is defined as an upper bound on the entropy rate of the specific scalar observable (i.e., the entropy rate of the stationary process induced by iterating the observable along orbits), which is observable-dependent and generally strictly less than or equal to the system's Kolmogorov-Sinai entropy. The reconstruction-error bound is obtained by controlling the growth of perturbations in the delay-embedded tangent bundle via the observable's information production rate; this yields a monotonic relation to the observable-specific entropy rate rather than solely to the invariant sum of positive Lyapunov exponents. The Ruelle inequality is invoked only to relate the observable entropy rate to the (invariant) Lyapunov spectrum as an upper bound, but the error estimate itself varies with the observable because different observables generate different entropy rates. We will revise the abstract, the statement of the main theorem, and the proof outline to make this distinction explicit and to emphasize that the bound is not derived solely from the invariant sum. revision: yes
-
Referee: [Technical conditions / Oseledets application] § on technical conditions (Oseledets applicability): the modest conditions required for the invariant filtration to exist on delay-embedded tangent bundles are stated but not verified for the concrete systems. In particular, ergodicity of the tetracosane trajectory and the precise relation between the observable-dependent cocycle and the original flow's filtration need explicit checks; without them the error bound cannot be asserted to hold for the reported observables.
Authors: We concur that explicit verification of the technical hypotheses is necessary for the concrete examples. In the revised manuscript we will add a dedicated subsection that (i) confirms ergodicity of the tetracosane trajectory by verifying convergence of time averages to ensemble averages for several independent observables and by estimating the correlation dimension to ensure the sampled trajectory densely explores the attractor; (ii) for all three systems (Lorenz-63, Hastings-Powell, tetracosane) we will explicitly construct the observable-dependent cocycle on the delay-embedded tangent bundle and show it is cohomologous to the original tangent cocycle via the embedding map, thereby inheriting the same Oseledets filtration. These checks will be supported by numerical diagnostics and, where possible, by reference to known ergodic properties of the low-dimensional maps. revision: yes
Circularity Check
Derivation relies on external theorems with no reduction to self-inputs
full rationale
The central claim derives an observable-dependent reconstruction error bound from the Oseledets Multiplicative Ergodic Theorem applied to tangent bundles of delay-embedded manifolds, combined with Ruelle's inequality relating the sum of positive Lyapunov exponents to Kolmogorov-Sinai entropy. These are standard external results whose statements are independent of the paper's definitions, fitted quantities, or empirical data. No step equates a prediction to a fitted input by construction, invokes a self-citation as the sole justification for a uniqueness claim, or renames a known pattern as a new derivation. The reported Spearman correlations are presented as separate empirical validation and do not enter the proof.
Axiom & Free-Parameter Ledger
axioms (3)
- standard math Oseledets Multiplicative Ergodic Theorem applies to the tangent bundles of delay-reconstructed manifolds
- standard math Ruelle inequality upper-bounds Kolmogorov-Sinai entropy by the sum of positive Lyapunov exponents
- domain assumption The underlying dynamical system is chaotic and ergodic
Reference graph
Works this paper leans on
-
[1]
The trajectories of each O i(t)produce embeddings Mi ⊆R 2m+1 via Takens’ Theorem upon which the time-evolved trajectories on the embedded manifolds, Ti(t), exist
-
[2]
All of the requirements of Takens’ Theorem be fulfilled including sufficient sampling time such that allM i will be diffeomorphically related to each other and the in- trinsic attractorM
-
[3]
Each orbitT i(t)onM i be ergodic with respect to mea- sureµ i pertaining to the dynamics on each manifold
-
[4]
,K i with associ- ated lyapunov exponentsλ i q
LetK i be the number of distinct Lyapunov exponents of (Ti,µ i)with indices indexedq=1, . . . ,K i with associ- ated lyapunov exponentsλ i q
-
[5]
That is to say each Oseledets subspace gets the same order of noise
The cocycle on eachM i admits an Oseledets split- tingR 2m+1 = L q Ei q and noise is isotropic across Oseledets subspaces (expected for uncorrelated ther- mal/instrument noise). That is to say each Oseledets subspace gets the same order of noise. Projections ε (q) i ∈E i q ofε i then satisfyE∥ε (q) i ∥2 ≍σ/Kuniformly ini,qwhereσ=tr(Cov(Ω)), and∥ε i∥2 = ∑q...
-
[6]
Candidate observables yield componentwise-ordered Lyapunov spectra: ifh KS,UB i ≤h KS,UB k thenλ i q ≤λ k q for everyq∈ {1, . . . ,K}. hKS,UB i := ∑q:λ iq>0 λ i q is the Ruelle upper bound [41] on the effective Kolmogorov–Sinai entropy of(Ti,µ i). Letε i be the lowest possible mean root mean squared er- ror in reconstruction of all pointsr(t)∈Mfromr i(t)∈...
-
[7]
S. H. Strogatz,Nonlinear dynamics and chaos : with applica- tions to physics, biology, chemistry, and engineering, 2nd ed. (Westview Press, 2015)
2015
-
[8]
Barabási,Linked: The New Science of Networks(Perseus Publishing, 2002)
A.-L. Barabási,Linked: The New Science of Networks(Perseus Publishing, 2002)
2002
-
[9]
Mitchell,Complexity: A Guided Tour(Oxford University Press, 2009)
M. Mitchell,Complexity: A Guided Tour(Oxford University Press, 2009)
2009
-
[10]
Goodfellow, Y
I. Goodfellow, Y . Bengio, and A. Courville,Deep Learning (MIT Press, 2015)
2015
-
[11]
R. M. May, Nature261, 459 (1976)
1976
-
[12]
H. R. Varian, Journal of Economic Perspectives28, 3 (2014)
2014
-
[13]
Mezi ´c, Nonlinear Dynamics41, 309 (2005)
I. Mezi ´c, Nonlinear Dynamics41, 309 (2005)
2005
-
[14]
M. O. Williams, I. G. Kevrekidis, and C. W. Rowley, J. Nonlin- ear Sci.25, 1307 (2015)
2015
-
[15]
S. L. Brunton, J. L. Proctor, and J. N. Kutz, Proc. Natl. Acad. Sci. USA113, 3932 (2016)
2016
-
[16]
Letellier and L
C. Letellier and L. A. Aguirre, Chaos: An Interdisciplinary Journal of Nonlinear Science12, 549 (2002)
2002
-
[17]
Letellier, L
C. Letellier, L. Aguirre, and J. Maquet, Communications in Nonlinear Science and Numerical Simulation11, 555 (2006)
2006
-
[18]
L. A. Aguirre and C. Letellier, Journal of Physics A: Mathemat- ical and General38, 6311 (2005)
2005
-
[19]
L. A. Aguirre and C. Letellier, Mathematical Problems in Engi- neering2009(2009)
2009
-
[20]
Latora and M
V . Latora and M. Baranger, Phys. Rev. Lett.82, 520 (1999)
1999
-
[21]
Pang, Y .-h
H. Pang, Y .-h. Shin, D. Ihm, E. K. Lee, and O. Kum, Phys. Rev. E62, 6516 (2000)
2000
-
[22]
Dzugutov, E
M. Dzugutov, E. Aurell, and A. Vulpiani, Phys. Rev. Lett.81, 1762 (1998)
1998
-
[23]
A. S. de Wijn, Phys. Rev. E71, 046211 (2005)
2005
-
[24]
Walters,An Introduction to Ergodic Theory(Springer-Verlag, 1982)
P. Walters,An Introduction to Ergodic Theory(Springer-Verlag, 1982). 12
1982
-
[25]
I. P. Cornfeld and Y . G. Fomin, S. V .and Sinai,Ergodic Theory (Springer-Verlag, 1982)
1982
-
[26]
V . I. Arnold,Mathematical Methods of Classical Mechanics, 2nd ed. (Springer-Verlag, 1989)
1989
-
[27]
Noether, Nachrichten von der Gesellschaft der Wis- senschaften zu Göttingen, Mathematisch-Physikalische Klasse 1918 , 235 (1918)
E. Noether, Nachrichten von der Gesellschaft der Wis- senschaften zu Göttingen, Mathematisch-Physikalische Klasse 1918 , 235 (1918)
1918
-
[28]
D. J. Wales,Energy Landscapes: Applications to Clusters, Biomolecules and Glasses(Cambridge University Press, 2003)
2003
-
[29]
Young, Journal of Statistical Physics108, 733 (2002)
L.-S. Young, Journal of Statistical Physics108, 733 (2002)
2002
-
[30]
Ruelle, Publications Mathématiques de l’IHÉS50, 27 (1979)
D. Ruelle, Publications Mathématiques de l’IHÉS50, 27 (1979)
1979
-
[31]
Stark, Journal of Nonlinear Science9, 255 (1999)
J. Stark, Journal of Nonlinear Science9, 255 (1999)
1999
-
[32]
Stark, D
J. Stark, D. S. Broomhead, M. Davies, and J. Huke, Journal of Nonlinear Science13, 519 (2003)
2003
-
[33]
D. S. Broomhead and G. P. King, Physica D: Nonlinear Phe- nomena20, 217 (1986)
1986
-
[34]
J. P. Eckmann and D. Ruelle, Rev. Mod. Phys.57, 617 (1985)
1985
-
[35]
Takens, inDynamical Systems and Turbulence, Warwick 1980, Lecture Notes in Mathematics, V ol
F. Takens, inDynamical Systems and Turbulence, Warwick 1980, Lecture Notes in Mathematics, V ol. 898, edited by D. A. Rand and L.-S. Young (Springer, Berlin, Heidelberg, 1981) pp. 366–381
1980
-
[36]
Grebogi, E
C. Grebogi, E. Ott, and J. A. Yorke, Phys. Rev. Lett.48, 1507 (1982)
1982
-
[37]
Kantz and T
H. Kantz and T. Schreiber,Nonlinear Time Series Analysis (Cambridge University Press, 2004)
2004
-
[38]
Casdagli, S
M. Casdagli, S. Eubank, J. D. Farmer, and J. Gibson, Physica D: Nonlinear Phenomena51, 52 (1991)
1991
-
[39]
Schreiber and P
T. Schreiber and P. Grassberger, Phys. Lett. A160, 411 (1991)
1991
-
[40]
Sauer, J
T. Sauer, J. A. Yorke, and M. Casdagli, Journal of Statistical Physics65, 579 (1991)
1991
-
[41]
M. W. Hirsch, R. L. Devaney, and S. Smale,Differential Equa- tions, Dynamical Systems, and an Introduction to Chaos, 2nd ed. (Academic Press, 2003)
2003
-
[42]
G. D. Birkhoff, Proceedings of the National Academy of Sci- ences17, 656 (1931)
1931
-
[43]
V . I. Oseledets, Trans. Moscow Math. Soc.19, 197 (1968)
1968
-
[44]
Arnold,Random Dynamical Systems(Springer-Verlag, 1998)
L. Arnold,Random Dynamical Systems(Springer-Verlag, 1998)
1998
-
[45]
J. P. Eckmann, S. O. Kamphorst, D. Ruelle, and S. Ciliberto, Phys. Rev. A34, 4971 (1986)
1986
-
[46]
M. T. Rosenstein, J. J. Collins, and C. J. De Luca, Physica D: Nonlinear Phenomena65, 117 (1993)
1993
-
[47]
Ruelle, Boletim da Sociedade Brasileira de Matemática - Bulletin/Brazilian Mathematical Society9, 83 (1978)
D. Ruelle, Boletim da Sociedade Brasileira de Matemática - Bulletin/Brazilian Mathematical Society9, 83 (1978)
1978
-
[48]
Pesin, Russian Mathematical Surveys32, 55 (1977)
Y . Pesin, Russian Mathematical Surveys32, 55 (1977)
1977
-
[49]
Ledrappier and L.-S
F. Ledrappier and L.-S. Young, Ann. Math.122, 509 (1985)
1985
-
[50]
Frenkel and B
D. Frenkel and B. Smit,Understanding Molecular Simula- tion: From algorithms to applications, V ol. 1 (Academic press, 2002)
2002
-
[51]
R. Roy, S. Hohng, and T. Ha, Nature Methods5, 507 (2008)
2008
-
[52]
Rudin,Principles of Mathematical Analysis, 3rd ed
W. Rudin,Principles of Mathematical Analysis, 3rd ed. (McGraw-Hill, New York, 1976)
1976
-
[53]
Spivak,Calculus on Manifolds: A Modern Approach to Classical Theorems of Advanced Calculus(W
M. Spivak,Calculus on Manifolds: A Modern Approach to Classical Theorems of Advanced Calculus(W. A. Benjamin, New York, 1965)
1965
-
[54]
Schölzel, Nonlinear measures for dynamical systems (2020)
C. Schölzel, Nonlinear measures for dynamical systems (2020)
2020
-
[55]
Topel,Reconstruction of dynamics from time series observa- tions: structural, ergodic, and information-theoretic perspec- tives, Ph.D
M. Topel,Reconstruction of dynamics from time series observa- tions: structural, ergodic, and information-theoretic perspec- tives, Ph.D. thesis, The University of Chicago (2025), knowl- edge@UChicago, record/14936
2025
-
[56]
Hastings and T
A. Hastings and T. Powell, Ecology72, 896 (1991)
1991
-
[57]
A. L. Ferguson, A. Z. Panagiotopoulos, P. G. Debenedetti, and I. G. Kevrekidis, Proceedings of the National Academy of Sci- ences of the United States of America107, 13597 (2010)
2010
-
[58]
Hairer, S
E. Hairer, S. P. Nørsett, and G. Wanner,Solving Ordinary Dif- ferential Equations I: Nonstiff Problems, 2nd ed., Springer Se- ries in Computational Mathematics, V ol. 8 (Springer-Verlag, Berlin, 1993)
1993
-
[59]
L. Cao, A. Mees, and K. Judd, Physica D: Nonlinear Phenom- ena121, 75 (1998)
1998
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.