REVIEW 4 major objections 5 minor 23 references
Revisiting PCA for time series reduction in temporal dimension
T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read PCA on sliding time windows preserves accuracy while cutting training cost by up to 40%.
desk verdict Useful empirical recipe for temporal PCA compression, but the 'no accuracy loss' claim is contradicted by the paper's own tables and the theory section has a clear error. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The mechanism is standard PCA applied to the window dimension of a dataset. The training windows are mean-centered, the temporal covariance matrix $C = \frac{1}{n-1} D_{\mathrm{centered}}^T D_{\mathrm{centered}}$ is computed, and each window is projected onto the top $k$ eigenvectors: $D_{\mathrm{pca}} = D_{\mathrm{centered}} V_k$. The covariance and eigenvectors are estimated once on the training set and applied to validation and test windows without re-estimation. The paper's justification is that windowing preserves enough temporal structure, the projection discards low-variance noise while retaining statistical characteristics like means, peaks, and higher-order moments, and the specific trend or periodicity of a series is not what models need, since coherent patterns are enough. For patch-based models, the same projection is applied to each patch separately, such as length-16 patches reduced to 2 components, before concatenation.
What would settle it
A replication on a classification dataset whose class labels are carried by low-amplitude, rare, or transient features, so that the top variance directions are class-blind, would falsify the no-accuracy-sacrifice claim if PCA preprocessing produces a large, reproducible accuracy drop; the paper's own results on SelfRegulationSCP1 with TimesNet (0.918 to 0.686) and UWaveGestureLibrary with Linear (0.575 to 0.409) are already candidate instances of this failure.
Extended reading notes
Core claim
The paper's central claim is that PCA preprocessing of sliding series windows is a valid and efficient form of temporal dimensionality reduction for deep time series models. The authors' argument is that windowing preserves partial temporal structure, so the PCA projection does not destroy the dependencies the model needs; instead it acts as a denoiser, keeping the top variance directions and discarding low-variance components as noise. On the paper's evidence, the reduced representation keeps the mean, peak, and higher-order moment information that models rely on, so accuracy is preserved while the input is shortened. The headline numbers are up to 40% faster Informer training and inference, a 30% GPU memory reduction for TimesNet, and comparable or better accuracy in roughly half of the classification, forecasting, and regression comparisons.
Load-bearing premise
The task-relevant signal in every window lies in the top-k variance directions of the temporal covariance estimated once on the training set, and that covariance stays the same between training and test windows.
Editorial extensions
If this is right
- Any fixed-window TSA model can take the PCA-projected series as a drop-in input, shortening 336-step windows to 48 steps, with comparable average accuracy and 10-40% faster training and inference.
- Because PCA adds no trainable parameters and is fit once on the training set, its efficiency gains do not come at the cost of model complexity, unlike adding a linear or 1D-CNN reduction layer.
- For patch-based models such as PatchTST, per-patch PCA preserves much of the forecasting performance only when instance normalization is removed, so the interaction between normalization and PCA preprocessing must be handled explicitly.
- Compared with shortening the input, downsampling, FFT, and DWT reductions, PCA is the only one of these methods that keeps forecasting accuracy near the unreduced baseline on ETT datasets.
- RNN models benefit most on a relative basis, with training time falling to about one-fourth and inference time to about one-third of the unreduced times.
Reading between the lines
- Inference: The accuracy drops the paper itself reports on SelfRegulationSCP1 and UWaveGestureLibrary suggest the method will fail when discriminative information lives in small-variance directions; a class-conditional or supervised PCA variant would be a natural extension.
- Inference: Because PCA is fit on training windows, it implicitly assumes the temporal covariance is stationary; on drifting or non-stationary streams an adaptive basis would need to be re-estimated periodically.
- Inference: The speedups are partly a model-length effect, so the same preprocessing should transfer to other architectures whose cost scales with input length, including modern linear and state-space sequence models.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes using Principal Component Analysis (PCA) on sliding windows of time series to reduce the temporal dimension before feeding inputs into deep learning models for time series classification, forecasting, and extrinsic regression. The authors argue that, contrary to the common belief that PCA disrupts temporal dependencies, applying PCA inside windows preserves statistical information and improves computational efficiency without sacrificing accuracy. They report experiments on 13 datasets with four model families (linear, Transformer, CNN, RNN) and compare PCA against shortening, downsampling, FFT, DWT, and learned reduction layers, as well as reporting training time and GPU memory savings.
Significance. If the central claim were fully supported, the paper would offer a simple, model-agnostic preprocessing step that reduces temporal dimensionality and computational cost while preserving accuracy, which would be of practical value to the time series community. The paper has strengths: it covers three tasks, multiple model families, and includes explicit efficiency measurements (Table 16, Figures 4–5) that show meaningful speedups and memory reductions. However, the accuracy claim as stated in the abstract ('without sacrificing model accuracy') is contradicted by the paper's own tables, with several large accuracy drops after PCA. The efficiency results remain credible and useful, so a revised version that honestly qualifies the accuracy claim and provides guidance on when PCA is safe would be a meaningful contribution.
major comments (4)
- [Abstract; §4.1, Table 2; §4.2, Table 3] The central claim that PCA improves efficiency 'without sacrificing model accuracy' is contradicted by the paper's own results. In Table 2, TimesNet on SelfRegulationSCP1 drops from 0.918 to 0.686 and Linear on UWaveGestureLibrary drops from 0.575 to 0.409 after PCA. In Table 3, FEDformer is worse with PCA in 21 of 32 metric cells. These are not small fluctuations; they show that class-discriminative and forecast-relevant information can reside in the discarded low-variance subspace. The abstract and the text in §4.1 ('PCA shows better performance in 10 metrics' and 'without performance loss') need to be qualified, for example by restricting the claim to datasets where a validation-based check confirms that the discarded subspace is not informative, or by reporting the proportion of settings where accuracy is maintained rather than an unconditional statement.
- [§3.2] The theoretical justification contains an incorrect claim: 'PCA preserves higher-order moments, including skewness and kurtosis, because its linear transformation ensures that these higher-order statistical characteristics remain intact.' A linear projection onto the top-k principal components is a rank-k map, and the skewness, kurtosis, and even the variance of the projected data are not invariant under this truncation. The same issue affects the claim about preserving 'relative mean/sum values,' since the projection is applied to mean-centered data. The denoising argument (treating low-variance components as noise) is an assumption, not a consequence of PCA, and the paper's own counterexamples in Table 2 show that this assumption fails on several datasets.
- [§4, all result tables] The experimental comparisons report a single run per configuration with no standard deviations, confidence intervals, or significance tests. Given that the 'maintains accuracy' claim rests on small differences (e.g., 'largely unchanged' for Linear and TimesNet in Table 3), the absence of variance information makes it impossible to distinguish genuine equivalence from run-to-run noise. The authors should report results from multiple seeds with error bars, or use a paired statistical test such as the Wilcoxon signed-rank test across the datasets, before claiming that accuracy is preserved.
- [Appendix B.4, Table 12] The comparison for PatchTST is not a like-for-like test of PCA. To apply PCA, the authors remove the instance normalization module from PatchTST and compare against PatchTST also without instance normalization; however, the original PatchTST with instance normalization achieves the best results, and removing instance normalization causes a large deterioration. Since instance normalization is an important component of the model, the claim that PCA is effective for patch-based models is not established. The authors should either integrate PCA with instance normalization in a way that is stable, or clearly state that PCA requires removing a beneficial component and therefore is not a drop-in replacement for all models.
minor comments (5)
- [Table 3, 'Better Count' row] The Better Count numbers do not sum consistently with the number of metric cells: for the Linear/Linear* pair, 19+17=36, but there are only 32 cells (4 datasets × 4 horizons × 2 metrics). This suggests a tabulation error that should be corrected.
- [§6, Conclusion] The conclusion uses the abbreviation 'TSRE' while the rest of the paper uses 'TSER' (time series extrinsic regression); the notation should be made consistent.
- [Figures 4 and 5] The figures report normalized training/inference time and GPU memory usage, but do not specify whether times are per epoch, per full training run, or averaged over which configurations, nor do they include error bars; the captions should clarify these details.
- [§5, Table 7] The FFT and DWT baselines are described as selecting the top k frequency components, but the text does not specify how complex-valued FFT coefficients are handled or whether any reconstruction is performed; the extremely poor MSE values (above 2.0) suggest a possible implementation issue that should be documented.
- [Appendix D] The description of the number of principal components mentions values 16, 48, and 96, but does not provide a complete mapping of which k is used for each dataset and task; a summary table would make the experimental setup reproducible.
Circularity Check
No significant circularity: the central empirical claim is independently tested, with one definitional framing move in the theoretical justification.
-
self definitional
[Section 3.2, 'Intuitional Justifications on PCA's Effectiveness in Time Series Reduction']
"PCA acts as an efficient tool for noise reduction within historical series. By projecting the original historical series onto a new set of orthogonal components, PCA effectively filters out the noise contained in the lower variance components, thus retaining the core information of the historical series."
This is the paper's theoretical justification for why PCA preprocessing preserves model accuracy. The argument reduces to the definition of PCA: 'noise' is equated with the lower-variance components that PCA discards, and 'core information' is equated with the high-variance components that PCA keeps. No independent measure of noise or core information is supplied, so the statement that PCA denoises while retaining essential information is true by construction rather than by derivation. The paper's own Table 2 contradicts the implied assumption that discarded directions are pure noise: TimesNet on SelfRegulationSCP1 drops from 0.918 to 0.686 and Linear on UWaveGestureLibrary drops from 0.575 to 0.409 after PCA.
full rationale
This is an empirical study rather than a derivation, so the central claim does not reduce to its inputs. PCA parameters are fitted on Dtrain and applied without re-estimation to Dval and Dtest (Section 3), and the test metrics in Tables 2-4 are reported honestly, including multiple settings where PCA hurts accuracy, e.g., TimesNet on SelfRegulationSCP1 (0.918 to 0.686) and Linear on UWaveGestureLibrary (0.575 to 0.409). The efficiency gains are a direct consequence of shortening the input from L to k, not a circular prediction. There is no load-bearing self-citation chain: the only self-citation (Gao et al. 2023 in Section 1) supports a peripheral claim about shortened windows and is independently corroborated by the paper's own ablation in Table 5. The one circular-adjacent element is the theoretical justification in Section 3.2, which equates low-variance components with 'noise' and the kept components with 'core information' by definition; this framing is tautological as a proof of accuracy preservation, but the empirical benchmark results remain independent evidence. Appendix B.4 honestly flags a limitation (PatchTST requires removing instance normalization and degrades without it), and Appendix D shows that k is a tuned hyperparameter; neither constitutes circularity. Overall the paper is self-contained against external benchmarks, with only a minor definitional framing issue.
Assumptions & free parameters
free parameters (2)
- k, number of principal components kept per window =
48 (TSF); 16/48/96 (TSC); 48/16 (TSER); 2 per patch (PatchTST)
- top-k frequency components for FFT and DWT baselines =
48, matched to k for PCA
assumptions (4)
- domain assumption PCA parameters fitted on the training set transfer to validation and test windows; the temporal covariance structure is stable across time and splits.
- domain assumption Task-relevant information concentrates in the top-k variance directions; the discarded low-variance subspace is noise or irrelevant.
- ad hoc to paper Linear transformations preserve higher-order moments such as skewness and kurtosis.
- ad hoc to paper Specific trends and periodic patterns in the historical series are not essential for TSA model learning.
Cite this review
Pith. "Pith review of Revisiting PCA for time series reduction in temporal dimension." pith.science (2026). https://pith.science/paper/37RIFPWZ
@misc{pith2026241219423,
author = {Pith},
title = {Pith review of: Revisiting PCA for time series reduction in temporal dimension},
year = {2026},
howpublished = {\url{https://pith.science/paper/37RIFPWZ}},
note = {Machine review of arXiv:2412.19423}
}
read the original abstract
Revisiting PCA for Time Series Reduction in Temporal Dimension; Jiaxin Gao, Wenbo Hu, Yuntian Chen; Deep learning has significantly advanced time series analysis (TSA), enabling the extraction of complex patterns for tasks like classification, forecasting, and regression. Although dimensionality reduction has traditionally focused on the variable space-achieving notable success in minimizing data redundancy and computational complexity-less attention has been paid to reducing the temporal dimension. In this study, we revisit Principal Component Analysis (PCA), a classical dimensionality reduction technique, to explore its utility in temporal dimension reduction for time series data. It is generally thought that applying PCA to the temporal dimension would disrupt temporal dependencies, leading to limited exploration in this area. However, our theoretical analysis and extensive experiments demonstrate that applying PCA to sliding series windows not only maintains model performance, but also enhances computational efficiency. In auto-regressive forecasting, the temporal structure is partially preserved through windowing, and PCA is applied within these windows to denoise the time series while retaining their statistical information. By preprocessing time-series data with PCA, we reduce the temporal dimensionality before feeding it into TSA models such as Linear, Transformer, CNN, and RNN architectures. This approach accelerates training and inference and reduces resource consumption. Notably, PCA improves Informer training and inference speed by up to 40% and decreases GPU memory usage of TimesNet by 30%, without sacrificing model accuracy. Comparative analysis against other reduction methods further highlights the effectiveness of PCA in improving the efficiency of TSA models.
Figures
Figures from the paper (5 more)
Reference graph
Works this paper leans on
-
[6]
Transformers in time series: A survey
Qingsong Wen, Tian Zhou, Chaoli Zhang, Weiqi Chen, Ziqing Ma, Junchi Yan, and Liang Sun. Transformers in time series: A survey. arXiv preprint arXiv:2202.07125,
-
[7]
Jiaxin Gao, Wenbo Hu, and Yuntian Chen. Client: Cross-variable linear integrated enhanced transformer for multivariate long-term time series forecasting. arXiv preprint arXiv:2305.18838,
-
[8]
Table 8: TSC experiments on the UCR datasets
The results demonstrate that PCA preprocessing retains the principal information of the series on the UCR dataset, matches the TSC performance of the original series, and enables faster training and inference. Table 8: TSC experiments on the UCR datasets. The accuracy metric is adopted. The * symbols after models indicate the application of PCA before inp...
work page 2020
-
[9]
Empirical evaluation of gated recurrent neural networks on sequence modeling
Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555,
-
[11]
Sparsetsf: Modeling long-term time series forecasting with 1k parameters
Shengsheng Lin, Weiwei Lin, Wentai Wu, Haojun Chen, and Junjie Yang. Sparsetsf: Modeling long-term time series forecasting with 1k parameters. arXiv preprint arXiv:2405.00946, 2024a. Sanparith Marukatat. Tutorial on pca and approximate pca and approximate kernel pca. Artificial Intelligence Review, 56(6):5445–5477,
-
[12]
Word Emdeddings through Hellinger PCA
Jie Lin, Ting-Zhu Huang, Xi-Le Zhao, Teng-Yu Ji, and Qibin Zhao. Tensor robust kernel pca for multidimensional data. IEEE Transactions on Neural Networks and Learning Systems, 2024b. R´emi Lebret and Ronan Collobert. Word emdeddings through hellinger pca. arXiv preprint arXiv:1312.5542,
-
[13]
How Many Components should be Retained from a Multivariate Time Series PCA?
Alethea Rea and William Rea. How many components should be retained from a multivariate time series pca? arXiv preprint arXiv:1610.03588,
-
[16]
Vector quantized time series generation with a bidirectional prior model
Daesoo Lee, Sara Malacarne, and Erlend Aune. Vector quantized time series generation with a bidirectional prior model. arXiv preprint arXiv:2303.04743,
Show all 23 references
-
[17]
Revisiting PCA for Time Series Reduction in Temporal Dimension
13 arXiv Template A PREPRINT Supplemental Materials for “Revisiting PCA for Time Series Reduction in Temporal Dimension” A Data and Model Description The experimental data comprises 13 widely-used datasets from various domains, each distinguished by unique attributes: • ETT [Z...
2021
-
[18]
oil temperature
There are 7 variables for each dataset, with 17,420 time steps for ETTh and 69,680 time steps for ETTm. The series in these datasets exhibit strong periodicity. For univariate forecasting, only the “oil temperature” variable is used for training and testing. • EthanolConcentra...
2018
-
[23]
The accuracy metric is adopted
Table 10: TSC experiments of T-Loss, TS2Vec, and TimeVQV AE. The accuracy metric is adopted. Bold font is the superior result. Linear+PCA T-Loss TS2Vec TimeVQV AE EthanolConcentration 0.300 0.289 0.287 0.203 Handwriting 0.127 0.255 0.397 0.218 SelfRegulationSCP1 0.805 0.780 0....
2022
-
[84]
The objective of this dataset is to predict the COVID-19 death rate on April 1, 2020, for each country using the daily confirmed cases over the preceding three months. The descriptions and implementations of the evaluated time series models are provided below: Linear [Zeng et ...
2020
-
[315]
• FloodModeling [Tan et al., 2021]: FloodModeling comprises three hourly datasets (FloodModeling1, Flood- Modeling2, and FloodModeling3)
In the experiments, the analysis is focused on the z-coordinate series. • FloodModeling [Tan et al., 2021]: FloodModeling comprises three hourly datasets (FloodModeling1, Flood- Modeling2, and FloodModeling3). These datasets aim to predict the maximum water depth for flood mod...
2021
-
[896]
SelfRegulationSCP2 comprises 380 series samples, each with a length of 1,152
In SelfRegulationSCP2, data from an artificially respirated ALS patient similarly involve cursor movement, with auditory and visual feedback regulating slow cortical potentials. SelfRegulationSCP2 comprises 380 series samples, each with a length of 1,152. The classification ob...
2018
-
[2009]
Fast and accurate time series classification with weasel
Patrick Sch¨afer and Ulf Leser. Fast and accurate time series classification with weasel. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, pages 637–646,
2017
-
[2015]
Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu
doi: 10.1109/ICDMW.2015.104. Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 11121–11128,
2015 doi
-
[2017]
Fredformer: Frequency debiased transformer for time series forecasting
Xihao Piao, Zheng Chen, Taichi Murayama, Yasuko Matsubara, and Yasushi Sakurai. Fredformer: Frequency debiased transformer for time series forecasting. arXiv preprint arXiv:2406.09009,
-
[2018]
arXiv preprint arXiv:1811.00075,
-
[2019]
Inceptiontime: Finding alexnet for time series classification
Hassan Ismail Fawaz, Benjamin Lucas, Germain Forestier, Charlotte Pelletier, Daniel F Schmidt, Jonathan Weber, Geoffrey I Webb, Lhassane Idoumghar, Pierre-Alain Muller, and Franc ¸ois Petitjean. Inceptiontime: Finding alexnet for time series classification. Data Mining and Kno...
1936
-
[2021]
Transformer multivariate forecasting: Less is more? arXiv preprint arXiv:2401.00230,
Jingjing Xu, Caesar Wu, Yuan-Fang Li, and Pascal Bouvry. Transformer multivariate forecasting: Less is more? arXiv preprint arXiv:2401.00230,
-
[2022]
Mts-mixers: Multivariate time series forecasting via factorized temporal and channel mixing
Zhe Li, Zhongwen Rao, Lujia Pan, and Zenglin Xu. Mts-mixers: Multivariate time series forecasting via factorized temporal and channel mixing. arXiv preprint arXiv:2302.04501,
-
[2023]
A time series is worth 64 words: Long-term forecasting with transformers
Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. arXiv preprint arXiv:2211.14730,
-
[2024]
Tslanet: Rethinking transformers for time series representation learning
Emadeldeen Eldele, Mohamed Ragab, Zhenghua Chen, Min Wu, and Xiaoli Li. Tslanet: Rethinking transformers for time series representation learning. arXiv preprint arXiv:2404.08472,
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.