REVIEW 3 major objections 4 minor 7 references
A Comparative Analysis of Principal Component Analysis (PCA) and Singular Value Decomposition (SVD) as Dimensionality Reduction Techniques
T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read PCA and SVD share a common optimization framework, but centering, data shape, and numerical conditioning determine which one to use.
desk verdict Standard textbook comparison of PCA and SVD with honest limitations but sloppy dimension errors; not a research contribution, but could be a useful expository note after corrections. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is the pair of symmetric decompositions: PCA diagonalizes the covariance matrix $C = X_c^T X_c/(n-1)$ after mean-centering the data, while SVD factors the data matrix directly as $X = U\Lambda V^T$. The comparison runs through the relationship between the eigenvalues of $X_c^T X_c$ and the squared singular values of $X_c$, which makes PCA and SVD share a common optimisation target; the analysis then traces how centering, matrix shape, and conditioning enter the two algorithms differently. The paper's table of qualitative trade-offs is the output of this comparative derivation.
What would settle it
Take a highly rectangular, ill-conditioned data matrix, compute its covariance matrix, and compare the numerical error in the top eigenvector from eigen-decomposition with the error in the top right singular vector from a direct SVD of the centered matrix; if SVD is not more accurate, the paper's stability ranking fails.
Extended reading notes
Core claim
The paper's central claim is that PCA and SVD are two views of the same underlying optimisation problem, and that their practical differences trace to three specific choices: whether the data is centered, the aspect ratio of the data matrix, and the numerical conditioning of the computation. From first-principles derivations, the paper concludes that PCA requires centering and is best suited to near-square matrices, while SVD does not require centering, is better suited to highly rectangular matrices, and offers higher numerical stability. The account is explicitly analytical; the guidelines are asserted as qualitative rules of thumb, with the absence of experiments listed as a limitation.
Load-bearing premise
The load-bearing premise is that the qualitative contrasts in Table 1 (centering, shape preference, and stability ranking) follow directly from the eigen-decomposition and SVD formulas presented in Sections 2 and 3, so a practitioner can trust these rules of thumb without experimental confirmation.
Editorial extensions
If this is right
- If the comparison is correct, practitioners can justify PCA-versus-SVD choices without running benchmarks, using only matrix shape and whether the data is centered.
- PCA's variance interpretation holds only after centering; without centering, the first component conflates the mean with the direction of greatest spread.
- SVD extends naturally to rectangular data and avoids forming the covariance matrix explicitly, which is where its numerical stability advantage is expected to come from.
- On centered data the two methods converge: PCA's principal directions are the right singular vectors of the centered matrix, so the residual differences lie in centering and the numerical path taken to compute the decomposition.
Reading between the lines
- A testable extension would be a benchmark comparing PCA via eigen-decomposition of the covariance matrix against SVD on a family of matrices with controlled aspect ratio and condition number, measuring reconstruction error and runtime; the paper itself calls for such experiments.
- The aspect-ratio guideline could be sharpened: PCA forms an $n \times n$ covariance matrix, so for $m \gg n$ the eigen-decomposition is often cheaper than a full SVD's $m$-column $U$; the 'near-square' preference may really be a computational-cost statement rather than an accuracy statement.
- The stability comparison depends on whether one computes the covariance matrix explicitly; using randomized algorithms for the SVD could change the trade-off at large scale.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This manuscript presents a purely analytical comparison of Principal Component Analysis (PCA) and Singular Value Decomposition (SVD) for dimensionality reduction of image data. It derives PCA from the covariance matrix eigencomposition, describes SVD as a matrix factorization, compares the two methods in Table 1, and offers rule-of-thumb guidelines for choosing between them based on data centering, matrix shape, and numerical stability. No experiments are reported; the authors explicitly frame the paper as a theoretical precursor to future empirical benchmarking.
Significance. If the derivations were correct and the comparative claims substantiated, the paper would serve as a concise pedagogical reference for practitioners selecting between PCA and SVD. However, the central analytical contribution is not currently demonstrated: Section 2.2 contains dimensional errors that invalidate the PCA derivation, the algebraic link between PCA and SVD is never stated, and Table 1's guidelines are asserted without proof or citation. The paper's honest statement of limitations (no experimental validation) is a strength, but the concluding claim that 'we have shown that PCA and SVD share a common optimisation framework' overstates what the manuscript actually proves. The topic is classical and the incremental contribution is mainly expository, so the paper's value hinges entirely on the correctness and completeness of the derivations.
major comments (3)
- [§2.2] The dimensions in the PCA derivation are inconsistent. After defining X_c∈R^{m×n}, the covariance matrix C = X_c^T X_c/(n−1) must be n×n (assuming features are columns), but the text states C∈R^{m×?} and later calls Q∈R^{m×n} an eigenvector matrix of C. Eigenvectors of an n×n matrix should form an n×n matrix. Consequently, the projection X_c Q_s is R^{m×s}, not R^{n×s} as written. These dimensional errors invalidate the PCA derivation as presented and must be corrected.
- [§3.1, §6] The claimed common optimisation framework is never proved. Section 3.1 introduces the SVD factorization A = UDV^T but never states the algebraic identity A^T A = V D^2 V^T, which is what connects the right singular vectors of X_c to the eigenvectors of the covariance matrix C. Without this identity, the conclusion in Section 6 that PCA and SVD share a common optimisation framework is an assertion, not a demonstrated result. The paper should explicitly state and prove this connection.
- [Table 1] The three qualitative entries in Table 1 (data centering requirement, best matrix shape, numerical stability) are introduced as trade-offs that 'emerge from the preceding derivations,' but no supporting argument is given. For example, the numerical stability ordering would require a conditioning analysis of the eigendecomposition versus the SVD, and the aspect-ratio guidance would require a cost comparison of forming and decomposing the covariance matrix versus computing a direct SVD. These entries must either be derived within the paper or explicitly attributed to the cited literature as rules of thumb; as written, they are unsupported assertions.
minor comments (4)
- [Throughout] There are several typographical errors, including 'matric' in Section 2.2, 'ful-sized' in the Introduction, and 'rail of toward noise' in Section 3.1. The manuscript would benefit from a careful proofreading pass.
- [References] References [5], [6], and [7] (QRPCA, Tropp–Webber, and KPCA-CAM) are listed in the bibliography but are not cited anywhere in the text. They should be cited in the relevant sections or removed.
- [§3.1] The notation for singular values is inconsistent: the text defines D as the diagonal matrix of singular values but later refers to a 'diagonal vector Λ,' and the rank is written as 'r = rank(x)' with a lowercase matrix symbol. These notations should be unified.
- [§4] The comparative analysis consists only of Table 1 without any accompanying discussion. A short paragraph explaining each row would help readers understand how the entries follow from the formulas in Sections 2 and 3.
Circularity Check
No circularity; the paper's weaknesses are unsupported derivations and borrowed heuristics, not a self-referential or fitted-parameter loop.
full rationale
The paper makes no empirical prediction, fits no parameter to data, and does not invoke a load-bearing self-citation. Its derivation chain is a textbook restatement: PCA is presented as an eigendecomposition of the centered covariance matrix, and SVD as a factorization X = UΛV^T. The conclusion that the two methods 'share a common optimisation framework' is asserted rather than proven, and the qualitative entries in Table 1 (near-square matrices for PCA, highly rectangular for SVD, higher numerical stability for SVD) are stated without a derivation from the preceding formulas. These are genuine analytical gaps and correctness risks: Section 2.2 contains dimensional inconsistencies (e.g., C ∈ R^{m×} and X_c Q_s ∈ R^{n×s} are incompatible with X_c ∈ R^{m×n}), and the algebraic link X^T X = VΣ^2 V^T that would justify the common-framework claim is never written down. However, none of these problems is a circular step under the review definition. The paper does not define PCA in terms of SVD, does not fit an input to produce a closely related output, and does not rely on a uniqueness theorem or ansatz smuggled in by citation. The self-declared limitation that 'This study provides no experimental validation' is an honest admission of scope, not an attempt to pass a fitted quantity off as a prediction. The central claim is under-supported borrowing of standard knowledge, which is a rigor defect, not a self-referential reduction; accordingly the circularity score is 0.
Assumptions & free parameters
assumptions (3)
- standard math Every real matrix has an SVD, and the spectral theorem guarantees eigendecomposition of symmetric matrices.
- domain assumption For PCA, the data matrix X is mean-centered by subtracting column means and the covariance matrix is X_c^T X_c/(n-1).
- domain assumption Image data are represented as matrices or tensors with pixel intensity values.
Cite this review
Pith. "Pith review of A Comparative Analysis of Principal Component Analysis (PCA) and Singular Value Decomposition (SVD) as Dimensionality Reduction Techniques." pith.science (2026). https://pith.science/paper/VIP3HUZP
@misc{pith2026250616663,
author = {Pith},
title = {Pith review of: A Comparative Analysis of Principal Component Analysis (PCA) and Singular Value Decomposition (SVD) as Dimensionality Reduction Techniques},
year = {2026},
howpublished = {\url{https://pith.science/paper/VIP3HUZP}},
note = {Machine review of arXiv:2506.16663}
}
read the original abstract
High-dimensional image data often require dimensionality reduction before further analysis. This paper provides a purely analytical comparison of two linear techniques-Principal Component Analysis (PCA) and Singular Value Decomposition (SVD). After the derivation of each algorithm from first principles, we assess their interpretability, numerical stability, and suitability for differing matrix shapes. We synthesize rule-of-thumb guidelines for choosing one out of the two algorithms without empirical benchmarking, building on classical and recent numerical literature. Limitations and directions for future experimental work are outlined at the end.
Reference graph
Works this paper leans on
-
[1]
On Lines and Planes of Closest Fit to Systems of Points in Space,
K. Pearson, “On Lines and Planes of Closest Fit to Systems of Points in Space, ”Philosoph- ical Magazine, vol. 2, pp. 559–572, 1901
work page 1901
-
[2]
The Approxima- tion of One Matrix by Another of Lower Rank,
C. Eckart and G. Young, “The Approxima- tion of One Matrix by Another of Lower Rank, ”Psychometrika, vol. 1, no. 3, pp. 211– 218, 1936
work page 1936
-
[3]
Singular Value Decomposition and Least Squares So- lutions,
G. H. Golub and C. Reinsch, “Singular Value Decomposition and Least Squares So- lutions, ”Numerische Mathematik, vol. 14, pp. 403–420, 1970
work page 1970
-
[4]
Principal Com- ponent Analysis: A Review and Recent De- velopments,
I. T. Jolliffe and J. Cadima, “Principal Com- ponent Analysis: A Review and Recent De- velopments, ” Philosophical Transactions of the Royal Society A , vol. 374, no. 2065, p. 20150202, 2016. doi:10.1098/rsta.2015.0202
arXiv 2016
-
[5]
QRPCA: A Pack- age for Fast Principal Component Anal- ysis with GPU Acceleration,
R. S. de Souza, X. Quanfeng, S. Shen, C. Peng, and Z. Mu, “QRPCA: A Pack- age for Fast Principal Component Anal- ysis with GPU Acceleration, ” Astronomy and Computing , vol. 41, p. 100633, 2022. doi:10.1016/j.ascom.2022.100633 Wingate University (2025) 3
-
[6]
Randomized Algorithms for Low-Rank Matrix Approxi- mation: Design, Analysis, and Applications,
J. A. Tropp and R. J. Webber, “Randomized Algorithms for Low-Rank Matrix Approxi- mation: Design, Analysis, and Applications, ” arXiv Preprint arXiv:2306.12418, 2023
arXiv 2023
-
[7]
KPCA-CAM: Visual Explainability of Deep Computer Vision Models Using Kernel PCA,
S. Karmani, T. Sivakaran, G. Prasad, M. Ali, W. Yang, and S. Tang, “KPCA-CAM: Visual Explainability of Deep Computer Vision Models Using Kernel PCA, ” in 2024 IEEE International Workshop on Multi- media Signal Processing , pp. 1–6, 2024. doi:10.1109/MMSP59620.2024.10345678 4 Wingate University (2025)
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.