Pith. sign in

REVIEW 2 major objections 4 minor 44 references

Spectral analysis of large dimensional Chatterjee's rank correlation matrix

T0 review · 2 major / 4 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Under independence, the symmetrized Chatterjee rank correlation matrix has an empirical spectral distribution that converges almost surely to Wigner's semicircle law, marking the first correlation matrix known to leave the Marchenko–Pastur

desk verdict The semicircle limit for Chatterjee's rank correlation matrix is real and novel, but Theorem 1.3 as displayed has an n-prefactor that makes it false; the fix is clear and the paper deserves peer review. read the letter →

arxiv 2510.07262 v2 pith:KURZQ5HQ submitted 2025-10-08 math.ST math.PRstat.TH

Zhaorui Dong , Fang Han , Jianfeng Yao This is my paper · ORCID
classification math.STmath.PRstat.TH MSC 62H1560B2062H10
keywords Chatterjee'srankcorrelationrelativeranksempiricalspectraldistributionsemicirclelawMarchenko–Pasturlinearstatisticsindependencetestingrandompermutations
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper studies the spectrum of the matrix of Chatterjee rank correlations computed from n independent observations of p independent continuous variables, in the regime where p/n converges to a positive constant. It claims that the symmetrized version of this matrix has an empirical spectral distribution converging almost surely to the semicircle law centered at 1 with radius 2 times the square root of gamma over 5. This would be the first example of a large correlation matrix whose limiting spectrum is semicircular rather than Marchenko–Pastur, showing that rank-based dependence matrices can belong to a different random matrix universality class. The paper further establishes that the squared deviation matrix follows a Marchenko–Pastur law, and proves a central limit theorem for its linear spectral statistics, which it uses to build tests of complete independence among the components.

What carries the argument

The proof rides on the dependence structure of relative ranks, the permutations sigma_v composed with sigma_u^{-1} that convert one column's ranks into another's. Proposition 2.2 characterizes when such relative ranks are mutually independent: exactly when the underlying undirected graph is a forest of non-overlapping trees. This tree-independence property lets the moment method match the classical Wigner and Wishart tree contributions, with each entry contributing the limiting variance 2/5, while a sharp bound on non-tree graphs (Proposition 4.4) rules out everything else.

What would settle it

Simulate the null model with n = 200 and p = 100 under independent standard normals, compute the eigenvalues of Phi_n, and compare the histogram to the semicircle density W(1, 2*sqrt(0.4)); a systematic mismatch in support or in odd trace moments would contradict Theorem 1.1. Alternatively, enumerate all n = 3 relative-rank configurations on a directed triangle and check whether the joint law of the three relative-rank permutations equals the product of their marginals; any discrepancy would invalidate the tree-independence lemma that carries the proof.

Watch

Extended reading notes

Core claim

The central claim is Theorem 1.1: under the assumption of independent continuous margins and p/n to gamma in (0, infinity), the empirical spectral distribution of Phi_n = (Xi_n + Xi_n^T)/2 converges almost surely to the semicircle law W(1, 2*sqrt(gamma/5)). The companion Theorem 1.2 states that Psi_n = (Xi_n - I_p)(Xi_n - I_p)^T converges to the Marchenko–Pastur law MP(1, 2 gamma/5). The paper also proves a central limit theorem for the centered traces of powers of Psi_n, with a closed-form covariance function, and applies it to construct tests of complete independence that are consistent against pairwise dependence and show high empirical power against nonlinear, zero-linear-correlation alt

Load-bearing premise

Everything rests on the claim that relative-rank permutations are independent exactly when the index graph is a forest of trees; if cycles or repeated edges introduce dependence that the proof's bounds do not capture, the moment calculations and both limiting laws fail.

Editorial extensions

If this is right

  • For any ratio gamma = p/n, the bulk spectrum of the symmetrized Chatterjee matrix is explicitly known, centered at 1 with radius 2*sqrt(gamma/5).
  • The squared deviation matrix follows the Marchenko–Pastur law with parameter 2 gamma/5, giving a complete spectral description of both symmetrizations.
  • The central limit theorem for linear spectral statistics provides Gaussian limits for traces of all powers of Psi_n, with an explicit covariance formula.
  • The resulting tests based on tr(Psi_n) and tr(Psi_n^2) detect dependence that Pearson, Spearman, and Kendall-type tests miss, including oscillatory and W-shaped alternatives with zero linear correlation.
  • The expectation E tr(Psi_n) is distribution-free and given by p(p-1)(n-2)(4n-7)/(10(n-1)^2(n+1)), allowing straightforward calibration.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The semicircle limit suggests a broader universality principle: any rank-based dependence matrix whose entries are functions of relative ranks with the same tree-independence structure and entry variance 2/5 may share the same limiting spectrum, not just Chatterjee's coefficient.
  • One testable extension is to replace Chatterjee's oscillation statistic with other permutation statistics inside the same matrix structure; the proof strategy would carry over as long as the tree independence and zero-mean cancellation lemmas hold.
  • The covariance formula in Theorem 1.3 could be used to compute asymptotic power against local alternatives where the dependence strength shrinks at rate 1/sqrt(n), something the paper does not pursue.
  • If the tree-independence characterization holds for more general random permutation models, the same graph-counting machinery might apply to conditional or grouped dependence structures, though the paper is limited to independent continuous margins.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper studies the spectral behaviour of the large-dimensional Chatterjee rank correlation matrix under independence of the p coordinates and the proportional-growth regime p/n → γ ∈ (0,∞). The main theoretical results are: (1) the empirical spectral distribution of the symmetrized matrix Φ_n = (Ξ_n + Ξ_nᵀ)/2 converges almost surely to the semicircle law W(1, 2√(γ/5)); (2) the ESD of Ψ_n = (Ξ_n − I_p)(Ξ_n − I_p)ᵀ converges to MP(1, 2γ/5); and (3) a CLT is stated for linear spectral statistics of Ψ_n, i.e. for centered traces tr(Ψ_n^k), with an explicit covariance function. The paper also proposes two independence tests based on these LSSs and provides simulations. The proofs use the moment method, a new combinatorial characterization of the dependence structure of relative rank permutations, and delicate graph enumeration.

Significance. If the results are correct, this is a substantive contribution: it appears to be the first example of a large correlation matrix whose ESD is non-Marchenko–Pastur, and the explicit LSS CLT provides new nonparametric independence tests. The combinatorial independence result for relative ranks (Proposition 2.2) and the detailed graph enumeration are interesting technical tools in their own right. The paper is careful to separate external bivariate asymptotics from the new spectral combinatorics, and the numerical figures support the ESD limits. However, the central CLT as stated in Theorem 1.3 has a scaling error that makes the statement false as written; this must be corrected before the result can be accepted. The application section depends exactly on this theorem, so the error is load-bearing.

major comments (2)
  1. [Theorem 1.3(i) and Propositions 4.9–4.10] The statement of Theorem 1.3(i) displays n(tr Ψ_n^k − E tr Ψ_n^k) ⇒ G_k with an O(1) covariance function. This is internally inconsistent: Proposition 4.9 proves lim Var(tr Ψ_n^k) is a finite constant depending only on γ and k, and Proposition 4.10 concludes convergence for the un-scaled vector {tr Ψ_n^k − E tr Ψ_n^k}. The test statistics Q_{ξ,2} and Q_{ξ,4} in Section 3 use no n-prefactor, and Figure 3 shows centered tr(Ψ^k) with O(1) spread (e.g. k=1 ranges about ±4 for n=200, p=500), which would be multiplied by n if the displayed scaling were correct. Thus Theorem 1.3(i) is false as written. The correct statement is the one proved in Proposition 4.10, with the leading n removed. Because the tests in Section 3 are justified by Theorem 1.3, this correction is essential.
  2. [Section 2.1, Proposition 2.2 proof] The proof of Proposition 2.2 says 'If ∆⁰ is not a tree, then it contains a cycle...' but the proposition is about the underlying undirected multigraph ∆ᵘ, not the skeleton ∆⁰. A pair of parallel directed edges, e.g. (u,v) and (u,v), gives ∆⁰ a tree but ∆ᵘ a non-tree and the two random variables are identical (hence dependent). The same issue arises with opposite directed edges (u,v) and (v,u). This is a gap in the proof as written. The intended use in the paper is for tree graphs with no parallel edges, so the main theorems may be unaffected, but the statement and proof need to be aligned, either by explicitly excluding parallel edges in the 'if' direction or by handling the parallel-edge case.
minor comments (4)
  1. [Proposition 4.4 statement] The exponent `#E(∆0)∧(#E(∆)+Q0)/2` is ambiguous; use parentheses such as `min(#E(∆0), (#E(∆)+Q0)/2)` for clarity.
  2. [Figures 1–2 captions] The captions read 'Histogram of Eigenvalues of n'; this appears to be a typographical remnant and should read 'Φ_n' and 'Ψ_n' respectively.
  3. [Theorem 1.3(ii)] The statement that E tr(Ψ_n^k) is 'distribution-free and can be computed numerically' is vague for k>1; an explicit numerical procedure or a reference to the simulation method used for Q_{ξ,4} would help.
  4. [Section 2.4 illustrative example] Equation (2.5) is stated without derivation of the counting constants; a brief explanation of the 1/100 factors would make the example more useful as a pedagogical preview.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the semicircle/MP limits and LSS CLT are derived by internal moment combinatorics from external bivariate estimates.

full rationale

The central claims are not assumed as inputs. Under Assumption 1.1 the rankings are i.i.d. uniform permutations; Proposition 2.2, the load-bearing characterization of dependence among relative ranks, is proved in Section 5.1.1 by a direct induction on tree size, not imported from a self-citation. The external results actually used (Chatterjee 2021, Zhang 2023, Xia et al. 2025) are bivariate limits, variances and second moments of individual entries; they contain no semicircle, Marchenko-Pastur, or LSS covariance result, so using them does not reduce the target theorems to themselves. The semicircle and MP limits follow from the paper's own Propositions 4.5-4.8 via the moment method, and Theorem 1.3's Gaussianity is derived internally in Proposition 4.10 via Wick's formula and graph counting. The cited works involving the authors (Drton et al. 2020, Han 2024, Lin & Han 2022) supply standard facts or proof techniques, but none is the source of the claimed spectral conclusions. The apparent n-prefactor mismatch in Theorem 1.3 relative to Propositions 4.9-4.10 is a scaling/correctness concern, not a circularity: the variance proof does not assume the theorem statement. Overall I find no step where a fitted parameter or a self-citation is renamed as a prediction.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

No free parameters are fitted: γ is the asymptotic p/n input, and no ad hoc constants are introduced. The central proofs rely on standard moment-method tools, prior bivariate moment/CLT results for Chatterjee's coefficient, and the internally proved tree-independence structure. No new physical or statistical entities are postulated.

assumptions (5)
  • domain assumption The sample ranks of n independent copies of a random vector with continuous independent margins are p independent uniform random permutations.
    Invoked throughout Section 2.1 after Assumption 1.1; false under dependence or discrete margins, and the null spectral results are derived under it.
  • standard math Moment convergence criteria of Riesz and Carleman imply weak convergence of ESD from convergence of expected moments plus variance bounds.
    Used in the proofs of Theorems 1.1 and 1.2, as stated at the start of Section 4.1.
  • standard math Wick's formula characterizes Gaussian moments and is used to identify the limit of centered LSS as Gaussian.
    Used in the proof of Proposition 4.10 to conclude Gaussianity from pair partitions.
  • domain assumption External bivariate asymptotic results from Chatterjee (2021), Zhang (2023), and Xia et al. (2025) provide CLT and moment formulas for single Chatterjee coefficients under independence.
    Used for entry variances and expectations, e.g., E[Ξ^2], E[Ξ12Ξ21], in Propositions 4.5, 4.7, and 4.9. These are prior literature results, independent of the target spectral limit.
  • standard math Combinatorial identity of Bowman and Regev (Lemma 4.6) for sums of products of Catalan numbers.
    Used in Proposition 4.9 to count one-cycle graph contributions to the covariance function.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Spectral analysis of large dimensional Chatterjee's rank correlation matrix." pith.science (2026). https://pith.science/paper/KURZQ5HQ

@misc{pith2026251007262,
  author       = {Pith},
  title        = {Pith review of: Spectral analysis of large dimensional Chatterjee's rank correlation matrix},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KURZQ5HQ}},
  note         = {Machine review of arXiv:2510.07262}
}
read the original abstract

This paper studies the spectral behavior of large dimensional Chatterjee's rank correlation matrix when observations are independent draws from a high-dimensional random vector with independent continuous components. Limits for the empirical spectral distributions of its two symmetrized versions are established in the proportional high-dimensional regime, one of them being the semicircle law, thereby giving a first example of a correlation matrix with a non-Marchenko--Pastur spectral limit, in contrast to the Pearson, Kendall, and Spearman cases. We further establish central limit theorems for linear spectral statistics of the symmetrized matrices. As an important application of this theory, we develop Chatterjee's rank correlation-based tests for the complete independence among the components.

Figures

Figures reproduced from arXiv: 2510.07262 by the authors.

Figure 1
Figure 1. Semicircle law of Φn with p = 100 and n = 200. Histogram of Eigenvalues of n 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 Eigenvalue 0 1 2 3 4 5 6 7 8 9 Density [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 3
Figure 3. Histograms of tr(Ψk n ) centered by sample means, with p = 500, n = 200, and over 500 replications. 2 Technical tools and proof strategy In the course of our proofs, we develop several key technical tools that are of independent interest beyond this work. We first highlight these tools in this section, and then provide an overview of the main proof strategy. 2.1 Rankings and random permutations Chatterjee’s rank cor… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 6 linked inside Pith

  1. [1]

    Angus, J. E. (1995). A coupling proof of the asymptotic normality of the permutation oscillation. Probability in the Engineering and Informational Sciences , 9(4):615--621

    1995

  2. [2]

    and Fuchs, S

    Ansari, J. and Fuchs, S. (2025). A direct extension of A zadkia & C hatterjee's rank correlation to multi-response vectors. arXiv preprint arXiv:2212.01621

    arXiv 2025

  3. [3]

    Arnold, L. (1971). On W igner's semicircle law for the eigenvalues of random matrices. Zeitschrift f \"u r Wahrscheinlichkeitstheorie und verwandte Gebiete , 19(3):191--198

    1971

  4. [4]

    Auddy, A., Deb, N., and Nandy, S. (2024). Exact detection thresholds and minimax optimality of C hatterjee's correlation coefficient. Bernoulli , 30(2):1640--1668

    2024

  5. [5]

    and Chatterjee, S

    Azadkia, M. and Chatterjee, S. (2021). A simple measure of conditional dependence . The Annals of Statistics , 49(6):3070--3102

    2021

  6. [6]

    Azadkia, M., Chen, L., and Han, F. (2025). Bias correction for C hatterjee's graph-based correlation coefficient. arXiv preprint arXiv:2508.09040

    arXiv 2025

  7. [7]

    and Silverstein, J

    Bai, Z. and Silverstein, J. W. (2010). Spectral Analysis of Large Dimensional Random Matrices . Springer

    2010

  8. [8]

    and Zhou, W

    Bai, Z. and Zhou, W. (2008). Large sample covariance matrices without independence structures in columns. Statistica Sinica , 18(2):425--442

    2008

Show all 44 references
  1. [9]

    S., Lodhia, A., and Rigollet, P

    Bandeira, A. S., Lodhia, A., and Rigollet, P. (2017). Mar c enko- P astur law for K endall's tau. Electronic Communication in Probability , 22:1--7

    2017

  2. [10]

    Bao, Z., Lin, L.-C., Pan, G., and Zhou, W. (2015). Spectral statistics of large dimensional spearman's rank correlation matrix and its application. Annals of Statistics , 43(6):2588--2623

    2015

  3. [11]

    and Dassios, A

    Bergsma, W. and Dassios, A. (2014). A consistent test of independence based on a sign covariance related to K endall's tau. Bernoulli , 20(2):1006--1028

    2014

  4. [12]

    Bickel, P. J. (2022). Measures of independence and functional dependence. arXiv preprint arXiv:2206.13663

    2022 arXiv

  5. [13]

    R., Kiefer, J., and Rosenblatt, M

    Blum, J. R., Kiefer, J., and Rosenblatt, M. (1961). Distribution free tests of independence based on the sample distribution function. Annals of Mathematical Statistics , 32(2):485--498

    1961

  6. [14]

    and Regev, A

    Bowman, D. and Regev, A. (2014). Counting symmetry classes of dissections of a convex regular polygon. Advances in Applied Mathematics , 56:35--55

    2014

  7. [15]

    and Dette, H

    B \"u cher, A. and Dette, H. (2024). On the lack of weak continuity of C hatterjee's correlation coefficient. arXiv preprint arXiv:2410.11418

    2024 arXiv

  8. [16]

    Cai, T. T. and Ma, Z. (2013). Optimal hypothesis testing for high dimensional covariance matrices. Bernoulli , 19(5B):2359--2388

    2013

  9. [17]

    Chatterjee, S. (2008). A new method of normal approximation. Annals of Probability , 36(4):1584--1610

    2008

  10. [18]

    Chatterjee, S. (2021). A new coefficient of correlation. Journal of the American Statistical Association , 116(535):2009--2022

    2021

  11. [19]

    Deb, N., Ghosal, P., and Sen, B. (2020). Measuring association on topological spaces using kernels and geometric graphs. arXiv preprint arXiv:2010.01768

    2020 arXiv

  12. [20]

    F., and Stoimenov, P

    Dette, H., Siburg, K. F., and Stoimenov, P. A. (2013). A copula-based non-parametric measure of regression dependence. Scandinavian Journal of Statistics , 40(1):21--41

    2013

  13. [21]

    Drton, M., Han, F., and Shi, H. (2020). High-dimensional consistent independence testing with maxima of rank correlations. Annals of Statistics , 48(6):3206--3227

    2020

  14. [22]

    Grenander, U. (1963). Probabilities on Algebraic Structures . John Wiley & Sons, Inc

    1963

  15. [23]

    Han, F. (2024). An introduction to permutation processes (version 0.5). arXiv preprint arXiv:2407.09664

    2024 arXiv

  16. [24]

    Han, F., Chen, S., and Liu, H. (2017). Distribution-free tests of independence in high dimensions. Biometrika , 104(4):813--828

    2017

  17. [25]

    and Huang, Z

    Han, F. and Huang, Z. (2024). Azadkia-- C hatterjee's correlation coefficient adapts to manifold data. Annals of Applied Probability , 34(6):5172--5210

    2024

  18. [26]

    Hoeffding, W. (1948). A non-parametric test of independence. Annals of Mathematical Statistics , 19(4):546--557

    1948

  19. [27]

    Jiang, T. (2004). The limiting distributions of eigenvalues of sample correlation matrices. Sankhy \=a : The Indian Journal of Statistics , 66(1):35--48

    2004

  20. [28]

    and Drton, M

    Leung, D. and Drton, M. (2018). Testing independence in high dimensions with sums of rank correlations. The Annals of Statistics , 46(1):280--307

    2018

  21. [29]

    Li, Z., Wang, Q., and Li, R. (2021). Central limit theorem for linear spectral statistics of large dimensional kendall's rank correlation matrices and its applications. The Annals of Statistics , 49(3):1569--1593

    2021

  22. [30]

    and Han, F

    Lin, Z. and Han, F. (2022). Limit theorems of C hatterjee's rank correlation. arXiv.org preprint

    2022

  23. [31]

    and Han, F

    Lin, Z. and Han, F. (2023). On boosting the power of C hatterjee's rank correlation. Biometrika , 110(2):283--299

    2023

  24. [32]

    and Han, F

    Lin, Z. and Han, F. (2024). On the failure of the bootstrap for C hatterjee's rank correlation. Biometrika , 111(3):1063--1070

    2024

  25. [33]

    Mar c enko, V. A. and Pastur, L. A. (1967). Distribution of eigenvalues for some sets of random matrices. Mathematics of the USSR-Sbornik , 1(4):457

    1967

  26. [34]

    Schott, J. R. (2005). Testing for complete independence in high dimensions. Biometrika , 92(4):951--956

    2005

  27. [35]

    Shi, H., Drton, M., and Han, F. (2022). On the power of Chatterjee's rank correlation . Biometrika , 109(2):317--333

    2022

  28. [36]

    Shi, H., Drton, M., and Han, F. (2024). On A zadkia- C hatterjee's conditional dependence coefficient. Bernoulli , 30(2):851--877

    2024

  29. [37]

    and Han, F

    Tran, L. and Han, F. (2024). On a rank-based A zadkia- C hatterjee correlation coefficient. arXiv preprint arXiv:2412.02668

    2024 arXiv

  30. [38]

    Weihs, L., Drton, M., and Meinshausen, N. (2018). Symmetric rank covariances: a generalized framework for nonparametric measures of dependence. Biometrika , 105(3):547--562

    2018

  31. [39]

    Wick, G. C. (1950). The evaluation of the collision matrix. Phys. Rev. , 80:268--272

    1950

  32. [40]

    Wigner, E. P. (1958). On the distribution of the roots of certain symmetric matrices. Annals of Mathematics , 67(2):325--327

    1958

  33. [41]

    Xia, L., Cao, R., Du, J., and Dai, J. (2025). Consistent complete independence test in high dimensions based on C hatterjee correlation coefficient. Statistical Papers , 66(1):1--32

    2025

  34. [42]

    Yin, Y., Zheng, S., and Zou, T. (2023). Central limit theorem of linear spectral statistics of high-dimensional sample correlation matrices. Bernoulli , 29(2):984--1006

    2023

  35. [43]

    Zhang, Q. (2023). On the asymptotic null distribution of the symmetrized C hatterjee's correlation coefficient. Statistics & Probability Letters , 194:109759

    2023

  36. [44]

    write newline

    " write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence '...

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.