pith. sign in

arxiv: 2606.26351 · v1 · pith:HKSQA3E5new · submitted 2026-06-24 · 📊 stat.ME

Robust Estimation of Polychoric Correlation for Complex Survey Designs Using Minimum Divergence Methods

Siqi Wei , David Kepplinger , Anand N. Vidyashankar This is my paper

Pith reviewed 2026-06-26 01:05 UTC · model grok-4.3

classification 📊 stat.ME
keywords polychoric correlationrobust estimationcomplex survey designsminimum divergenceHellinger distancenegative exponential disparityHorvitz-Thompson estimation
0
0 comments X

The pith

Minimum divergence estimators using Hellinger distance and negative exponential disparity yield robust polychoric correlation estimates for complex survey data.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops estimators for polychoric correlation that resist contamination in survey responses by minimizing divergence measures instead of maximizing likelihood. It incorporates complex survey designs by adjusting cell frequencies with Horvitz-Thompson weights derived from the sampling scheme. Penalized versions of the Hellinger approach regularize nuisance parameters while leaving the target correlation unpenalized, and both estimator families are shown to be consistent and asymptotically normal with a design-adjusted sandwich covariance. Simulations across contamination patterns demonstrate that the Hellinger variants perform best under concordant contamination while negative exponential disparity handles discordant cases more effectively.

Core claim

Robust estimators for polychoric correlation under complex survey designs are obtained by minimizing Hellinger distance and negative exponential disparity, with survey weights entering through Horvitz-Thompson adjusted cell frequencies; penalized Ridge and Lasso variants of the Hellinger estimator regularize nuisance parameters, and the resulting procedures are consistent and asymptotically normal with a sandwich covariance that reflects the sampling design.

What carries the argument

Minimum divergence estimation applied to Horvitz-Thompson adjusted cell frequencies, with Hellinger distance and negative exponential disparity as the divergence measures.

If this is right

  • Penalized Hellinger estimators achieve the lowest mean squared error under concordant upper and lower contamination.
  • Negative exponential disparity estimators perform best under discordant mixed-corner contamination and compound misspecification.
  • The influence function of the Hellinger estimator remains finite, supporting robustness to the examined contamination patterns.
  • Practical selection guidelines can be derived from anticipated contamination type in survey practice.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The approach may extend to other ordinal association measures in weighted survey settings.
  • Further bounding of the influence function could strengthen robustness guarantees for very sparse cells.
  • Real-data applications in large-scale surveys could test whether the simulation advantages translate to reduced bias from interviewer effects or careless responses.

Load-bearing premise

Horvitz-Thompson adjusted cell frequencies are assumed to adequately incorporate the complex survey design.

What would settle it

An analysis in which maximum likelihood estimation produces lower mean squared error than the minimum divergence estimators under the examined contamination geometries would falsify the robustness claim.

Figures

Figures reproduced from arXiv: 2606.26351 by Anand N. Vidyashankar, David Kepplinger, Siqi Wei.

Figure 1
Figure 1. Figure 1: Cell probabilities πij under ρ = 0.5. Cells with red borders indicate contamination targets. that causes HD-based estimators to degrade heavily under this contamination. MNEDE and EE are less affected by this inverse-square-root sensitivity because their estimating equations use bounded residual adjustments rather than the HD’s square-root scaling, allowing them to remain robust and substantially outperfor… view at source ↗
Figure 2
Figure 2. Figure 2: MSE of ˆρ across contamination scenarios under PPS sampling (N = 5,000, n ≈ 500, 100 replications). The rows show standard vs. non-standard marginals, whereas the columns separate upper, lower (concordant), and mixed (discordant) corner contamination. 12 [PITH_FULL_IMAGE:figures/full_fig_p012_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Mean bias of ˆρ under standard marginals. Left: concordant (upper corner) contamina￾tion, where all estimators are biased upward but penalized MHDE variants (Lasso MHDE, Ridge MHDE) achieve the smallest bias. Right: discordant (mixed corner) contamination, where MNEDE maintains near-zero bias while HD-based methods and ML degrade severely. 4. Application: validating self-reported against measured BMI We il… view at source ↗
Figure 4
Figure 4. Figure 4: Self-reported vs. measured BMI, NHANES 2021–2023, ages 18–22. (a) design-weighted [PITH_FULL_IMAGE:figures/full_fig_p026_4.png] view at source ↗
read the original abstract

Standard maximum likelihood estimation of polychoric correlations is highly sensitive to contamination in survey data, including response errors, interviewer effects, and careless responding, yet assigns equal weight to all observations regardless of data quality. We develop robust estimators for polychoric correlation under complex survey designs based on two minimum divergence criteria -- Hellinger distance (HD) and negative exponential disparity (NED) -- incorporating survey weights through Horvitz--Thompson adjusted cell frequencies. For HD, we propose penalized Ridge and Lasso variants that regularize nuisance parameters while leaving the correlation unpenalized, and establish consistency and asymptotic normality with a sandwich covariance reflecting the sampling design. The influence function is finite but not uniformly bounded, reflecting Hellinger's sensitivity to sparse cells. Simulations under Poisson proportional-to-size sampling examine three contamination geometries -- concordant upper, concordant lower, and discordant mixed corner -- crossed with standard and non-standard latent marginals. The two estimator classes offer complementary advantages: penalized HD methods achieve the lowest mean squared error under concordant contamination, while NED performs best under discordant contamination and under compound misspecification--contamination effects. We provide practical guidelines for method selection based on anticipated contamination patterns in survey practice.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper develops robust estimators for polychoric correlation under complex survey designs based on Hellinger distance (HD) and negative exponential disparity (NED) minimum divergence criteria. Survey weights are incorporated via Horvitz-Thompson adjusted cell frequencies. For HD, penalized Ridge and Lasso variants are proposed that regularize nuisance parameters while leaving the correlation unpenalized. Consistency and asymptotic normality are established with a sandwich covariance reflecting the sampling design. Simulations under Poisson proportional-to-size sampling examine three contamination geometries (concordant upper, concordant lower, discordant mixed corner) crossed with standard and non-standard latent marginals, showing complementary advantages of the estimator classes.

Significance. If the results hold, the work addresses a practical gap in survey statistics by providing robust alternatives to MLE for polychoric correlations that account for complex designs and contamination. Strengths include the explicit incorporation of design weights, the penalization strategy that leaves the parameter of interest unpenalized, the derivation of design-consistent asymptotics, and the simulation design that directly probes the contamination geometries referenced in the abstract. The complementary performance of HD and NED variants under different scenarios offers actionable guidance for practitioners.

major comments (2)
  1. [influence function derivation] The finite but non-uniformly bounded influence function for the HD estimator is explicitly noted as reflecting sensitivity to sparse cells; however, it is unclear from the derivation whether this property still guarantees the claimed robustness under the discordant mixed-corner contamination geometry when cell frequencies become sparse due to the survey design (see the influence function section and the simulation results under non-standard marginals).
  2. [asymptotic normality result] The claim that Horvitz-Thompson adjusted cell frequencies adequately incorporate the complex survey design for the minimum divergence estimators relies on the finite influence function sufficing for robustness; a concrete check (e.g., via the sandwich covariance formula) would confirm that the design effect does not inflate the variance beyond what the simulations report under Poisson sampling.
minor comments (2)
  1. [simulation section] The abstract states that penalized HD methods achieve the lowest mean squared error under concordant contamination while NED performs best under discordant contamination, but the corresponding simulation tables or figures should explicitly report the MSE values and standard errors for each method and geometry to allow direct comparison.
  2. [methodology] Notation for the penalized estimators (Ridge and Lasso variants) should be introduced with explicit definitions of the penalty parameters and how they are selected, as this is central to the practical implementation.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments and the recommendation of minor revision. The feedback helps clarify the robustness properties under design-induced sparsity and the role of the sandwich covariance. We respond to each major comment below.

read point-by-point responses

  1. Referee: [influence function derivation] The finite but non-uniformly bounded influence function for the HD estimator is explicitly noted as reflecting sensitivity to sparse cells; however, it is unclear from the derivation whether this property still guarantees the claimed robustness under the discordant mixed-corner contamination geometry when cell frequencies become sparse due to the survey design (see the influence function section and the simulation results under non-standard marginals).

    Authors: The influence function derivation establishes finiteness for all positive cell probabilities, including those that become arbitrarily small under the survey design or non-standard marginals; this finiteness is the key condition used to prove consistency and asymptotic normality. The non-uniform bound arises precisely because the Hellinger distance downweights but does not completely eliminate the effect of very small cells, yet the resulting estimator remains consistent. The simulation design already crosses the discordant mixed-corner contamination with non-standard latent marginals, which deliberately produce sparse cells; in those cells the NED estimator (and to a lesser extent the penalized HD variants) achieve the lowest MSE, confirming that the finite influence function continues to deliver the claimed robustness under the exact conditions raised. revision: no

  2. Referee: [asymptotic normality result] The claim that Horvitz-Thompson adjusted cell frequencies adequately incorporate the complex survey design for the minimum divergence estimators relies on the finite influence function sufficing for robustness; a concrete check (e.g., via the sandwich covariance formula) would confirm that the design effect does not inflate the variance beyond what the simulations report under Poisson sampling.

    Authors: The asymptotic normality theorem is proved by embedding the Horvitz-Thompson adjusted cell frequencies into the minimum-divergence estimating equations and deriving the corresponding sandwich covariance that includes both the model-based Hessian and the design-based outer-product term. Because the simulations are generated under the same Poisson proportional-to-size sampling used in the asymptotics, the reported MSE values already embed the realized design effect. A direct numerical comparison between the sandwich formula evaluated at the estimated parameters and the Monte-Carlo variance of the estimators is not tabulated in the current manuscript; we can add such a table in revision to make the agreement explicit. revision: partial

Circularity Check

0 steps flagged

No significant circularity detected

full rationale

The derivation relies on established minimum-divergence criteria (Hellinger distance and negative exponential disparity) applied to Horvitz-Thompson adjusted cell frequencies for complex survey designs. Consistency, asymptotic normality, and sandwich covariance are derived from standard M-estimation theory under the stated sampling design, without any reduction of the target polychoric correlation estimator to a fitted parameter or self-referential definition. Penalized variants regularize only nuisance parameters while leaving the correlation unpenalized. No load-bearing self-citations, ansatz smuggling, or renaming of known results appear in the central argument. The influence function and simulation results are presented as direct consequences of the chosen divergence measures rather than tautological inputs.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Based on abstract only; no specific free parameters, axioms, or invented entities identifiable without full text.

pith-pipeline@v0.9.1-grok · 5748 in / 1024 out tokens · 36184 ms · 2026-06-26T01:05:26.880477+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

34 extracted references · 10 canonical work pages

  1. [1]

    Psychometrika , author=

    Identifiability of Polychoric Models with Latent Elliptical Distributions , volume=. Psychometrika , author=. 2025 , pages=. doi:10.1017/psy.2024.25 , number=

    work page doi:10.1017/psy.2024.25 2025

  2. [2]

    Philosophical Transactions of the Royal Society of London

    Mathematical Contributions to the Theory of Evolution. Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character , author =. 1900 , pages =. doi:10.1098/rsta.1900.0022 , number =

    work page doi:10.1098/rsta.1900.0022 1900

  3. [3]

    Psychometrika , author =

    Maximum Likelihood Estimation of the Polychoric Correlation Coefficient , volume =. Psychometrika , author =. 1979 , pages =. doi:10.1007/BF02296207 , number =

    work page doi:10.1007/bf02296207 1979

  4. [4]

    Applied Cognitive Psychology , author =

    Response Strategies for Coping With the Cognitive Demands of Attitude Measures in Surveys , volume =. Applied Cognitive Psychology , author =. 1991 , pages =. doi:10.1002/acp.2350050305 , number =

    work page doi:10.1002/acp.2350050305 1991

  5. [5]

    and Fowler, Floyd J

    Groves, Robert M. and Fowler, Floyd J. and Couper, Mick P. and Lepkowski, James M. and Singer, Eleanor and Tourangeau, Roger , year =. Survey Methodology , edition =

  6. [6]

    Journal of Business and Psychology , author =

    Detecting and Deterring Insufficient Effort Responding to Surveys , volume =. Journal of Business and Psychology , author =. 2012 , pages =. doi:10.1007/s10869-011-9231-8 , number =

    work page doi:10.1007/s10869-011-9231-8 2012

  7. [7]

    Survey Methodology , author =

    Outlier Robust. Survey Methodology , author =. 1995 , pages =

    1995

  8. [8]

    On confidence sequences

    Minimum. The Annals of Statistics , author =. 1977 , pages =. doi:10.1214/aos/1176343842 , number =

    work page doi:10.1214/aos/1176343842 1977

  9. [9]

    Hofmann, H

    Minimum. Journal of the American Statistical Association , author =. 1987 , pages =. doi:10.1080/01621459.1987.10478501 , number =

    work page doi:10.1080/01621459.1987.10478501 1987

  10. [10]

    and Chen, Zhiqiang and Gin

    Efficiency Versus Robustness:. The Annals of Statistics , author =. 1994 , pages =. doi:10.1214/aos/1176325512 , number =

    work page doi:10.1214/aos/1176325512 1994

  11. [11]

    Statistical Inference:

    Basu, Ayanendranath and Shioya, Hiroyuki and Park, Chanseok , year =. Statistical Inference:

  12. [12]

    , year =

    Kepplinger, David and Vidyashankar, Anand N. , year =. Minimum. doi:10.1007/s40065-026-00618-3 , journal =

    work page doi:10.1007/s40065-026-00618-3

  13. [13]

    Van der Vaart, A. W. , year =. Asymptotic Statistics , isbn =

  14. [14]

    and Ronchetti, Elvezio M

    Hampel, Frank R. and Ronchetti, Elvezio M. and Rousseeuw, Peter J. and Stahel, Werner A. , year =. Robust Statistics:

  15. [15]

    Psychometrika , author =

    Robust Estimation of Polychoric Correlation , volume =. Psychometrika , author =. 2026 , pages =. doi:10.1017/psy.2025.10066 , number =

    work page doi:10.1017/psy.2025.10066 2026

  16. [16]

    2025 , url =

    R: A Language and Environment for Statistical Computing , author =. 2025 , url =

    2025

  17. [17]

    Journal of Computational and Graphical Statistics , year =

    On Multivariate Normal Probabilities of Rectangles: Their Computation and Some Applications , author =. Journal of Computational and Graphical Statistics , year =

  18. [18]

    W. N. Venables and B. D. Ripley , publisher =. Modern Applied Statistics with. 2002 , address =

    2002

  19. [19]

    2016 , address =

    Hadley Wickham , publisher =. 2016 , address =

    2016

  20. [20]

    Psychometrika , volume =

    Robust Estimation of Polychoric Correlation , author =. Psychometrika , volume =. 2026 , doi =

    2026

  21. [21]

    Welz, Max and Alfons, Andreas and Mair, Patrick , year =

  22. [22]

    Lumley, Thomas , year =

  23. [23]

    and McClendon, McKee J

    Billiet, Jaak B. and McClendon, McKee J. , title =. Structural Equation Modeling: A Multidisciplinary Journal , volume =. 2000 , doi =

    2000

  24. [24]

    , title =

    Bollinger, Christopher R. , title =. Journal of Labor Economics , volume =. 1998 , doi =

    1998

  25. [25]

    Current Population Survey , year =

  26. [26]

    European Social Survey , year =

  27. [27]

    , title =

    Greenleaf, Eric A. , title =. Public Opinion Quarterly , volume =. 1992 , doi =

    1992

  28. [28]

    Kish, Leslie , title =

  29. [29]

    and Welch, Finis , title =

    Lillard, Lee and Smith, James P. and Welch, Finis , title =. Journal of Political Economy , volume =. 1986 , doi =

    1986

  30. [30]

    Public Opinion Quarterly , volume =

    Smith, David Horton , title =. Public Opinion Quarterly , volume =. 1967 , publisher =

    1967

  31. [31]

    , title =

    Van Vaerenbergh, Yves and Thomas, Troy D. , title =. International Journal of Public Opinion Research , volume =. 2013 , doi =

    2013

  32. [32]

    and Tremblay, M

    Connor Gorber, S. and Tremblay, M. and Moher, D. and Gorber, B. , title =. Obesity Reviews , year =

  33. [33]

    Larson, M. R. , title =. International Journal of Obesity and Related Metabolic Disorders , year =

  34. [34]

    King, B. M. and Cespedes, V. M. and Burden, G. K. and Brady, S. K. and Clement, L. R. and Abbott, E. M. and Baughman, K. S. and Joyner, S. E. and Clark, M. M. and Pury, C. L. S. , title =. Obesity Science & Practice , year =