REVIEW 3 major objections 5 minor 1 cited by
Profile least squares estimation in networks with covariates
T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read This paper proposes an iterative profile least squares algorithm that estimates covariate effects and latent network structure simultaneously by alternating spectral embedding of the covariate-adjusted adjacency matrix with linear…
desk verdict Useful, clearly written extension of GRDPG-with-covariates to continuous and mixed covariates, but the alternating estimator has no convergence or consistency theory and the simulations never test the covariate-latent confounding regime that the real applications likely live in. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the indefinite inner product kernel $f(x,y;q,s)=\sum_{l=1}^{q} x_l y_l - \sum_{l=q+1}^{q+s} x_l y_l$, which lets the residual part of the model capture both homophily (first $q$ dimensions) and heterophily (last $s$ dimensions). The argument is carried by an alternating scheme: for fixed $\gamma$, spectral embedding of $Y(\gamma)=A-X\gamma$ through the eigendecomposition retaining the $d$ largest eigenvalues in signed magnitude estimates the residual kernel, with the latent dimension chosen by a profile-likelihood scree rule; for that kernel estimate, the update $\hat\gamma = (\tilde X^T\tilde X)^{-1}\tilde X^T \mathrm{vec}(A - \Lambda I_{qs}\Lambda^T)$ is the usual least squares solution. A Bayesian bootstrap that assigns random weights to the estimating equations supplies confidence intervals for $\gamma$ and for the residual intensities.
What would settle it
Simulate networks under the paper's model with $d=2$, $q=s=1$, and with edge covariates generated as functions of the latent positions so that covariate and latent structure are correlated; if Algorithm 1's estimates of $\gamma$ show bias that persists as $n$ grows, or if nominal 95% bootstrap intervals undercover substantially, the assumed separation of covariate and latent effects is not achieved.
Extended reading notes
Core claim
The central claim is that the covariate coefficient vector and the latent node positions can be estimated jointly by profile least squares even though the projection filter used to embed the residual matrix is nonlinear, so no closed-form estimator exists. For a fixed coefficient vector $\gamma$, the matrix $Y(\gamma)=A-X\gamma$ is treated as a generalized random dot product graph adjacency matrix: its top signed eigenvalues and eigenvectors give an estimate of the residual kernel $\Lambda I_{qs}\Lambda^T$. For this estimated kernel, the coefficient vector updates by ordinary least squares regression of $A$ minus the estimated kernel on the edge covariates. The paper argues that alternating these two steps to convergence yields estimates of $\gamma$ and of the residual structure, with simulations demonstrating the behavior of the estimates as $n$ grows and with the bootstrap providing uncertainty statements.
Load-bearing premise
The load-bearing premise is that at each iteration the covariate-adjusted residual matrix $A-X\gamma$ is close to a low-rank symmetric matrix whose top eigenvectors consistently estimate the true latent positions, even while $\gamma$ is being estimated and even if the covariates are correlated with the latent structure.
Editorial extensions
If this is right
- Users of the method can decompose the estimated edge probability matrix into a covariate effect matrix $x_{ij}^T\hat\gamma$ and a residual kernel estimate, so the visual contribution of observed attributes versus unobserved factors is directly inspectable.
- The method applies to continuous, categorical, and mixed covariates, covering cases where earlier spectral community-detection approaches with vertex covariates only handle discrete attributes.
- Bayesian bootstrap weights on the estimating equations yield confidence intervals for $\gamma$ and for residual intensities, enabling significance statements such as geographic distance in the tree network, city of practice in the physician network, and trade and joint democracy in the alliance networks.
- Simulations under both block-model residuals and general low-rank residuals show that mean squared error of $\hat\gamma$ decreases and adjusted Rand index increases with network size, and the displayed bootstrap intervals contain the true parameters.
Reading between the lines
- Editorial extension: the same profile least squares scheme could be run on directed or weighted networks by replacing the symmetric GRDPG embedding with a directed or weighted spectral embedding; the estimating equations for $\gamma$ would remain least squares.
- Editorial extension: a stress test the paper does not run is to generate covariates as functions of the latent positions, so that covariate and latent structure are correlated, and to track bias in $\hat\gamma$; the alternating procedure's identifiability in that regime is the main open question.
- Editorial extension: because the type II residual matrix is not a valid probability matrix, the method effectively functions as a matrix-decomposition diagnostic; applying it to networks with known ground-truth covariate effects could benchmark how much of the latent structure the covariate term absorbs.
- Editorial extension: a formal consistency proof for the alternating estimator would be the natural next step; the simulation evidence suggests the separation works in the regimes tested, but no theorem yet guarantees it.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies a single undirected binary network model in which the edge probability matrix is P_ij = x_ij^T gamma + alpha_i^T I_qs alpha_j, a linear edge-covariate term plus an indefinite-inner-product latent-position term. It proposes an iterative profile least squares estimator in which the residual matrix A - X gamma is spectrally embedded and gamma is updated by OLS on the resulting residuals (Algorithm 1), and it develops bootstrap inference based on the generalized bootstrap of Chatterjee and Bose (2005) with Bayesian bootstrap weights and GMM clustering of the estimated latent positions. The method is evaluated on simulations with SBM-type and general low-rank residual structures and applied to tree, physician-friendship, and military-alliance networks.
Significance. If the algorithm and bootstrap procedure were theoretically supported, the paper would offer a useful practical decomposition of network structure into observed covariate effects and latent heterogeneity, with a computationally simple estimation recipe and bootstrap-based uncertainty quantification. The paper's genuine strengths are its explicit low-rank indefinite-kernel formulation, the breadth of simulation settings (categorical and continuous covariates; stochastic-blockmodel and indefinite low-rank residuals), the comparison with Mu et al. (2022) in the categorical case, and the four real-data illustrations. However, the central methodological claims are currently supported only by simulation and application: no identifiability, convergence, or consistency results are proved for the alternating procedure, and the simulation design does not cover covariate-latent dependence, which is precisely the regime in which the separation between X gamma and Lambda I_qs Lambda^T is most questionable.
major comments (3)
- [Section 3.1, Eqs. (3.3)-(3.6) and Algorithm 1] The paper provides no identifiability condition separating the linear term X gamma from the latent term Lambda I_qs Lambda^T, and no convergence or consistency result for the alternating profile least squares procedure. This is a load-bearing gap because the spectral embedding step in Step 3 of Algorithm 1 is only justified when the matrix being embedded is the true low-rank residual plus noise; if covariates are correlated with latent positions, A - X hat_gamma can carry a distorted low-rank signal and the OLS update in Eq. (3.6) can absorb part of Lambda into hat_gamma. The simulation study in Section 5 generates covariates independently of the latent positions z_i and alpha_i, so it never exercises this confounding regime. I ask the authors either to state explicit identifiability and compatibility conditions and provide (or cite) theory for the alternating estimator under those conditions, or to weaken the paper's claims to a heuristic method and add a simulation setting in which covariates depend on the latent cluster labels (e.g., x_i drawn from a cluster-dependent distribution) to assess robustness.
- [Section 4, Eqs. (4.1)-(4.2), and Figs. 4/8] The bootstrap inference is built on Chatterjee and Bose (2005), but no condition of that theory is verified for this nonconvex spectral-profile estimator, and the resampling step for Lambda is not shown to solve the corresponding resampled estimating equations. The de-weighting step alpha*_ib = alpha^w_ib / sqrt(W_bi) is motivated by a heuristic reading of Eq. (4.2) rather than derived from the weighted estimating equations. Moreover, 'correct coverage' is claimed on the basis of Figs. 4 and 8, which show confidence intervals from a single randomly generated network per setting; no coverage proportions over the 250 replications are reported. Please provide either the distributional conditions and coverage theory, or clearly label the bootstrap as heuristic and report empirical coverage rates over replications.
- [Section 5.1, Fig. 3 and following text] The reported ARI improvement is obtained by manually re-running the MClust step with K=2 for 11 of 250 replications after the default pipeline selected K=3. This is a post-hoc modification of the clustering procedure after seeing the results, and it inflates the apparent cluster recovery of the default method. Please report both the default and the overridden ARI values, and specify in advance a reproducible rule for overriding the model-selection step (for example, a threshold on the ratio of MClust uncertainties, or a clearly stated visual-check criterion). Without such a rule the cluster-recovery results are not reproducible.
minor comments (5)
- [Algorithm 1, Step 3] The notation |Y_m| = (Y_m^T Y_m)^{1/2} combined with 'arranged in decreasing order (based on their actual, signed, value)' is ambiguous; clarify that the eigendecomposition of Y_m is used and that the d eigenvalues largest in absolute value are retained, with q_m and s_m equal to the numbers of positive and negative retained eigenvalues.
- [Section 5.2, settings (b) and (c)] The text refers to a '3 x 3 block-form' and to blocks (1,3), (2,3), and (3,3) even though only K=2 residual blocks were defined; please clarify whether three blocks are intended or correct the block indexing, as this affects the reproducibility of the simulation design.
- [Section 4, bootstrap weights] The statement that Exponential(alpha) weights with alpha=1 and alpha=O(n^{-1/2}) have performances comparable to the naive and m-out-of-n bootstraps is not accompanied by a reference or simulation evidence; please cite a source or qualify the statement.
- [Figs. 16 and 18 captions] The captions state l=1,...,13 for the alliance data, but this dataset has p=6 covariates; update the captions to the correct index range.
- [Section 4, parameterization of Exponential weights] The choice W_bi ~ Exponential(alpha) is ambiguous about the rate/scale parameterization; state the density or mean explicitly so that the reported alpha = n^{-1/2} is reproducible.
Circularity Check
No significant circularity: the estimator is a genuine alternating fixed-point procedure and the bootstrap is checked on fresh simulations; the only self-citation is an external bootstrap theorem.
full rationale
I examined the derivation chain from model (2.2), through the profile least squares objective (3.1)-(3.5), Algorithm 1, the bootstrap section, and the simulation and data sections. The gamma update in Eq. (3.6) is a genuine OLS regression of the residualized adjacency on covariates, and the spectral step embeds Y(gamma) = A - X gamma; neither step is defined in terms of the quantity it is used to validate. The paper explicitly acknowledges that the spectral filter F is nonlinear and that no closed-form PLS estimator exists, and then proposes an iterative algorithm; this is an honest algorithmic construction rather than a renamed input. Simulation results in Section 5 generate fresh networks from the model and compare estimates to known true parameters, so coverage and MSE claims are not forced by construction. The only self-citation is the generalized bootstrap of Chatterjee and Bose (2005), one of whose authors is a co-author of the present paper; it is used as an external published theorem for resampling estimating equations, not as an assertion that the present model's conditions are verified. The main weaknesses, namely absent identifiability and consistency theorems for the alternating estimator and simulations that draw covariates independently of latent structure, are correctness and evidence gaps rather than circular reductions. Accordingly, no circular step is identified.
Assumptions & free parameters
free parameters (5)
- Latent dimension d (and q, s) =
estimated via Zhu and Ghodsi (2006)
- Number of residual clusters K =
selected via BIC in Mclust
- Number of iterations M =
500
- Initialization constant c and grid size T =
c in [0.15, 2], T=20
- Bootstrap weight exponent alpha =
alpha = n^{-1/2}
assumptions (4)
- standard math The GRDPG adjacency spectral embedding consistently estimates latent positions up to orthogonal transformation (Rubin-Delanchy et al., 2022).
- domain assumption The edge probability matrix admits the exact additive decomposition P = X gamma + Lambda I_{qs} Lambda^T with identity link and low-rank residual.
- domain assumption The latent positions and covariates are additively separable and identifiable.
- standard math The generalized bootstrap of Chatterjee and Bose (2005) applies to the estimating equations derived from profile least squares.
Cite this review
Pith. "Pith review of Profile least squares estimation in networks with covariates." pith.science (2026). https://pith.science/paper/IRU6FBGG
@misc{pith2026241216298,
author = {Pith},
title = {Pith review of: Profile least squares estimation in networks with covariates},
year = {2026},
howpublished = {\url{https://pith.science/paper/IRU6FBGG}},
note = {Machine review of arXiv:2412.16298}
}
read the original abstract
Many real world networks exhibit edge heterogeneity with different pairs of nodes interacting with different intensities. Further, nodes with similar attributes tend to interact more with each other. Thus, in the presence of observed node attributes (covariates), it is of interest to understand the extent to which these covariates explain interactions between pairs of nodes and to suitably estimate the remaining structure due to unobserved factors. For example, in the study of international relations, the extent to which country-pair specific attributes such as the number of material/verbal conflicts and volume of trade explain military alliances between different countries can lead to valuable insights. We study the model where pairwise edge probabilities are given by the sum of a linear edge covariate term and a residual term to model the remaining heterogeneity from unobserved factors. We approach estimation of the model via profile least squares and show how it leads to a simple algorithm to estimate the linear covariate term and the residual structure that is truly latent in the presence of observed covariates. Our framework lends itself naturally to a bootstrap procedure which is used to draw inference on model parameters, such as to determine significance of the homophily parameter or covariates in explaining the underlying network structure. Application to four real network datasets and comparisons using simulated data illustrate the usefulness of our approach.
Figures
Figures from the paper (15 more)
Forward citations
Cited by 1 Pith paper
-
Testing for correlation between network structure and high-dimensional node covariates
Four permutation tests (ridge, group LASSO, and two CCA variants) detect association between node covariates and random-dot-product-graph latent structure, with consistency theorems and cheaper computation than prior ...
Reference graph
Works this paper leans on
-
[1]
Athreya, A., Fishkind, D. E., Tang, M., Priebe, C. E., Park, Y., Vogelstein, J. T., Levin, K., Lyzinski, V., and Qin, Y. (2017). Statistical inference on random dot product graphs: a survey. The Journal of Machine Learning Research , 18(1):8393--8484
work page 2017
-
[2]
Binkiewicz, N., Vogelstein, J., and Rohe, K. (2017). Covariate-assisted spectral clustering. Biometrika , 104(2):361--377
work page 2017
-
[3]
Burt, R. S. (1987). Social contagion and innovation: Cohesion versus structural equivalence. American journal of Sociology , 92(6):1287--1335
work page 1987
-
[4]
Chandna, S., Olhede, S., and Wolfe, P. (2021). Local linear graphon estimation using covariates. Biometrika
work page 2021
-
[5]
Chatterjee, S. and Bose, A. (2005). Generalized bootstrap for estimating equations. Annals of Statistics , pages 414--436
work page 2005
-
[6]
Chatterjee, S., Diaconis, P., Sly, A., et al. (2011). Random graphs with a given degree sequence. The Annals of Applied Probability , 21(4):1400--1435
work page 2011
-
[7]
Choi, D. S., Wolfe, P. J., and Airoldi, E. M. (2012). Stochastic blockmodels with a growing number of classes. Biometrika , 99(2):273--284
work page 2012
-
[8]
Coleman, J., Katz, E., and Menzel, H. (1957). The diffusion of an innovation among physicians. Sociometry , 20(4):253--270
work page 1957
Show all 42 references
-
[9]
J., Desmarais, B
Cranmer, S. J., Desmarais, B. A., and Menninga, E. J. (2012). Complex dependencies in the alliance network. Conflict Management and Peace Science , 29(3):279--313
2012
-
[10]
Davison, A. C. and Hinkley, D. V. (1997). Bootstrap methods and their application . Number 1. Cambridge university press
1997
-
[11]
De Nicola , G., Sischka, B., and Kauermann, G. (2022). Mixture models and networks: The stochastic blockmodel. Statistical Modelling , 22(1-2):67--94
2022
-
[12]
Dzemski, A. (2019). An empirical model of dyadic link formation in a network with unobserved heterogeneity. Review of Economics and Statistics , 101(5):763--776
2019
-
[13]
and Huang, T
Fan, J. and Huang, T. (2005). Profile likelihood inferences on semiparametric varying-coefficient partially linear models. Bernoulli , 11(6):1031--1057
2005
-
[14]
Fordham, B. O. (2010). Trade and asymmetric alliances. Journal of Peace Research , 47(6):685--696
2010
-
[15]
Graham, B. S. (2017). An econometric model of network formation with degree heterogeneity. Econometrica , 85(4):1033--1063
2017
-
[16]
Gr \"u n, B. (2019). Model-based clustering. In Handbook of mixture analysis , pages 157--192. Chapman and Hall/CRC
2019
-
[17]
H \"a rdle, W., Liang, H., and Gao, J. (2000). Partially linear models . Springer Science & Business Media
2000
-
[18]
D., Raftery, A
Hoff, P. D., Raftery, A. E., and Handcock, M. S. (2002). Latent space approaches to social network analysis. Journal of the American Statistical Association , 97(460):1090--1098
2002
-
[19]
Huang, S., Sun, J., and Feng, Y. (2018). Pairwise covariates-adjusted block model for community detection. arXiv preprint arXiv:1807.03469
2018 arXiv
-
[20]
N., Handcock, M
Krivitsky, P. N., Handcock, M. S., Raftery, A. E., and Hoff, P. D. (2009). Representing degree distributions, clustering, and homophily in social networks with latent cluster random effects models. Social networks , 31(3):204--213
2009
-
[21]
and Reiter, D
Lai, B. and Reiter, D. (2000). Democracy, political similarity, and international alliances, 1816-1992. Journal of Conflict Resolution , 44(2):203--227
2000
-
[22]
Latouche, P., Robin, S., and Ouadah, S. (2018). Goodness of fit of logistic regression models for random graphs. Journal of Computational and Graphical Statistics , 27(1):98--109
2018
-
[23]
Leeds, B., Ritter, J., Mitchell, S., and Long, A. (2002). Alliance treaty obligations and provisions, 1815-1944. International Interactions , 28(3):237--260
2002
-
[24]
Leeds, B. A. (1999). Domestic political institutions, credible commitments, and international cooperation. American Journal of Political Science , 43(4):979--1002
1999
-
[25]
Leeds, B. A. (2018). Alliance treaty obligations and provisions (atop) codebook version 4.0
2018
-
[26]
and Joyce, K
Maoz, Z. and Joyce, K. A. (2016). The effects of shocks on international networks: Changes in the attributes of states and the structure of international alliance networks. Journal of Peace Research , 53(3):292--309
2016
-
[27]
Mariadassou, M., Robin, S., and Vacher, C. (2010). Uncovering latent structure in valued graphs: a variational approach
2010
-
[28]
P., Damiani, G., and Riccardi, W
Mascia, D., Cicchetti, A., Fantini, M. P., Damiani, G., and Riccardi, W. (2011). Physicians' propensity to collaborate and their attitude towards ebm: A cross-sectional study. BMC Health Services Research , 11(172):1--8
2011
-
[29]
P., and Cicchetti, A
Mascia, D., Di Vincenzo , F., Iacopino, V., Fantini, M. P., and Cicchetti, A. (2015). Unfolding similarity in interphysician networks: the impact of institutional and professional homophily. BMC Health Services Research , 15(92):1--8
2015
-
[30]
Mele, A., Hao, L., Cape, J., and Priebe, C. E. (2023). Spectral estimation of large stochastic blockmodels with discrete nodal covariates. Journal of Business & Economic Statistics , 41(4):1364--1376
2023
-
[31]
Mu, C., Mele, A., Hao, L., Cape, J., Athreya, A., and Priebe, C. E. (2022). On spectral algorithms for community detection in stochastic blockmodel graphs with vertex covariates. IEEE Transactions on Network Science and Engineering , 9(5):3373--3384
2022
-
[32]
Park, J. H. and Sohn, Y. (2020). Detecting Structural Changes in Longitudinal Network Data . Bayesian Analysis , 15(1):133 -- 157
2020
-
[33]
Roy, S., Atchad \'e , Y., and Michailidis, G. (2019). Likelihood inference for large scale stochastic blockmodels with covariates based on a divide-and-conquer parallelizable algorithm with communication. Journal of Computational and Graphical Statistics , 28(3):609--619
2019
-
[34]
Rubin-Delanchy, P., Cape, J., Tang, M., and Priebe, C. E. (2022). A statistical interpretation of spectral embedding: The generalised random dot product graph. Journal of the Royal Statistical Society Series B: Statistical Methodology , 84(4):1446--1473
2022
-
[35]
Speckman, P. (1988). Kernel smoothing in partial linear models. Journal of the Royal Statistical Society Series B: Statistical Methodology , 50(3):413--436
1988
-
[36]
K., and Lee, T
Su, Y., Wong, R. K., and Lee, T. C. (2020). Network estimation via graphon with node features. IEEE Transactions on Network Science and Engineering
2020
-
[37]
Sweet, T. M. (2015). Incorporating covariates into stochastic blockmodels. Journal of Educational and Behavioral Statistics , 40(6):635--664
2015
-
[38]
Vacher, C., Piou, D., and Desprez-Loustau, M.-L. (2008). Architecture of an antagonistic tree/fungus network: the asymmetric influence of past evolutionary history. PloS one , 3(3):e1740
2008
-
[39]
Warren, T. C. (2010). The geometry of security: Modeling interstate alliances as evolving networks. Journal of Peace Research , 47(6):697--709
2010
-
[40]
E., and Leng, C
Yan, T., Jiang, B., Fienberg, S. E., and Leng, C. (2019). Statistical inference in a directed network model with covariates. Journal of the American statistical association , 114(526):857--868
2019
-
[41]
Young, S. J. and Scheinerman, E. R. (2007). Random dot product graph models for social networks. In International Workshop on Algorithms and Models for the Web-Graph , pages 138--149. Springer
2007
-
[42]
and Ghodsi, A
Zhu, M. and Ghodsi, A. (2006). Automatic dimensionality selection from the scree plot via the use of profile likelihood. Computational Statistics & Data Analysis , 51(2):918--930
2006
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.