REVIEW 2 major objections 5 minor 40 references
Generalized Reduced-Rank Regression with Homogeneity Pursuit
T0 review · 2 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read This paper proves that when columns of a multi-response regression coefficient matrix form tight groups, the estimation error bound shrinks from the sparse reduced-rank rate toward the much faster univariate-response rate, with the…
desk verdict A genuinely useful unification of column homogeneity, low rank, and sparsity in GLMs, with simulations that back it up; the main rate theorem's stated conditions are unsound for the very GLMs it targets, so the theory needs a boundedness or local-RSC fix before publication. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The mechanism is a Laplacian fusion penalty $P_{km}(B)=\min_{\mathcal G,\mu}\sum_{k=1}^K\sum_{j\in\mathcal G_k}\|\beta_j-\mu_k\|_2^2$ that encourages the columns of the coefficient matrix to cluster, combined with the factorization $B=UV^\top$ with $V^\top V=I_r$ that enforces low rank and converts row sparsity on $B$ into row sparsity on $U$. Proposition 2 supplies the key equivalence $\operatorname{trace}\{B L B^\top\}=2\sum_{k=1}^K\sum_{j\in\mathcal G_k}\|\beta_j-\mu_k\|_2^2$, which turns the penalty into a K-means update on the rows of $V$. The algorithm iterates proximal gradient steps with hard thresholding on $U$, orthogonal Procrustes updates on $V$, and K-means updates on the group memberships, and Theorem 2 shows each blockwise update reduces the surrogate objective.
What would settle it
Take a Bernoulli logistic regression with a two-column coefficient matrix in the feasible set, and compute the smallest eigenvalue of the Hessian of the negative log-likelihood along a sequence of matrices whose norms grow to infinity while staying in the feasible set. If that eigenvalue tends to zero, Condition 1 cannot hold with a fixed $\kappa>0$, and the claimed $O_P(\xi_{m,K}(\gamma)/\sqrt n)$ rate for that setting lacks a valid premise.
Extended reading notes
Core claim
The paper's central claim is Theorem 1: under restricted strong convexity of the link function, a bounded sparse-operator norm of the design, and sub-Gaussian noise, the constrained estimator defined by problem (3.1) satisfies $\|\hat A - A^*\|_F + \|\hat B - B^*\|_F = O_P(\rho \xi_{m,K}(\gamma)/(\kappa \sqrt n))$, where $\xi_{m,K}(\gamma)$ combines three terms: $\sqrt{r\gamma(s+m)}$, $\sqrt{s\log p + m\log K}$, and $\sqrt{[s+\min(K^2,m)]\min(r,K)\log m}$. The rate explicitly depends on the within-group similarity $\gamma$ and the number of groups $K$, so the paper can state precisely when homogeneity improves accuracy: a small $\gamma$ (relative to the noise floor $\zeta_n^2$) and a small $K$ shrink the error, while a large $\gamma$ or indistinct groups makes the bound collapse to the known sparse reduced-rank regression rate.
Load-bearing premise
The whole rate proof rests on the assumption that the log-likelihood stays strongly convex with a fixed positive curvature constant across the entire (unbounded) feasible set of coefficient matrices, but for logistic and Poisson models the curvature shrinks toward zero as the natural parameters grow without bound, so this uniformity can fail for exactly the models the paper targets.
Editorial extensions
If this is right
- When all columns are identical ($\gamma=0$, $K=1$), the bound becomes $O_P(\sqrt{s\log(p\vee m)}/\sqrt n)$, matching the high-dimensional univariate-response regression rate.
- When the group structure is absent or weak, the error bound reduces to the sparse reduced-rank regression rate, so the method does not lose accuracy compared with SRRR.
- Both the within-group similarity $\gamma$ and the group number $K$ enter the rate explicitly, giving a quantitative way to decide when column homogeneity will help.
- The blockwise algorithm decreases the objective at every update and converges to a critical point, so the estimator is computationally reproducible.
- Tuning $K$, $\lambda$, rank, and sparsity by cross-validation lets practitioners compare homogeneity, low-rank, and sparsity effects on a common benchmark.
Reading between the lines
- A practical rule implied by the theorem: pursue homogeneity only when the estimated within-group column spread is below the noise floor $\zeta_n^2$; otherwise the SRRR rate already applies, and the extra penalty buys little.
- Because the fusion penalty is equivalent to K-means on the rows of $V$, the same framework could be extended to two-way homogeneity, clustering rows and columns simultaneously, and to other losses such as quantile regression.
- The theorem suggests that a data-driven estimator of $\gamma$ and $K$ from the fitted $V$ could replace cross-validation on a two-dimensional grid, making the method more scalable in applications with many responses.
- The BCI application indicates the clustering output has ecological interpretability, which would be strengthened by quantifying uncertainty in the estimated groups, a topic the paper leaves open.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a unified regularized maximum likelihood framework for multi-response generalized linear models under three structural assumptions: column homogeneity, low rank, and row sparsity. The coefficient matrix B is factorized as UV^T, homogeneity is encouraged through a normalized Laplacian fusion penalty on the columns of B, sparsity is imposed by a hard row constraint on U, and the rank is fixed through V^T V = I_r. The main theoretical result, Theorem 1, gives a consistency rate for the constrained estimator (3.1) under Conditions 1-3, with the rate depending explicitly on the within-group similarity parameter γ, the number of groups K, the rank r, the sparsity level s, and the dimensions m and p. An iterative block-coordinate majorization-minimization algorithm is proposed and shown to be monotone in Theorem 2. Numerical experiments compare the proposed Laplacian SRRR method with SRRR and gLasso in linear and multinomial logistic settings, and the method is applied to tree species data from Barro Colorado Island.
Significance. If the main theorem can be made sound, this is a valuable contribution: it offers a single convergence rate that interpolates between sparse reduced-rank regression (large K, large γ) and a univariate-like regression rate (K = 1, γ = 0), and it extends existing reduced-rank methodology beyond the linear model to a broad GLM family. The rate's explicit dependence on γ and K is a genuinely falsifiable comparative prediction, and the numerical results in Tables 2-5 are consistent with the qualitative claim that stronger homogeneity improves estimation. The paper does not fit constants in the proof, and the algorithmic convergence guarantee in Theorem 2 is useful. The main weakness is that Condition 1 is stated too strongly for the very GLMs that the paper targets, so the central theoretical claim currently rests on an unsatisfiable assumption.
major comments (2)
- [Section 3.1, Condition 1] The restricted strong convexity condition is imposed uniformly over the entire solution set ~Brsγ, which is unbounded. A is completely unconstrained, and B may be scaled by an arbitrary c > 0 along any direction that is exactly homogeneous (then Pkm(B) = 0, rank(B) = 1, and ∥B∥_{2,0} ≤ s remain true for every c), so pairs in ~Brsγ can be arbitrarily far apart. For the Bernoulli log-partition b(θ) = log(1 + e^θ), b''(θ) = e^θ/(1 + e^θ)^2 → 0 as θ → ∞; for the Poisson log-partition b(θ) = e^θ, b''(θ) → 0 as θ → −∞; and for the multinomial logistic log-partition b(θ) = log(1 + ∑_j e^{θ_j}), the directional second derivative along θ = (t, 0, ..., 0) tends to 0 as t → ∞. Since Z contains an intercept and X takes values on R^p, these unbounded directions belong to ~Brsγ. Hence no fixed κ > 0 can satisfy the displayed inequality, and Theorem 1 is vacuous as stated for the Bernoulli and multinomial models emphasized in Section 2 and for the Poisson row of Table 1. The proof should either work with a bounded parameter set (e.g., ∥~B∥_F ≤ R, with the final rate then depending on R) or impose a local restricted strong convexity in a neighborhood of ~B* together with an initialization or restricted-step guarantee that keeps the iterates in that neighborhood.
- [Section 3.1, Condition 3 and Table 1] Condition 3 assumes that E = Y − E(Y) has sub-Gaussian tails, with P(|a^T e_i| > δ) ≤ 2 exp(−δ²/(τ∥a∥²)). This holds for bounded responses such as Bernoulli and multinomial indicators, but not for the Poisson responses listed in Table 1: a Poisson variable has P(|Y − λ| > t) decaying like exp(−ct log t), not exp(−ct²). Since the paper's GLM framework explicitly includes Poisson in Table 1, the theorem's scope is narrower than claimed; it covers the linear model and bounded-response GLMs but not Poisson. Please either remove the Poisson claim from the stated scope or replace Condition 3 with a tail or moment condition that Poisson satisfies, and adjust the theorem accordingly.
minor comments (5)
- [Section 2.1 and Section 4.2] There are several typos and grammatical errors, including 'speficially' in Section 2.1, 'procruste' in the text after equation (2.9), 'seqe' in Section 4.2, and 'Whose the j-th row' in Section 2.2.2. The paper should be carefully proofread.
- [Tables 2-5] In the (b) panels of Tables 2-5, the rows for gLasso, SRRR, and Laplacian are not labeled, so the reader must reconstruct the methods from the order in panel (a). Please repeat the method names in every panel.
- [Section 3.1, Condition 1] The notation b(~B) in Condition 1 is not defined. In Section 2, b(θ_i) is the per-observation log-partition, whereas Condition 1 evaluates b at the whole parameter matrix and includes a factor nκ on the right-hand side. Please clarify whether b(~B) denotes the summed log-partition ∑_{i=1}^n b(θ_i) and define κ accordingly.
- [Section 3, discussion after Theorem 1] The statement that for large γ and K the rate reduces to O(√(s log p) + √(r(s+m) log m)), 'the same as sparse reduced-rank regression excluding a logarithmic factor', is too strong. In Theorem 1(b), when K is close to m the bound still contains √(m log m) from √(s log p + m log K) and the third entropy term, so the comparison with the SRRR rate should be qualified.
- [Section 4.2] For the Laplacian SRRR method, the rank and sparsity hyperparameters are taken from the cross-validation results of SRRR rather than being re-tuned jointly with K and λ. Please justify that this does not bias the comparison, since the proposed method is otherwise tuned on a larger grid.
Circularity Check
No circular derivation: Theorem 1 is an asymptotic bound under external assumptions; self-citations are background and non-load-bearing.
full rationale
The paper's derivation chain for Theorem 1 is self-contained in the relevant sense: the constrained estimator in (3.1) is defined without using the theorem's conclusion, and the bound depends on structural parameters (s, r, K, gamma) that enter as constraints on the true parameter or as inputs, not as fitted constants. The gamma term arises because Pkm(B*) <= gamma bounds the approximation error between B* and its K-column grouped version; that is a standard bias bound, not a renaming of the target. Conditions 1-3 are external assumptions (restricted strong convexity, bounded restricted operator norm, sub-Gaussian noise) and are not derived from the rate; no parameter is fitted and then called a prediction. Self-citations (Hessellund et al. 2022b for the BCI model, Chen et al. 2022 and Wen et al. 2023 for background techniques) are not load-bearing: the consistency proof and the main rate do not invoke them as the source of the result. The principal weakness is that Condition 1's uniform restricted strong convexity over the unbounded set ~Brsgamma is likely unsatisfiable for Bernoulli, multinomial-logit, and Poisson links without a bounded-parameter or local-RSC modification; however, that is a correctness and assumption-validity problem, not a circularity problem. No step satisfies the defined circularity patterns.
Assumptions & free parameters
free parameters (5)
- s (sparsity level) =
tuned by cross-validation from {10,...,20}
- r (rank) =
tuned by cross-validation from {2,...,10}
- K (number of groups) =
tuned by cross-validation from {2,...,10}
- lambda (fusion penalty) =
tuned by CV grid exp(seq(-4.605,0,length=50))
- eta (step size) =
not specified; order O(n*kappa/(||X||^2+lambda))
assumptions (4)
- domain assumption Condition 1: restricted strong convexity of the link function b over the whole set ~Brsgamma with uniform kappa > 0
- domain assumption Condition 2: restricted maximum eigenvalue bound n^{-1/2} ||~X||_{2s,op} <= rho
- domain assumption Condition 3: sub-Gaussian noise with tail bound P(|a^T e_i| > delta) <= 2 exp(-delta^2 / (tau ||a||^2))
- domain assumption The true coefficient matrix B* admits a grouping into K* groups, and the estimator uses the same K (or K is tuned); Theorem 1 assumes the group structure is compatible with P_km(B) <= gamma
Cite this review
Pith. "Pith review of Generalized Reduced-Rank Regression with Homogeneity Pursuit." pith.science (2026). https://pith.science/paper/A6BOP64Z
@misc{pith2026250115016,
author = {Pith},
title = {Pith review of: Generalized Reduced-Rank Regression with Homogeneity Pursuit},
year = {2026},
howpublished = {\url{https://pith.science/paper/A6BOP64Z}},
note = {Machine review of arXiv:2501.15016}
}
read the original abstract
Homogeneity, low rank, and sparsity are three widely adopted assumptions in multi-response regression models to address the curse of dimensionality and improve estimation accuracy. However, there is limited literature that examines these assumptions within a unified framework. In this paper, we investigate the homogeneity, low rank, and sparsity assumptions under the generalized linear model with high-dimensional responses and covariates, encompassing a wide range of practical applications. Our work establishes a comprehensive benchmark for comparing the effects of these three assumptions and introduces a regularized maximum likelihood estimation method to fit the corresponding models. Under mild conditions,we prove the statistical consistency of our estimator. Theoretical results provide insights into the role of homogeneity and offer a quantitative analysis of scenarios where homogeneity improves estimation accuracy. The proposed method's effectiveness is demonstrated through numerical simulations and an empirical analysis of tree species data from Barro Colorado Island.
Figures
Reference graph
Works this paper leans on
-
[1]
Florentina Bunea, Yiyuan She, and Marten H. Wegkamp. Optimal selection of reduced rank estimators of high-dimensional matrices . The Annals of Statistics, 39 0 (2): 0 1282--1309, 2011. doi:10.1214/11-AOS876. URL https://doi.org/10.1214/11-AOS876
-
[2]
Florentina Bunea, Yiyuan She, and Marten H. Wegkamp. Joint variable and rank selection for parsimonious estimation of high-dimensional matrices . The Annals of Statistics, 40 0 (5): 0 2359 -- 2388, 2012
work page 2012
-
[3]
Reduced rank regression via adaptive nuclear norm penalization
Kun Chen, Hongbo Dong, and Kung-Sik Chan. Reduced rank regression via adaptive nuclear norm penalization. Biometrika, 100 0 (4): 0 901--920, 2013
2013
-
[4]
Fast stagewise sparse factor regression
Kun Chen, Ruipeng Dong, Wanwan Xu, and Zemin Zheng. Fast stagewise sparse factor regression. The Journal of Machine Learning Research, 23 0 (1): 0 12411--12455, 2022
work page 2022
-
[5]
Sparse reduced-rank regression for simultaneous dimension reduction and variable selection
Lisha Chen and Jianhua Z Huang. Sparse reduced-rank regression for simultaneous dimension reduction and variable selection. Journal of the American Statistical Association, 107 0 (500): 0 1533--1545, 2012
2012
-
[6]
Ming Chen and Zhi-Long Chen. Recent developments in dynamic pricing research: multiple products, competition, and limited demand information. Production and Operations Management, 24 0 (5): 0 704--731, 2015
work page 2015
-
[7]
Modeling multimodal continuous heterogeneity in conjoint analysis—a sparse learning approach
Yupeng Chen, Raghuram Iyengar, and Garud Iyengar. Modeling multimodal continuous heterogeneity in conjoint analysis—a sparse learning approach. Marketing Science, 36 0 (1): 0 140--156, 2017
work page 2017
-
[8]
Choice-based conjoint analysis
Felix Eggers, Henrik Sattler, Thorsten Teichert, and Franziska V \"o lckner. Choice-based conjoint analysis. In Handbook of market research, pages 781--819. Springer, 2021
work page 2021
Show all 40 references
-
[9]
A convex optimization approach to modeling consumer heterogeneity in conjoint estimation
Theodoros Evgeniou, Massimiliano Pontil, and Olivier Toubia. A convex optimization approach to modeling consumer heterogeneity in conjoint estimation. Marketing Science, 26 0 (6): 0 805--818, 2007
2007
-
[10]
A selective overview of variable selection in high dimensional feature space
Jianqing Fan and Jinchi Lv. A selective overview of variable selection in high dimensional feature space. Statistica Sinica, 20 0 (1): 0 101, 2010
2010
-
[11]
Asymptotic properties for combined l 1 and concave regularization
Yingying Fan and Jinchi Lv. Asymptotic properties for combined l 1 and concave regularization. Biometrika, 101 0 (1): 0 57--70, 2014
2014
-
[12]
Group network hawkes process
Guanhua Fang, Ganggang Xu, Haochen Xu, Xuening Zhu, and Yongtao Guan. Group network hawkes process. Journal of the American Statistical Association, 119 0 (547): 0 2328--2344, 2024
2024
-
[13]
No spurious local minima in nonconvex low rank problems: A unified geometric analysis
Rong Ge, Chi Jin, and Yi Zheng. No spurious local minima in nonconvex low rank problems: A unified geometric analysis. In International Conference on Machine Learning, pages 1233--1242. PMLR, 2017
2017
-
[14]
Bayesian reduced rank regression in econometrics
John Geweke. Bayesian reduced rank regression in econometrics. Journal of econometrics, 75 0 (1): 0 121--146, 1996
1996
-
[15]
Procrustes problems, volume 30
John C Gower and Garmt B Dijksterhuis. Procrustes problems, volume 30. OUP Oxford, 2004
2004
-
[16]
A practical guide to geostatistical mapping, volume 52
Tomislav Hengl. A practical guide to geostatistical mapping, volume 52. Hengl Amsterdam, Netherlands, 2009
2009
-
[17]
Second-order semi-parametric inference for multivariate log gaussian cox processes
Kristian Bj rn Hessellund, Ganggang Xu, Yongtao Guan, and Rasmus Waagepetersen. Second-order semi-parametric inference for multivariate log gaussian cox processes. Journal of the Royal Statistical Society Series C: Applied Statistics, 71 0 (1): 0 244--268, 2022 a
2022
-
[18]
Semiparametric multinomial logistic regression for multivariate point pattern data
Kristian Bj rn Hessellund, Ganggang Xu, Yongtao Guan, and Rasmus Waagepetersen. Semiparametric multinomial logistic regression for multivariate point pattern data. Journal of the American Statistical Association, 117 0 (539): 0 1500--1515, 2022 b
2022
-
[19]
On iterative hard thresholding methods for high-dimensional m-estimation
Prateek Jain, Ambuj Tewari, and Purushottam Kar. On iterative hard thresholding methods for high-dimensional m-estimation. Advances in neural information processing systems, 27, 2014
2014
-
[20]
Dynamic assortment personalization in high dimensions
Nathan Kallus and Madeleine Udell. Dynamic assortment personalization in high dimensions. Operations Research, 68 0 (4): 0 1020--1037, 2020
2020
-
[21]
Menu-choice modeling
Wagner A Kamakura and Kyuseop Kwak. Menu-choice modeling. Available at SSRN 2162019, 2012
2012
-
[22]
Dynamic pricing with an unknown demand model: Asymptotically optimal semi-myopic policies
N Bora Keskin and Assaf Zeevi. Dynamic pricing with an unknown demand model: Asymptotically optimal semi-myopic policies. Operations research, 62 0 (5): 0 1142--1167, 2014
2014
-
[23]
Plants of the world online
Royal Botanic Gardens Kew. Plants of the world online. https://powo.science.kew.org/taxon/urn:lsid:ipni.org:names:23279-1/general-information, 2017. Accessed: 2024-12-18
2017
-
[24]
Two-way homogeneity pursuit for quantile network vector autoregression
Wenyang Liu, Ganggang Xu, Jianqing Fan, and Xuening Zhu. Two-way homogeneity pursuit for quantile network vector autoregression. arXiv preprint arXiv:2404.18732, 2024
2024 arXiv
-
[25]
Vegetative growth of the common bean in response to phosphorus nutrition
Jonathan Lynch, Andr \'e L \"a uchli, and Emanuel Epstein. Vegetative growth of the common bean in response to phosphorus nutrition. Crop science, 31 0 (2): 0 380--387, 1991
1991
-
[26]
Modelling the choice of residential location
Daniel McFadden. Modelling the choice of residential location. Technical report, Cowles Foundation for Research in Economics, Yale University, 1977
1977
-
[27]
A cluster elastic net for multivariate regression
Bradley S Price and Ben Sherwood. A cluster elastic net for multivariate regression. Journal of Machine Learning Research, 2018
2018
-
[28]
Detecting clusters in multivariate response regression
Bradley S Price, Corban Allenbrand, and Ben Sherwood. Detecting clusters in multivariate response regression. Wiley Interdisciplinary Reviews: Computational Statistics, 14 0 (3): 0 e1551, 2022
2022
-
[29]
Multivariate Reduced-Rank Regression: Theory, Methods and Applications, volume 225
Gregory C Reinsel, Raja P Velu, and Kun Chen. Multivariate Reduced-Rank Regression: Theory, Methods and Applications, volume 225. Springer Nature, 2022
2022
-
[30]
Selective factor extraction in high dimensions
Yiyuan She. Selective factor extraction in high dimensions. Biometrika, 104 0 (1): 0 97--110, 2017
2017
-
[31]
Supervised multivariate learning with simultaneous feature auto-grouping and dimension reduction
Yiyuan She, Jiahui Shen, and Chao Zhang. Supervised multivariate learning with simultaneous feature auto-grouping and dimension reduction. Journal of the Royal Statistical Society Series B: Statistical Methodology, 84 0 (3): 0 912--932, 2022
2022
-
[32]
On the use of minimum penalties in statistical learning
Ben Sherwood and Bradley S Price. On the use of minimum penalties in statistical learning. Journal of Computational and Graphical Statistics, 33 0 (1): 0 138--151, 2024
2024
-
[33]
A blockwise descent algorithm for group-penalized multiresponse and multinomial regression
Noah Simon, Jerome Friedman, and Trevor Hastie. A blockwise descent algorithm for group-penalized multiresponse and multinomial regression. arXiv preprint arXiv:1311.6529, 2013
2013 arXiv
-
[34]
Guaranteed matrix completion via non-convex factorization
Ruoyu Sun and Zhi-Quan Luo. Guaranteed matrix completion via non-convex factorization. IEEE Transactions on Information Theory, 62 0 (11): 0 6535--6579, 2016
2016
-
[35]
Sofar: Large-scale association network learning
Yoshimasa Uematsu, Yingying Fan, Kun Chen, Jinchi Lv, and Wei Lin. Sofar: Large-scale association network learning. IEEE transactions on information theory, 65 0 (8): 0 4924--4939, 2019
2019
-
[36]
Simultaneous dimension reduction and variable selection for multinomial logistic regression
Canhong Wen, Zhenduo Li, Ruipeng Dong, Yijin Ni, and Wenliang Pan. Simultaneous dimension reduction and variable selection for multinomial logistic regression. INFORMS Journal on Computing, 2023
2023
-
[37]
Rate minimaxity of the lasso and dantzig selector for the lq loss in lr balls
Fei Ye and Cun-Hui Zhang. Rate minimaxity of the lasso and dantzig selector for the lq loss in lr balls. The Journal of Machine Learning Research, 11: 0 3519--3540, 2010
2010
-
[38]
Scalable interpretable multi-response regression via seed
Zemin Zheng, M Taha Bahadori, Yan Liu, and Jinchi Lv. Scalable interpretable multi-response regression via seed. Journal of Machine Learning Research, 20 0 (107): 0 1--34, 2019
2019
-
[39]
Learning social infectivity in sparse low-rank networks using multi-dimensional hawkes processes
Ke Zhou, Hongyuan Zha, and Le Song. Learning social infectivity in sparse low-rank networks using multi-dimensional hawkes processes. In Artificial intelligence and statistics, pages 641--649. PMLR, 2013
2013
-
[40]
Simultaneous estimation and group identification for network vector autoregressive model with heterogeneous nodes
Xuening Zhu, Ganggang Xu, and Jianqing Fan. Simultaneous estimation and group identification for network vector autoregressive model with heterogeneous nodes. Journal of Econometrics, page 105564, 2023
2023
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.