arxiv: 2605.01571 · v1 · submitted 2026-05-02 · 📊 stat.OT · stat.CO

Recognition: unknown

Functional Liu Regression for Scalar-on-Functional Models in High-Dimensional Settings

Farrukh Javed, Ismail Shah, Shaista Ashraf, Stephen Becker

Pith reviewed 2026-05-10 15:10 UTC · model grok-4.3

classification 📊 stat.OT stat.CO

keywords functional data analysisscalar-on-function regressionLiu estimatorshrinkage estimationhigh-dimensionalmulticollinearityGCVMSE decomposition

0 comments

The pith

The functional Liu estimator yields an explicit optimal shrinkage parameter via one-dimensional convex risk minimization for scalar-on-function regression.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops a functional Liu-type shrinkage estimator called fLiu for scalar-on-function regression models when functional predictors exhibit strong multicollinearity and the setting is high-dimensional. It extends the classical Liu estimator by incorporating directional shrinkage together with smoothness regularization on the functional coefficients after basis expansion. An explicit mean squared error decomposition is derived that separates the effects of variance reduction from shrinkage bias, which in turn produces a practical plug-in rule for selecting the shrinkage parameter by solving a one-dimensional convex minimization problem. The analysis further shows that in underdetermined high-dimensional regimes, generalized cross-validation and equivalent leave-one-out criteria become constant with respect to the shrinkage parameter and therefore provide no guidance for tuning. A sympathetic reader would care because this supplies a theoretically justified alternative to cross-validation precisely when standard tuning methods break down in functional data applications.

Core claim

The fLiu estimator combines directional shrinkage with smoothness regularization in the functional setting. An explicit MSE decomposition is derived, characterizing the risk in terms of variance reduction and shrinkage bias. This decomposition yields an explicit optimal choice of the shrinkage parameter through a one-dimensional convex risk minimization problem, leading to a practical plug-in tuning rule. In high-dimensional underdetermined settings, GCV, PRESS, and LOO-CV are shown to be constant with respect to the shrinkage parameter and thus uninformative for selection. Numerical experiments demonstrate competitive predictive accuracy relative to existing methods.

What carries the argument

The explicit mean squared error decomposition of the fLiu estimator, which separates variance reduction from shrinkage bias and enables direct one-dimensional convex minimization for the shrinkage parameter.

If this is right

The shrinkage parameter can be selected directly from the plug-in rule without grid search or cross-validation.
In high-dimensional underdetermined regimes, tuning must rely on the risk-minimization approach rather than GCV or PRESS.
The estimator maintains competitive predictive accuracy while allowing flexible control over bias-variance trade-off via the shrinkage and smoothness parameters.
The theoretical analysis explains why Liu-type methods have previously focused on overdetermined regimes.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The constant-GCV result points to a general limitation of cross-validation in underdetermined functional models, suggesting that explicit risk decompositions may be needed for tuning other shrinkage estimators in similar settings.
The plug-in rule could be tested as a replacement for cross-validation in related functional regression techniques such as functional principal component regression or penalized spline methods when the number of basis functions exceeds the sample size.
Applied fields that routinely encounter smooth but highly correlated functional predictors, such as spectroscopy or longitudinal biomedical data, could adopt the plug-in rule to obtain more stable coefficient estimates.

Load-bearing premise

The mean squared error decomposition and resulting plug-in tuning rule remain valid under the functional basis expansion and smoothness regularization chosen by the user.

What would settle it

A numerical experiment in an underdetermined scalar-on-function setting where the true mean squared error of the fLiu estimator is not minimized at the shrinkage value returned by the plug-in rule derived from the MSE decomposition.

Figures

Figures reproduced from arXiv: 2605.01571 by Farrukh Javed, Ismail Shah, Shaista Ashraf, Stephen Becker.

**Figure 1.** Figure 1: Monthly temperature variables in W = (wij ) ∈ R 35×12, where wij denotes the temperature recorded at station i during month j, shown through a pairwise scatterplot matrix for the twelve months (Jan–Dec). Each point represents one station observed jointly for the corresponding pair of months. Diagonal panels display the marginal distributions of each monthly variable, lower-triangular panels show pairwise s… view at source ↗

**Figure 2.** Figure 2: Estimated coefficient functions β(s) for OLS, Ridge, Liu, Generalized Ridge, and fLiu over the annual cycle. All estimators exhibit similar seasonal patterns, while the Generalized Ridge estimator shows the largest deviation. cipitation. The curves alternate between positive and negative regions, suggesting varying seasonal effects on precipitation. Differences among estimators are modest overall, with fLi… view at source ↗

**Figure 3.** Figure 3: Distribution of test residuals across the five competing estimators. The box plot [PITH_FULL_IMAGE:figures/full_fig_p021_3.png] view at source ↗

**Figure 4.** Figure 4: GCV as a function of the Liu parameter d for the Canadian Weather data. Left panel: over-determined setting (K = 11), where the GCV criterion varies substantially with d. Right panel: under-determined setting (K = 35), where the GCV curve is essentially constant, confirming the degeneracy result of Theorem 3. 5 Conclusion This paper proposes and analyzes a functional Liu-type shrinkage estimator (fLiu) for… view at source ↗

read the original abstract

This study develops a functional Liu-type shrinkage estimator (fLiu) for scalar-on-function regression in the presence of strong multicollinearity and high-dimensional functional predictors. The approach extends the classical Liu estimator to the functional setting by combining directional shrinkage with smoothness regularization, providing flexible control over the bias-variance trade-off. Theoretical analysis is used to examine the behavior of the estimator and the associated parameter selection problem. In particular, an explicit mean squared error (MSE) decomposition is derived, characterizing the risk of the estimator in terms of variance reduction and shrinkage bias. This further yields an explicit optimal choice of the shrinkage parameter of the fLiu estimator through a one-dimensional convex risk minimization problem, leading to a practical plug-in tuning rule. Moreover, it is shown that in high-dimensional (underdetermined) settings, commonly used criterion such as GCV (and equivalently PRESS/LOO-CV) become constant with respect to the parameter d, thus uninformative for tuning. This provides a theoretical explanation for the predominant focus on the overdetermined regime in existing Liu-type methods. Numerical results demonstrate that the estimator achieves competitive predictive accuracy relative to existing methods. Implementation is carried out in R using the fda package, and in Python via the fLiu.py package developed for this study.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

fLiu gives a clean MSE decomposition for tuning the shrinkage parameter in high-dimensional scalar-on-functional regression and proves GCV is constant (hence useless) when underdetermined, but the decomposition's validity with the smoothness penalty needs explicit verification.

read the letter

The paper introduces a functional Liu estimator that adds directional shrinkage to smoothness-regularized basis expansions for scalar-on-function regression. It derives an MSE decomposition that splits variance reduction from shrinkage bias, then uses that to set up a one-dimensional convex problem for the optimal shrinkage parameter d. It also shows that GCV, PRESS, and LOO-CV become constant in d underdetermined regimes, which explains why those criteria are uninformative there and why most prior Liu work stayed in the overdetermined case. The R and Python code packages are a practical plus for anyone wanting to try it.

Referee Report

2 major / 2 minor

Summary. The paper develops a functional Liu-type shrinkage estimator (fLiu) for scalar-on-functional regression under multicollinearity and high-dimensional predictors. It extends the classical Liu estimator by combining directional shrinkage with smoothness regularization, derives an explicit MSE decomposition separating variance reduction from shrinkage bias, obtains an optimal shrinkage parameter d via one-dimensional convex risk minimization with a plug-in tuning rule, proves that GCV/PRESS/LOO-CV become constant (hence uninformative) with respect to d in underdetermined regimes, and reports competitive numerical performance with R and Python implementations.

Significance. If the MSE decomposition and GCV constancy hold after basis expansion and smoothness penalization, the work supplies a theoretically justified tuning procedure for shrinkage in high-dimensional functional regression where standard cross-validation fails, extending Liu-type methods beyond the overdetermined regime. The explicit 1D convex optimization and open implementations (fda in R, fLiu.py in Python) are practical strengths that support reproducibility.

major comments (2)

[Theoretical analysis (MSE decomposition)] Theoretical analysis section (MSE decomposition): The derivation of the explicit MSE decomposition and resulting 1D convex risk minimization for d must explicitly track the interaction between the finite basis expansion (e.g., B-splines), the smoothness penalty matrix, and the covariance operator of the functional predictor. It is not shown whether the decomposition remains valid without additional assumptions (such as commutativity of the penalty with the covariance operator or orthonormality of the basis) when the number of basis functions exceeds sample size; this is load-bearing for the plug-in tuning rule.
[High-dimensional underdetermined settings (GCV constancy)] Section on GCV constancy in underdetermined settings: The claim that GCV (and PRESS/LOO-CV) is constant with respect to d requires a demonstration that the trace of the hat matrix remains independent of d after the smoothness penalty is incorporated into the effective design operator. No explicit verification is provided that this independence survives the functional regularization in the p > n regime, which underpins the explanation for why existing Liu methods focus on overdetermined cases.

minor comments (2)

[Numerical results] The abstract states that numerical results are 'competitive' but provides no details on the simulation design, basis choice, or specific competitors; a table summarizing MSE or prediction error across methods would improve clarity.
[Methods] Notation for the shrinkage parameter d and the penalty matrix should be introduced consistently in the methods section to avoid ambiguity when the functional predictor is projected onto the basis.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and detailed comments, which help clarify the presentation of our theoretical results. We address each major comment below and will revise the manuscript accordingly to make the derivations more explicit.

read point-by-point responses

Referee: Theoretical analysis section (MSE decomposition): The derivation of the explicit MSE decomposition and resulting 1D convex risk minimization for d must explicitly track the interaction between the finite basis expansion (e.g., B-splines), the smoothness penalty matrix, and the covariance operator of the functional predictor. It is not shown whether the decomposition remains valid without additional assumptions (such as commutativity of the penalty with the covariance operator or orthonormality of the basis) when the number of basis functions exceeds sample size; this is load-bearing for the plug-in tuning rule.

Authors: We agree that explicit tracking of these interactions improves clarity. The MSE decomposition is derived after the finite basis expansion (e.g., B-splines) and incorporation of the smoothness penalty matrix P into the regularized normal equations, yielding an estimator of the form (X^T X + λP + d I)^{-1}(X^T y + d β_0) in the discretized space. The bias-variance decomposition follows directly from the general linear estimator formula and holds without requiring commutativity between P and the covariance operator or orthonormality of the basis functions; it relies only on the finite-dimensional post-expansion model. In the revision we will add a dedicated paragraph (and supporting steps in an appendix) that explicitly substitutes the penalized design operator and confirms the decomposition remains valid in the underdetermined regime (basis dimension > n) under the paper's stated assumptions. This will directly support the plug-in tuning rule. revision: yes
Referee: Section on GCV constancy in underdetermined settings: The claim that GCV (and PRESS/LOO-CV) is constant with respect to d requires a demonstration that the trace of the hat matrix remains independent of d after the smoothness penalty is incorporated into the effective design operator. No explicit verification is provided that this independence survives the functional regularization in the p > n regime, which underpins the explanation for why existing Liu methods focus on overdetermined cases.

Authors: We appreciate the request for explicit verification. The constancy result follows because, after basis expansion in the p > n regime, the effective hat matrix H(d) = X (X^T X + λP + d I)^{-1} X^T has trace equal to the rank of the column space of X (which is at most n), independent of d; the penalty term λP and shrinkage d affect only the conditioning within the column space but not the projection onto it when the system is underdetermined. In the revision we will insert the algebraic steps computing tr(H(d)) explicitly, showing that the derivative with respect to d vanishes once the effective dimension exceeds n, with the smoothness penalty P included. This will strengthen the explanation for the focus on overdetermined regimes in prior Liu-type literature. revision: yes

Circularity Check

0 steps flagged

No circularity: MSE decomposition and optimal shrinkage derived independently from estimator definition

full rationale

The paper derives an explicit MSE decomposition for the fLiu estimator from its functional form, basis expansion, and smoothness penalty, then minimizes the resulting risk expression to obtain the optimal shrinkage parameter d via a one-dimensional convex problem. This is a standard first-principles derivation that does not reduce to the inputs by construction or rename a fitted quantity as a prediction. The plug-in tuning rule applies data-based estimates to the already-derived optimal expression, which is not circular under the enumerated patterns. The constancy of GCV/PRESS in the underdetermined regime follows from algebraic properties of the hat matrix after regularization and is presented as a theoretical result rather than a self-referential fit. No self-citation chains, ansatz smuggling, or uniqueness theorems imported from the authors' prior work are load-bearing for the central claims. The derivation remains self-contained against the model assumptions stated in the abstract.

Axiom & Free-Parameter Ledger

1 free parameters · 1 axioms · 0 invented entities

The central claim rests on standard functional data analysis assumptions plus one data-dependent tuning parameter; no new physical or mathematical entities are postulated.

free parameters (1)

shrinkage parameter d
Chosen via plug-in minimization of the derived MSE risk expression.

axioms (1)

domain assumption Functional predictors admit a basis expansion whose coefficients satisfy a smoothness penalty that can be combined with directional shrinkage.
Invoked throughout the extension of the classical Liu estimator to the functional setting.

pith-pipeline@v0.9.0 · 5530 in / 1278 out tokens · 27964 ms · 2026-05-10T15:10:56.998543+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

16 extracted references

[1]

& Ka¸ ciranlar, S

Akdeniz, F. & Ka¸ ciranlar, S. (2007), ‘A generalized Liu estimator for handling multicollinear- ity’,Journal of Statistical Planning and Inference137, 1872–1880

2007
[2]

Barata, J. C. A. & Hussein, M. S. (2012), ‘The Moore-Penrose pseudoinverse: A tutorial review of the theory’,Brazilian Journal of Physics42(1), 146–165

2012
[3]

& Sarda, P

Cardot, H., Ferraty, F. & Sarda, P. (2003), ‘Spline estimators for the functional linear model’, Statistica Sinica13(3), 571–591. 24

2003
[4]

& Wahba, G

Craven, P. & Wahba, G. (1979), ‘Smoothing noisy data with spline functions: Estimating the correct degree of smoothing by the method of generalized cross-validation’,Numerische Mathematik31(4), 377–403

1979
[5]

Eilers, P. H. & Marx, B. D. (1996), ‘Flexible smoothing with B-splines and penalties’,Sta- tistical Science11(2), 89–121

1996
[6]

& Kurnaz, F

Filzmoser, P. & Kurnaz, F. S. (2016), ‘A robust liu regression estimator’,Communications in Statistics—Theory and Methods47(6), 1285–1298

2016
[7]

Gruber, M. H. J. (2010), ‘Liu and ridge estimators: A comparison’,Communications in Statistics—Theory and Methods39(8), 1485–1494

2010
[8]

Hoerl, A. E. & Kennard, R. W. (1970), ‘Ridge regression: Biased estimation for nonorthog- onal problems’,Technometrics12(1), 55–67

1970
[9]

Kibria, B. M. G. (2003), ‘Some improved ridge regression estimators and their applications’, Journal of Modern Applied Statistical Methods2, 133–144

2003
[10]

(1993), ‘A new class of biased estimate in linear regression’,Communications in Statistics — Theory and Methods22, 393–402

Liu, K. (1993), ‘A new class of biased estimate in linear regression’,Communications in Statistics — Theory and Methods22, 393–402

1993
[11]

(2003), ‘On the statistical properties of the Liu estimator’,Communications in Statistics — Theory and Methods32(5), 1009–1020

Liu, K. (2003), ‘On the statistical properties of the Liu estimator’,Communications in Statistics — Theory and Methods32(5), 1009–1020

2003
[12]

& Maity, A

Mehrotra, S. & Maity, A. (2022), ‘Simultaneous variable selection, clustering, and smoothing in function-on-scalar regression’,Canadian Journal of Statistics50(1), 180–199. ¨Ozkale, M. R. & Ka¸ ciranlar, S. (2007), ‘The restricted and unrestricted two-parameter esti- mator’,Statistics and Probability Letters77(4), 438–446

2022
[13]

O., Hooker, G

Ramsay, J. O., Hooker, G. & Graves, S. (2009),Functional Data Analysis with R and MAT-

2009
[14]

Ramsay, J. O. & Silverman, B. W. (2002),Applied Functional Data Analysis: Methods and Case Studies, 1st edn, Springer, New York

2002
[15]

Ramsay, J. O. & Silverman, B. W. (2005),Functional Data Analysis, 2 edn, Springer

2005
[16]

Tikhonov, A. N. & Arsenin, V. Y. (1977),Solutions of Ill-Posed Problems, Winston and

1977