Recognition: unknown
Theta-regularized Kriging: Modelling and Algorithms
Pith reviewed 2026-05-10 09:28 UTC · model grok-4.3
The pith
Theta-regularized Kriging penalizes the theta hyperparameter in Gaussian stochastic processes using Lasso, Ridge, or Elastic-net, yielding higher accuracy and stability than prior penalized Kriging variants on numerical tests and engineering cases.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Compared with other penalized Kriging models, the proposed model performs better in terms of accuracy and stability.
Load-bearing premise
That penalizing theta specifically, derived from the maximum-likelihood perspective, produces genuine gains that generalize beyond the nine numerical functions and two engineering examples shown, without the improvements arising from post-hoc tuning or limited test diversity.
Figures
read the original abstract
To obtain more accurate model parameters and improve prediction accuracy, we proposed a regularized Kriging model that penalizes the hyperparameter theta in the Gaussian stochastic process, termed the Theta-regularized Kriging. We derived the optimization problem for this model from a maximum likelihood perspective. Additionally, we presented specific implementation details for the iterative process, including the regularized optimization algorithm and the geometric search cross-validation tuning algorithm. Three distinct penalty methods, Lasso, Ridge, and Elastic-net regularization, were meticulously considered. Meanwhile, the proposed Theta-regularized Kriging models were tested on nine common numerical functions and two practical engineering examples. The results demonstrate that, compared with other penalized Kriging models, the proposed model performs better in terms of accuracy and stability.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Theta-regularized Kriging, which penalizes the hyperparameter theta in the Gaussian process correlation function using Lasso, Ridge, or Elastic-net penalties. This is derived from a penalized maximum likelihood objective, and the authors detail iterative optimization algorithms and a geometric-search cross-validation procedure for tuning the regularization parameter lambda. The method is tested on nine numerical functions and two engineering examples, where it is claimed to outperform other penalized Kriging models in terms of prediction accuracy and stability.
Significance. Should the empirical advantages prove robust under controlled comparisons, this regularization strategy on theta could improve the reliability of Kriging surrogates in engineering design and optimization tasks by mitigating overfitting in hyperparameter estimation. The explicit algorithmic contributions, including the CV tuning method, add practical value and support reproducibility.
major comments (2)
- [§5 (Numerical Experiments)] §5 (Numerical Experiments): The central claim that the proposed model outperforms other penalized Kriging models in accuracy and stability is not fully supported, as there is no explicit confirmation that baseline methods were tuned using the identical geometric-search cross-validation procedure described for Theta-regularized Kriging. Without this, observed differences may stem from unequal hyperparameter search effort rather than the specific penalization of theta.
- [§5.1, Tables 1-2] §5.1, Tables 1-2: The reported RMSE and stability metrics lack error bars, standard deviations from repeated runs, or formal statistical significance tests, which undermines the ability to verify that the claimed improvements are reliable rather than artifacts of the specific test functions or initialization.
minor comments (2)
- [Introduction] Introduction: Expand the discussion of why penalizing theta (as opposed to the nugget or other parameters) is particularly effective, with additional citations to existing sensitivity analyses in Gaussian process literature.
- [Algorithm 1] Algorithm 1: Clarify the convergence criteria and initialization strategy for the iterative regularized optimization to improve reproducibility.
Simulated Author's Rebuttal
We thank the referee for the detailed and constructive comments on our manuscript. We address each major comment below, indicating the revisions we will make to improve the clarity and rigor of the numerical experiments section.
read point-by-point responses
-
Referee: §5 (Numerical Experiments): The central claim that the proposed model outperforms other penalized Kriging models in accuracy and stability is not fully supported, as there is no explicit confirmation that baseline methods were tuned using the identical geometric-search cross-validation procedure described for Theta-regularized Kriging. Without this, observed differences may stem from unequal hyperparameter search effort rather than the specific penalization of theta.
Authors: We acknowledge that the manuscript does not explicitly confirm that the baseline penalized Kriging models were tuned with the identical geometric-search cross-validation procedure. In the original experiments, baselines followed the hyperparameter settings and tuning approaches reported in their respective source papers (typically grid search over theta with fixed or default regularization). To address this, we will revise §5 to include a dedicated subsection detailing the uniform application of the geometric-search CV procedure to all compared methods, including the baselines. Where necessary, we will re-run the baseline comparisons under this consistent tuning protocol to ensure the performance differences can be attributed to the theta penalization. revision: yes
-
Referee: §5.1, Tables 1-2: The reported RMSE and stability metrics lack error bars, standard deviations from repeated runs, or formal statistical significance tests, which undermines the ability to verify that the claimed improvements are reliable rather than artifacts of the specific test functions or initialization.
Authors: We agree that the current presentation of results in Tables 1-2 would be strengthened by quantitative measures of variability and statistical testing. We will revise the numerical experiments section to report means and standard deviations computed over 10 independent runs with randomized initializations for each method and test function. Additionally, we will include results from paired statistical tests (e.g., Wilcoxon signed-rank or t-tests with appropriate corrections) to assess the significance of observed differences in RMSE and stability metrics. revision: yes
Circularity Check
No circularity: regularization is an explicit additive penalty on the standard Kriging MLE; performance claims rest on external benchmarks.
full rationale
The derivation begins from the ordinary maximum-likelihood objective for Kriging and inserts an explicit penalty term on the correlation-length vector theta (Lasso, Ridge, or Elastic-net). This construction is not self-referential; the penalized objective is a standard regularized extension rather than a re-expression of its own fitted values. The geometric-search CV procedure is described as a separate tuning step for the penalty strength, not as a post-hoc fit that is then relabeled a prediction. All reported accuracy and stability gains are measured on nine independent numerical test functions plus two engineering examples, none of which are used to define the model itself. No self-citation chain, uniqueness theorem, or ansatz smuggling appears in the load-bearing steps. The central empirical claim therefore remains falsifiable against external data and does not reduce to its own inputs by construction.
Axiom & Free-Parameter Ledger
free parameters (1)
- regularization strength lambda
axioms (1)
- domain assumption The observed responses follow a Gaussian stochastic process whose correlation structure is controlled by the hyperparameter theta.
Reference graph
Works this paper leans on
-
[1]
N. Hosseini, M. Tadjfar, A. Abba, Configuration optimization of two tandem airfoils at low Reynolds numbers, Appl. Math. Model. (102) (2022) 828–846.doi:10.1016/j.apm.2021.10.029
-
[2]
D. Zhan, H. Xing, A fast Kriging-assisted evolutionary algorithm based on incremental learning, IEEE Trans. Evol. Comput. 25 (5) (2021) 941–955.doi:10.1109/TEVC.2021.3067015. Xuelin Xie et al.:Preprint submitted to Elsevier Page 24 of 26 Theta-regularized Kriging: Modelling and Algorithms
-
[3]
M. A. Alves, P. J. Oliveira, F. T. Pinho, Numerical methods for viscoelastic fluid flows, Annu. Rev. Fluid Mech. 53 (1) (2020) 1–33. doi:10.1146/annurev-fluid-010719-060107
-
[4]
M. A. Sanchez, B. Cockburn, N.-C. Nguyen, J. Peraire, Symplectic Hamiltonian finite element methods for linear elastodynamics, Comput. Meth. Appl. Mech. Eng. 381 (2021) 113843.doi:10.1016/j.cma.2021.113843
-
[5]
T. Chugh, Y. Jin, K. Miettinen, J. Hakanen, et al., A surrogate-assisted reference vector guided evolutionary algorithm for computationally expensive many-objective optimization, IEEE Trans. Evol. Comput. 22 (1) (2018) 129–142.doi:10.1109/TEVC.2016.2622301
-
[6]
X.Peng,J.Kou,W.Zhang,Multi-fidelitynonlinearunsteadyaerodynamicmodelinganduncertaintyestimationbasedonHierarchicalKriging, Appl. Math. Model. 122 (2023) 1–21.doi:10.1016/j.apm.2023.05.031
-
[7]
R. T. Johnson, The design and analysis of computer experiments, Springer Science & Business Media, New York, 2008
2008
-
[8]
P. V. Arun, A comparative analysis of different DEM interpolation methods, Egypt. J. Remote Sens. Space Sci. 16 (2013) 133–139. doi:10.1016/j.ejrs.2013.09.001
-
[9]
K. Pavlicek, V. Kotlan, I. Dolezel, Applicability and comparison of surrogate techniques for modeling of selected heating problems, Comput. Math. Appl. 78 (9) (2019) 2897–2910.doi:10.1016/j.camwa.2019.02.013
-
[10]
A. Johari, H. Fooladi, System reliability analysis of site slope using the conditional spatial variability of soil properties, Iran. J. Sci. Technol.- Trans. Civ. Eng. 47 (6) (2023) 3743–3761.doi:10.1007/s40996-023-01200-z
-
[11]
A. Johari, H. Fooladi, Simulation of the conditional models of borehole’s characteristics for slope reliability assessment, Transp. Geotech. 35 (2022). doi:10.1016/j.trgeo.2022.100778
-
[12]
N. Pouladi, A. B. Moller, S. Tabatabai, M. H. Greve, Mapping soil organic matter contents at field level with Cubist, Random Forest and Kriging, Geoderma 342 (2019) 85–92.doi:10.1016/j.geoderma.2019.02.019
-
[13]
X. Wu, Q. Lin, W. Lin, Y. Ye, et al., A Kriging model-based evolutionary algorithm with support vector machine for dynamic multimodal optimization, Eng. Appl. Artif. Intell. 122 (2023) 106039.doi:10.1016/j.engappai.2023.106039
-
[14]
E. Park, B. W. Brorsen, A. Harri, Using Bayesian Kriging for spatial smoothing in crop insurance rating, Am. J. Agr. Econ. 101 (1) (2019) 330–351.doi:10.1093/ajae/aay045
-
[15]
K.Wu,G.A.Rodriguez,M.Zajc,E.Jacquemin,etal.,Anewdrone-borneGPRforsoilmoisturemapping,RemoteSens.Environ.235(2019) 111456. doi:10.1016/j.rse.2019.111456
-
[16]
L. Chen, H. Qiu, L. Gao, C. Jiang, et al., A screening-based gradient-enhanced Kriging modeling method for high-dimensional problems, Appl. Math. Model. 69 (2019) 15–31.doi:10.1016/j.apm.2018.11.048
-
[17]
X. Cai, L. Gao, X. Li, Efficient generalized surrogate-assisted evolutionary algorithm for high-dimensional expensive problems, IEEE Trans. Evol. Comput. 24 (2) (2020) 365–379.doi:10.1109/TEVC.2019.2919762
-
[18]
Dietterich, Overfitting and undercomputing in machine learning, ACM Comput
T. Dietterich, Overfitting and undercomputing in machine learning, ACM Comput. Surv. 27 (3) (1995) 326–327.doi:10.1145/212094. 212114
-
[19]
Z. Luo, S. Cai, G. Chen, J. Gao, et al., Improving data analytics with fast and adaptive regularization, IEEE Trans. Knowl. Data Eng. 33 (2) (2021) 551–568.doi:10.1109/TKDE.2019.2916683
-
[20]
L. Calatroni, G. Garrigos, L. Rosasco, S. Villa, Accelerated iterative regularization via dual diagonal descent, SIAM J. Optim. 31 (1) (2021) 754–784.doi:10.1137/19M1308888
-
[21]
Z. Sun, M. Dai, Y. Wang, S.-B. Lin, Nystrom regularization for time series forecasting, J. Mach. Learn. Res. 23 (312) (2022) 1–42
2022
-
[22]
W. Gong, Z. Huang, L. Yang, Accurate regularized Tucker decomposition for image restoration, Appl. Math. Model. 123 (2023) 75–86. doi:10.1016/j.apm.2023.06.031
-
[23]
Y. Fan, W. Yang, A backpropagation learning algorithm with graph regularization for feedforward neural networks, Inf. Sci. 607 (2022) 263–277.doi:10.1016/j.ins.2022.05.121
-
[24]
Hung, Penalized blind Kriging in computer experiments, Stat
Y. Hung, Penalized blind Kriging in computer experiments, Stat. Sin. 21 (3) (2011) 1171–1190.doi:10.5705/ss.2009.226
-
[25]
Y. Zhang, W. Yao, S. Ye, X. Chen, A regularization method for constructing trend function in Kriging model, Struct. Multidiscip. Optim. 59 (4) (2019) 1221–1239.doi:10.1007/s00158-018-2127-8
-
[26]
Park, Lasso Kriging for efficiently selecting a global trend model, Struct
I. Park, Lasso Kriging for efficiently selecting a global trend model, Struct. Multidiscip. Optim. 64 (3) (2021) 1527–1543.doi:10.1007/ s00158-021-02939-7
2021
-
[27]
Y.Zhao,Z.Feng,M.Li,X.Li,ModifiedPenalizedblindKrigingforefficientlyselectingaglobaltrendmodel,J.Stat.Comput.Simul.93(17) (2023) 3052–3066.doi:10.1080/00949655.2023.2216335
-
[28]
D. G. Krige, A statistical approach to some basic mine valuation problems on the Witwatersrand, J. S. Afr. Inst. Min. Metall. 52 (6) (1952) 119–139
1952
-
[29]
Matheron, The intrinsic random functions and their applications, Adv
G. Matheron, The intrinsic random functions and their applications, Adv. Appl. Probab. 5 (3) (1973) 439–468.doi:10.2307/1425829
-
[30]
Alex Shtoff, Elie Abboud, Rotem Stram, and Oren Somekh
J. Sacks, W. J. Welch, T. J. Mitchell, H. P. Wynn, Design and analysis of computer experiments, Stat. Sci. 4 (4) (1989) 409–423.doi: 10.1214/ss/1177012413
-
[31]
Lophaven, H
S. Lophaven, H. Nielsen, S. J., DACE - A MATLAB Kriging Toolbox (version 2.0) [software], Technical University of Denmark, August 1,
-
[32]
http://www2.imm.dtu.dk/pubdb/p.php?1460
-
[33]
Y. Pang, X. Lai, S. Zhang, Y. Wang, et al., A Kriging-assisted global reliability-based design optimization algorithm with a reliability- constrained expected improvement, Appl. Math. Model. 121 (2023) 611–630.doi:10.1016/j.apm.2023.05.018
-
[34]
Y.Song,X.Wang,G.Wright,D.Thatcher,etal.,Trafficvolumepredictionwithsegment-basedregressionKriginganditsimplementationin assessing the impact of heavy vehicles, IEEE Trans. Intell. Transp. Syst. 20 (1) (2019) 232–243.doi:10.1109/TITS.2018.2805817
-
[35]
X.Zhang,L.Wang,J.D.Sorensen,REIF:Anovelactive-learningfunctiontowardadaptiveKrigingsurrogatemodelsforstructuralreliability analysis, Reliab. Eng. Syst. Saf. 185 (2019) 440–454.doi:10.1016/j.ress.2019.01.014
-
[36]
D. Huang, T. Allen, W. Notz, N. Zeng, Global optimization of stochastic black-box systems via sequential Kriging meta-models, J. Glob. Optim. 34 (3) (2006) 441–466.doi:10.1007/s10898-005-2454-3. Xuelin Xie et al.:Preprint submitted to Elsevier Page 25 of 26 Theta-regularized Kriging: Modelling and Algorithms
-
[37]
62 (12) (2016) 4308–4322.doi:10.1002/aic.15352
O.T.Kajero,R.B.Thorpe,T.Chen,B.Wang,etal.,Krigingmeta-modelassistedcalibrationofcomputationalfluiddynamicsmodels,AICHE J. 62 (12) (2016) 4308–4322.doi:10.1002/aic.15352
-
[38]
Y. Zhou, Z. Lu, An enhanced Kriging surrogate modeling technique for high-dimensional problems, Mech. Syst. Signal Proc. 140 (2020) 106687. doi:10.1016/j.ymssp.2020.106687
-
[39]
Forrester, A
A. Forrester, A. Sobester, A. Keane, Engineering design via surrogate modelling: a practical guide, John Wiley & Sons, Chichester, 2008
2008
-
[40]
J. Roshanian, M. Ebrahimi, Latin hypercube sampling applied to reliability-based multidisciplinary design optimization of a launch vehicle, Aerosp. Sci. Technol. 28 (1) (2013) 297–304.doi:10.1016/j.ast.2012.11.010
-
[41]
S. Gupta, A. Paudel, M. Thapa, S. B. Mulani, et al., Optimal sampling-based neural networks for uncertainty quantification and stochastic optimization, Aerosp. Sci. Technol. 133 (2023).doi:10.1016/j.ast.2023.108109
-
[42]
Y. Li, J. Shi, J. Shen, H. Cen, et al., An adaptive Kriging method with double sampling criteria applied to hydrogen preparation case, Int. J. Hydrog. Energy 45 (56) (2020) 31689–31705.doi:10.1016/j.ijhydene.2020.08.174
-
[43]
S.S.Sathya,M.V.Radhika,Convergenceofnomadicgeneticalgorithmonbenchmarkmathematicalfunctions,Appl.SoftComput.13(2013) 2759–2766.doi:10.1016/j.asoc.2012.11.011
-
[44]
W. Morokoff, R. Caflisch, Quasi-Monte Carlo Integration, J. Comput. Phys. 122 (2) (1995) 218–230.doi:10.1006/jcph.1995.1209
-
[45]
C. Yang, W. Gao, N. Liu, C. Song, Low-discrepancy sequence initialized particle swarm optimization algorithm with high-order nonlinear time-varying inertia weight, Appl. Soft. Comput. 29 (2015) 386–394.doi:10.1016/j.asoc.2015.01.004
-
[46]
3 (4) (1990) 467–483.doi:10.1016/0893-6080(90)90029-K
M.Styblinski,T.Tang,Experimentsinnonconvexoptimization:Stochasticapproximationwithfunctionsmoothingandsimulatedannealing, Neural Netw. 3 (4) (1990) 467–483.doi:10.1016/0893-6080(90)90029-K
-
[47]
P.Kersaudy,B.Sudret,N.Varsier,O.Picon,etal.,AnewsurrogatemodelingtechniquecombiningKrigingandpolynomialchaosexpansions- Applicationtouncertaintyanalysisincomputationaldosimetry,J.Comput.Phys.286(2015)103–117. doi:10.1016/j.jcp.2015.01.034
-
[48]
S.Xiong,P.Z.G.Qian,C.F.J.Wu,Sequentialdesignandanalysisofhigh-accuracyandlow-accuracycomputercodes,Technometrics55(1) (2013) 37–46.doi:10.1080/00401706.2012.723572
-
[49]
A. Okoro, F. Khan, S. Ahmed, Dependency effect on the reliability-based design optimization of complex offshore structure, Reliab. Eng. Syst. Saf. 231 (2023).doi:10.1016/j.ress.2022.109026
-
[50]
Eldred, C
M. Eldred, C. Webster, P. G. Constantine, Evaluation of non-intrusive approaches for Wiener-Askey generalized polynomial chaos, in: 49th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, 2008, pp. 1–22
2008
-
[51]
A. H. Amjadi, A. Johari, Stochasticnonlinear ground response analysis considering existing boreholes locations by the geostatistical method, Bull. Earthq. Eng. 20 (5) (2022) 2285–2327.doi:10.1007/s10518-022-01322-1
-
[52]
A. Johari, H. Fooladi, Comparative study of stochastic slope stability analysis based on conditional and unconditional random field, Comput. Geotech. 125 (2020) 103707.doi:10.1016/j.compgeo.2020.103707
-
[53]
Y. Cho, Y. Cao, Y. Zagayevskiy, T. Wong, et al., Kriging-based monitoring of reservoir gas saturation distribution using time-lapse multicomponent borehole gravity measurements: Case study, Hastings Field, J. Pet. Sci. Eng. 190 (2020).doi:10.1016/j.petrol.2020. 107054
-
[54]
M. Fischer, C. Proppe, Enhanced universal Kriging for transformed input parameter spaces, Probab. Eng. Mech. 74 (2023).doi:10.1016/ j.probengmech.2023.103486. Xuelin Xie et al.:Preprint submitted to Elsevier Page 26 of 26
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.