Analysis of gradual changes in nonparametric regression based on a new optimization method in the non-unique case
Pith reviewed 2026-07-01 01:23 UTC · model grok-4.3
The pith
A new optimization method enables consistent estimation of the largest minimizer for gradual change points in nonparametric regression even when the minimizer is not unique.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
We define and compare various consistent estimators based on a new general optimization method in the case where the aim is to estimate the largest minimization point of some objective function. We discuss rates of convergence and estimating the regression function based on the gradual change structure. Bootstrap bias approximation is discussed. Further applications in a two sample case are considered, where two continuous regression functions first equal and then change at some point of interest.
What carries the argument
A new general optimization method for estimating the largest minimization point of an objective function in the non-unique case.
If this is right
- Consistent estimators for the gradual change point can be constructed using the optimization method.
- Rates of convergence for these estimators can be analyzed.
- The regression function can be estimated incorporating the gradual change structure.
- Bootstrap methods can approximate bias in the estimators.
- The approach applies to two-sample problems with change points in regression functions.
Where Pith is reading between the lines
- This optimization method could potentially be adapted to other statistical estimation problems involving non-unique optima.
- Applications might extend to time series or spatial data with gradual transitions.
- Comparing the estimators in finite samples could reveal practical advantages of the new method.
- Integration with machine learning techniques for nonparametric regression might enhance performance.
Load-bearing premise
The regression function is continuous, starts at zero from the left, changes at an unknown point, and the optimization method consistently estimates the largest minimizer without needing uniqueness or specific details on the objective function.
What would settle it
A simulation study or real data example where the estimated change point fails to converge to the true value as sample size increases, despite the function starting at zero and being continuous.
Figures
read the original abstract
Consider a nonparametric regression model with one-dimensional covariates and a continuous regression function. Assume that the regression function from the left of the covariate support starts equal to zero and then changes at some unknown point. Our aim is to estimate this gradual change point. We define and compare various consistent estimators based on a new general optimization method in the case where the aim is to estimate the largest minimization point of some objective function. We discuss rates of convergence and estimating the regression function based on the gradual change structure. Bootstrap bias approximation is discussed. Further applications in a two sample case are considered, where two continuous regression functions first equal and then change at some point of interest.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper considers a nonparametric regression model with one-dimensional covariates and a continuous regression function that equals zero to the left of the covariate support before changing at an unknown point. It defines and compares various consistent estimators for this gradual change point using a new general optimization method for estimating the largest minimization point of an objective function in the non-unique case, discusses rates of convergence, estimation of the regression function exploiting the change structure, bootstrap bias approximation, and extensions to two-sample problems where two regression functions coincide before diverging at a point of interest.
Significance. If the proposed optimization method can be rigorously shown to yield consistent estimators for the largest minimizer under verifiable conditions, the work could offer a flexible framework for change-point estimation in nonparametric settings that handles non-uniqueness, with potential extensions to two-sample comparisons. The emphasis on rates, bootstrap, and structural exploitation of the zero-left-tail assumption adds practical value, but the significance is currently constrained by the lack of explicit regularity conditions supporting the central consistency claims.
major comments (3)
- [Definition of optimization method and consistency results] The central consistency claims for the estimators rest on the new optimization method targeting the largest minimizer in the non-unique case, yet no explicit conditions on the objective function, the structure of its minimizer set, or the resolution rule for non-uniqueness are stated to guarantee recovery of the largest point (see the section defining the general optimization method and the subsequent consistency theorems).
- [Rates of convergence and bootstrap bias] The rates of convergence and bootstrap bias approximation are asserted to follow from the method, but without the regularity conditions on the objective or the regression function beyond continuity and the left-zero assumption, it is unclear whether the claimed rates hold or reduce to tautological statements (see the sections on rates of convergence and bootstrap bias).
- [Two-sample case applications] The two-sample extension assumes the same optimization method applies when two continuous regression functions coincide before a change point, but again lacks specification of how non-uniqueness is handled to ensure the largest minimizer is recovered, undermining the claim that the method is general (see the section on further applications in the two-sample case).
minor comments (2)
- [Abstract and introduction] The abstract and introduction would benefit from a brief statement of the key regularity conditions required for the optimization method to achieve consistency.
- [Model setup] Notation for the objective function and the change point should be introduced more explicitly before the method is applied to the regression model.
Simulated Author's Rebuttal
We thank the referee for the careful reading and constructive comments on our manuscript. We address each major comment below, agreeing that additional explicit conditions are needed to support the claims, and will revise accordingly.
read point-by-point responses
-
Referee: [Definition of optimization method and consistency results] The central consistency claims for the estimators rest on the new optimization method targeting the largest minimizer in the non-unique case, yet no explicit conditions on the objective function, the structure of its minimizer set, or the resolution rule for non-uniqueness are stated to guarantee recovery of the largest point (see the section defining the general optimization method and the subsequent consistency theorems).
Authors: We agree that the manuscript would benefit from more explicit conditions. In the revised version, we will introduce a new subsection detailing the assumptions: the objective function is continuous, the minimizer set is a closed interval, and the method selects the rightmost point. These will be used to establish consistency in the theorems. revision: yes
-
Referee: [Rates of convergence and bootstrap bias] The rates of convergence and bootstrap bias approximation are asserted to follow from the method, but without the regularity conditions on the objective or the regression function beyond continuity and the left-zero assumption, it is unclear whether the claimed rates hold or reduce to tautological statements (see the sections on rates of convergence and bootstrap bias).
Authors: We agree that further regularity conditions are required. The revision will add assumptions including Lipschitz continuity of the regression function and quadratic growth of the objective away from the minimizer set. Under these, explicit rates will be derived and the bootstrap justification strengthened. revision: yes
-
Referee: [Two-sample case applications] The two-sample extension assumes the same optimization method applies when two continuous regression functions coincide before a change point, but again lacks specification of how non-uniqueness is handled to ensure the largest minimizer is recovered, undermining the claim that the method is general (see the section on further applications in the two-sample case).
Authors: The two-sample case applies the same general method. We will explicitly restate that the continuity and closed-interval minimizer conditions carry over to the two-sample objective, ensuring recovery of the largest minimizer and clarifying generality. revision: yes
Circularity Check
No circularity detected; derivation chain not exhibited in visible text
full rationale
The provided abstract and description define consistent estimators via a new optimization method targeting the largest minimizer in the non-unique case, with discussion of rates and bootstrap. No equations, explicit derivations, self-citations, or load-bearing premises are quoted that reduce any claim to its own inputs by construction. The reader's note confirms no equations are visible, precluding any specific reduction. Per hard rules, circularity requires quotable evidence of equivalence to inputs; absent that, the paper is treated as self-contained with independent content.
Axiom & Free-Parameter Ledger
Reference graph
Works this paper leans on
-
[1]
Akritas and Ingrid Van Keilegom
Michael G. Akritas and Ingrid Van Keilegom. Non-parametric estimation of the residual distribution. Scand. J. Statist., 28 0 (3): 0 549--567, 2001. ISSN 0303-6898,1467-9469. doi:10.1111/1467-9469.00254. URL https://doi.org/10.1111/1467-9469.00254
-
[2]
Limit properties of the monotone rearrangement for density and regression function estimation
Dragi Anevski and Anne-Laure Foug\`eres. Limit properties of the monotone rearrangement for density and regression function estimation. Bernoulli, 25 0 (1): 0 549--583, 2019. ISSN 1350-7265,1573-9759. doi:10.3150/17-bej998. URL https://doi.org/10.3150/17-bej998
-
[3]
Change-point problem and bootstrap
Jarom\'ir Antoch and Marie Hu s kov\' a . Change-point problem and bootstrap. J. Nonparametr. Statist., 5 0 (2): 0 123--144, 1995. ISSN 1048-5252,1029-0311. doi:10.1080/10485259508832639. URL https://doi.org/10.1080/10485259508832639
-
[4]
Tests for continuity of regression functions
Jarom\'ir Antoch, Gerard Gregoire, and Marie Hu s kov\' a . Tests for continuity of regression functions. J. Statist. Plann. Inference, 137 0 (3): 0 753--777, 2007. ISSN 0378-3758,1873-1171. doi:10.1016/j.jspi.2006.06.007. URL https://doi.org/10.1016/j.jspi.2006.06.007
-
[5]
Gradual changes in functional time series
Patrick Bastian and Holger Dette. Gradual changes in functional time series. J. Time Ser. Anal., 2025. doi:10.1111/jtsa.12809. URL https://doi.org/10.1111/jtsa.12809
-
[6]
Estimation of semiparametric models when the criterion function is not smooth
Xiaohong Chen, Oliver Linton, and Ingrid Van Keilegom. Estimation of semiparametric models when the criterion function is not smooth. Econometrica, 71 0 (5): 0 1591--1608, 2003. ISSN 0012-9682,1468-0262. doi:10.1111/1468-0262.00461. URL https://doi.org/10.1111/1468-0262.00461
-
[7]
Holger Dette, Natalie Neumeyer, and Kay F. Pilz. A simple nonparametric estimator of a strictly monotone regression function. Bernoulli, 12 0 (3): 0 469--490, 2006. ISSN 1350-7265,1573-9759. doi:10.3150/bj/1151525131. URL https://doi.org/10.3150/bj/1151525131
-
[8]
Asymmetric cusp estimation in regression models
Maik D\"oring. Asymmetric cusp estimation in regression models. Statistics, 49 0 (6): 0 1279--1297, 2015 a . ISSN 0233-1888,1029-4910. doi:10.1080/02331888.2015.1016027. URL https://doi.org/10.1080/02331888.2015.1016027
-
[9]
Rate of convergence of a change point estimator in a misspecified regression model
Maik D\"oring. Rate of convergence of a change point estimator in a misspecified regression model. Stochastic Models, Statistics and Their Applications: Wroclaw, Poland, Springer, pages 49--55, 2015 b
2015
-
[10]
Smooth change point estimation in regression models with random design
Maik D\"oring and Uwe Jensen. Smooth change point estimation in regression models with random design. Ann. Inst. Statist. Math., 67 0 (3): 0 595--619, 2015. ISSN 0020-3157,1572-9052. doi:10.1007/s10463-014-0467-8. URL https://doi.org/10.1007/s10463-014-0467-8
-
[11]
Dietmar Ferger. A continuous mapping theorem for the argmin-set functional with applications to convex stochastic processes. Kybernetika (Prague), 57 0 (3): 0 426--445, 2021. ISSN 0023-5954,1805-949X. doi:10.14736/kyb-2021-3-0426. URL https://doi.org/10.14736/kyb-2021-3-0426
-
[12]
Dietmar Ferger. On semi-continuity and continuity of the smallest and largest minimizing point of real convex functions with applications in probability and statistics. Eur. J. Math., 10 0 (1): 0 Paper No. 14, 18, 2024. ISSN 2199-675X,2199-6768. doi:10.1007/s40879-024-00728-2. URL https://doi.org/10.1007/s40879-024-00728-2
-
[13]
Nonparametric Functional Data Analysis
Frederic Ferraty and Philippe Vieu. Nonparametric Functional Data Analysis. Springer Series in Statistics. Springer, New York, 2006. ISBN 978-0387-30369-7
2006
-
[14]
Nonparametric regression analysis of growth curves
Theo Gasser, Hans-Georg M\" u ller, Walter Kohler, Luciano Molinari, and Andrea Prader. Nonparametric regression analysis of growth curves. Ann. Statist., 12 0 (1): 0 210--229, 1984. URL https://doi.org/10.1214/aos/1176346402
-
[15]
Data-driven discontinuity detection in derivatives of a regression function
Irene Gijbels and Anne-C\' e cile Goderniaux. Data-driven discontinuity detection in derivatives of a regression function. Comm. Statist. Theory Methods, 33 0 (4): 0 851--871, 2004. ISSN 0361-0926,1532-415X. doi:10.1081/STA-120028730. URL https://doi.org/10.1081/STA-120028730
-
[16]
Piet Groeneboom and Geurt Jongbloed. Nonparametric estimation under shape constraints, volume 38 of Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, New York, 2014. ISBN 978-0-521-86401-5. doi:10.1017/CBO9781139020893. URL https://doi.org/10.1017/CBO9781139020893. Estimators, algorithms and asymptotics
-
[17]
Bruce E. Hansen. Uniform convergence rates for kernel estimation with dependent data. Econometric Theory, 24 0 (3): 0 726--748, 2008. ISSN 0266-4666,1469-4360. doi:10.1017/S0266466608080304. URL https://doi.org/10.1017/S0266466608080304
-
[18]
Inequalities
Godfrey Harold Hardy, John Edensor Littlewood, and George P\'olya. Inequalities. Cambridge Mathematical Library. Cambridge University Press, Cambridge, 1988. ISBN 0-521-35880-9. Reprint of the 1952 edition
1988
-
[19]
Two-sample gradual change analysis
Zden e k Hl\' a vka and Marie Hu s kov\' a . Two-sample gradual change analysis. REVSTAT, 15 0 (3): 0 355--372, 2017. ISSN 1645-6726,2183-0371
2017
-
[20]
Kernel type estimators for gradual changes
Lajos Horv\' a th and Marie Hu s kov\' a . Kernel type estimators for gradual changes. In Limit theorems in probability and statistics, V ol. II ( B alatonlelle, 1999) , pages 175--187. J\'anos Bolyai Math. Soc., Budapest, 2002. ISBN 963-9453-013
1999
-
[21]
Estimators in the location model with gradual changes
Marie Hu s kov\' a . Estimators in the location model with gradual changes. Comment. Math. Univ. Carolin., 39 0 (1): 0 147--157, 1998. ISSN 0010-2628,1213-7243
1998
-
[22]
Limit theorems for a class of tests of gradual changes
Marie Hu s kov\' a and Josef Steinebach. Limit theorems for a class of tests of gradual changes. J. Statist. Plann. Inference, 89 0 (1-2): 0 57--77, 2000. ISSN 0378-3758,1873-1171. doi:10.1016/S0378-3758(00)00094-X. URL https://doi.org/10.1016/S0378-3758(00)00094-X
-
[23]
Asymptotic tests for gradual changes
Marie Hu s kov\' a and Josef Steinebach. Asymptotic tests for gradual changes. Statist. Decisions, 20 0 (2): 0 137--151, 2002. ISSN 0721-2631
2002
-
[24]
Dimitrios A. Ioannides. Consistent nonparametric regression: some generalizations in the fixed design case. J. Nonparametr. Statist., 2 0 (3): 0 203--213, 1993. ISSN 1048-5252,1029-0311. doi:10.1080/10485259308832553. URL https://doi.org/10.1080/10485259308832553
-
[25]
Testing appearance of polynomial trend
Daniela Jaru s kov\' a . Testing appearance of polynomial trend. Extremes, 2 0 (1): 0 25--37, 1999. ISSN 1386-1999,1572-915X. doi:10.1023/A:1009951807623. URL https://doi.org/10.1023/A:1009951807623
-
[26]
Fully functional weighted testing for abrupt and gradual location changes in functional time series
Claudia Kirch, Hedvika Rano s ov\' a , and Martin Wendler. Fully functional weighted testing for abrupt and gradual location changes in functional time series. ArXiv preprint arXiv: 2601.16058v1, 2026+
-
[27]
Michael R. Kosorok. Introduction to empirical processes and semiparametric inference. Springer Series in Statistics. Springer, New York, 2008. ISBN 978-0-387-74977-8. doi:10.1007/978-0-387-74978-5. URL https://doi.org/10.1007/978-0-387-74978-5
-
[28]
A note on uniform consistency of monotone function estimators
Natalie Neumeyer. A note on uniform consistency of monotone function estimators. Statist. Probab. Lett., 77 0 (7): 0 693--703, 2007. ISSN 0167-7152,1879-2103. doi:10.1016/j.spl.2006.11.004. URL https://doi.org/10.1016/j.spl.2006.11.004
-
[29]
Simulation and the asymptotics of optimization estimators
Ari\' e l Pakes and David Pollard. Simulation and the asymptotics of optimization estimators. Econometrica, 57 0 (5): 0 1027--1057, 1989. ISSN 0012-9682,1468-0262. doi:10.2307/1913622. URL https://doi.org/10.2307/1913622
-
[30]
Minimax estimation of sharp change points
Marc Raimondo. Minimax estimation of sharp change points. Ann. Statist., 26 0 (4): 0 1379--1397, 1998. ISSN 0090-5364,2168-8966. doi:10.1214/aos/1024691247. URL https://doi.org/10.1214/aos/1024691247
-
[31]
Jean Michel Rakotoson. Relative rearrangement. Lecture Notes in Mathematics. Springer Nature Switzerland, 2025. doi:10.1007/978-3-032-02228-8. URL https://doi.org/10.1007/978-3-032-02228-8
-
[32]
Designs for regression problems with correlated errors
Jerome Sacks and Donald Ylvisaker. Designs for regression problems with correlated errors. III . Ann. Math. Statist., 41: 0 2057--2074, 1970. ISSN 0003-4851. doi:10.1214/aoms/1177696705. URL https://doi.org/10.1214/aoms/1177696705
-
[33]
Limit theorems for rank statistics detecting gradual changes
Ale s Slab\' y . Limit theorems for rank statistics detecting gradual changes. Comment. Math. Univ. Carolin., 42 0 (3): 0 591--600, 2001. ISSN 0010-2628,1213-7243
2001
- [34]
-
[35]
Alexandre B. Tsybakov. Introduction to nonparametric estimation. Springer Series in Statistics. Springer, New York, 2009. ISBN 978-0-387-79051-0. doi:10.1007/b13794. URL https://doi.org/10.1007/b13794. Revised and extended from the 2004 French original, Translated by Vladimir Zaiats
-
[36]
Aad W. van der Vaart. Asymptotic statistics, volume 3 of Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, Cambridge, 1998. ISBN 0-521-49603-9; 0-521-78450-6. doi:10.1017/CBO9780511802256. URL https://doi.org/10.1017/CBO9780511802256
-
[37]
Aad W. van der Vaart and Jon A. Wellner. Weak convergence and empirical processes. Springer Series in Statistics. Springer, New York, 1996. ISBN 0-387-94640-3. doi:10.1007/978-1-4757-2545-2. URL https://doi.org/10.1007/978-1-4757-2545-2. With applications to statistics
-
[38]
Detecting gradual changes in locally stationary processes
Michael Vogt and Holger Dette. Detecting gradual changes in locally stationary processes. Ann. Statist., 43 0 (2): 0 713--740, 2015. ISSN 0090-5364,2168-8966. doi:10.1214/14-AOS1297. URL https://doi.org/10.1214/14-AOS1297
-
[39]
Matyskova, Zdenek Hl\' a vka, L
Zden e k S umn\' i k, J. Matyskova, Zdenek Hl\' a vka, L. Durdilova, O. Sou c ek, and Dana Zemkov\' a . Reference data for jumping mechanography in healthy children and adolescents aged 6-18 years. Journal of musculoskeletal & neuronal interactions, 13: 0 259--73, 09 2013
2013
-
[40]
Gradual changes in long memory processes with applications
Lihong Wang. Gradual changes in long memory processes with applications. Statistics, 41 0 (3): 0 221--240, 2007. ISSN 0233-1888,1029-4910. doi:10.1080/02331880701223555. URL https://doi.org/10.1080/02331880701223555
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.