REVIEW 3 major objections 5 minor 27 references
Early Stopping Bayesian Optimization for Controller Tuning
T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read Stopping bad controller trials early can cut Bayesian optimization time by up to 48 percent in simulation and 35 percent on hardware, without sacrificing final controller performance.
desk verdict A clean, practical BO-for-control paper that makes a genuinely new move (early stopping inside the optimizer) and backs it with code and hardware, but its noise-blind stopping rule means the 'no quality loss' claim is only empirical, not guaranteed. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is the stopping rule $S(j_{1,k}, \ldots, j_{t,k}, J^*_k)$, which returns true as soon as the cumulative cost $\sum_{\tau=1}^{t} j_{\tau,k}$ reaches the current best observed cost $J^*_k$; when this happens the episode is aborted. Because the partial sum underestimates the true cost, ESBO builds virtual data points: ESBO-C assigns pessimistic Gaussian-process predictions capped by crash-constraint bounds, ESBO-TR reinterprets fitness as the negative time at which cumulative cost would cross $J^*_k$, and ESBO-GP trains separate Gaussian processes on the unobserved time sections and samples a probabilistic completion of the partial episode. The virtual data points guide the acquisition function away from regions whose episodes were stopped early, while preserving the higher-fidelity information of fully completed episodes.
What would settle it
A decisive test would run ESBO-GP and vanilla BO on a noisy benchmark with a known optimum and high observation noise, tracking final regret after a fixed budget; if ESBO-GP's final regret is consistently worse than vanilla BO's when the initial samples contain an unusually low $J^*_k$, the threshold rule is the cause. A second check compares the two on a problem whose cost is concentrated late in the episode, where early stopping should save little time and could remove useful information.
Extended reading notes
Core claim
For objectives that are sums of nonnegative per-time-step costs, the paper claims that the standard fixed-episode evaluation in Bayesian optimization can be replaced by an early-stopping rule without corrupting the surrogate model. The rule stops an episode at the first time $T_k$ where $\sum_{\tau=1}^{T_k} j_{\tau,k} \ge J^*_k$, the current best observed cost. Because the resulting partial sum underestimates the true cost, the paper constructs virtual observations: a pessimistic crash-style completion, a time-reformulated fitness, or a probabilistic completion from Gaussian processes trained on the missing time sections. In the experiments, the probabilistic completion (ESBO-GP) gives the largest time savings and a final solution comparable to vanilla BO.
Load-bearing premise
The load-bearing premise is that the current best observed cost $J^*_k$ is a reliable threshold for declaring a trial hopeless, even though it is a noisy finite-sample estimate; if an unlucky low observation makes the threshold too aggressive, the optimizer can be steered away from good parameters.
Editorial extensions
If this is right
- Practitioners tuning controllers with Bayesian optimization can switch to ESBO-GP and expect the optimization campaign to finish in roughly half the simulation time, with final controller quality comparable to vanilla BO.
- The query efficiency of ESBO stays close to vanilla BO, meaning the time savings come from shorter episodes rather than from needing fewer parameter evaluations.
- Random search with the same early-stopping rule becomes more competitive than without stopping, but still underperforms the Bayesian optimization variants, so the speedup is not merely an artifact of the stopping rule.
- The three heuristics provide a baseline set for future work on partial observations in Bayesian optimization, with methods that use all available partial data (ESBO-GP) tending to perform best.
Reading between the lines
- The early-stopping threshold $J^*_k$ is a noisy estimate, so under high observation noise the rule could systematically avoid regions whose early samples were unlucky; a natural extension is a threshold that accounts for uncertainty in $J^*_k$ rather than using the raw best observation.
- The same pattern should transfer to any black-box optimization whose objective is a time-integral of nonnegative costs, not just control: simulation-based design or hyperparameter tuning where trials can be cut short could benefit from the same early-stopping machinery.
- A risk-aware variant could stop even earlier by predicting the probability that the remaining cost can still beat $J^*_k$, using the section-specific Gaussian processes; the paper notes such a probabilistic early stop as a future possibility.
- The hardware result at 35 percent time saving suggests real-world applicability, but the noise-free simulation savings may shrink as noise grows; testing on a noisy benchmark with a known optimum would quantify the bias from the noisy threshold.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Early Stopping Bayesian Optimization (ESBO) for controller tuning with time-integrated costs. It introduces a stopping rule (Eq. 12) that aborts an episode once the cumulative cost reaches the current best observed cost, and three heuristics (ESBO-C, ESBO-TR, ESBO-GP) to incorporate the resulting partial observations into a GP surrogate. The method is evaluated on five simulation tasks and one three-tank hardware experiment, reporting time savings up to 48% in simulation and 35% in hardware relative to vanilla BO while reaching comparable final performance.
Significance. If the empirical claims hold, ESBO offers a practically useful way to reduce experimentation time in controller tuning, with a simple, parameter-free stopping rule and three heuristics for handling partial observations. The paper contributes a new problem formulation, a public implementation, and a benchmark suite spanning five simulation tasks and a hardware testbed. Strengths include the clear hyperparameter settings, the reproducibility of the code, and the comparison against standard baselines. The main limitations are the lack of any theoretical or systematic analysis of the stopping rule's behavior under observation noise and the limited statistical power of the empirical evaluation; the central claim is plausible but not yet fully established.
major comments (3)
- [Sec. V-A, Eq. (12)] The stopping rule uses the current best observed cost J*_k as a threshold, but J*_k is a noisy finite-sample estimate (Eq. (4)). A single unlucky low-noise observation can set J*_k below the true cost of the global optimum; thereafter the rule aborts any trial whose cumulative cost crosses that artificially low value, even if the full-episode cost would have beaten the incumbent. Because stopped trials enter the surrogate only via virtual data (Sec. V-B), the search is systematically steered away from regions that could contain the optimum. The paper does not analyze this bias or provide a noise-robust modification, and the single hardware noise level (Sec. VI-D) cannot certify the general claim in Sec. VII that ESBO works 'without sacrificing final solution quality.' Please add a synthetic benchmark that sweeps the observation-noise level or a theoretical analysis of the stopping rule's safety.
- [Sec. VI-C, Fig. 2] The simulation results are reported only as median curves without error bars, confidence bands, or significance tests. The headline 48% speedup is a point estimate based on the median final performance of ESRS, and with 10 seeds per task the observed differences may be within noise. Please report per-task results with interquartile ranges or bootstrap intervals, and make explicit how many seeds are used for each of the five tasks.
- [Sec. VI-D, Fig. 4] The hardware experiment shows a statistically significant advantage only in a transient interval (around 15-26 min); after about 30 minutes the rank-sum test is insignificant. This weakens the 'up to 35%' speedup claim, since the time-to-target comparison is based on a single noise realization and no correction for multiple comparisons is applied. Please report the distribution of times-to-target across the eight runs and apply a proper multiple-testing correction or a time-to-target summary.
minor comments (5)
- [Sec. VI-A] The sentence 'The simulation and hardware experiments aim at determinining the of-the-box-performance' contains typos; it should read 'determining the out-of-the-box performance.'
- [Sec. VI-C] In the sentence 'The simulation time needed to achieve the median final performance of ESRS is reduced by 48% from 62% to 32% percent', the word 'percent' is redundant, and the baseline 'median final performance of ESRS' should be defined explicitly as a target value.
- [Sec. V-B, Eq. (17)] The index p is used before it is defined; please specify that p denotes the section containing T_l.
- [Algorithm 1, line 6] The phrase 'identify current optimum from Dk' is ambiguous in noisy settings; it should be 'identify the current best observed cost from Dk' to distinguish it from the true optimum J*.
- [Sec. V-B, ESBO-C] The crash-constraint formula in Eq. (13) is imported from reference [13] with only a brief explanation; please provide a derivation or cite a peer-reviewed source so that the min/max bounds are self-contained.
Circularity Check
No significant circularity: the early-stopping rule is a valid sufficient condition, and the empirical claims are benchmarked against external baselines.
full rationale
The central derivation chain is self-contained and does not reduce to its inputs. Equation (12) stops an episode when the cumulative non-negative cost reaches the current best observed cost. Because the full-episode cost is the sum of non-negative per-step costs, reaching the current best before the end implies the full cost cannot be smaller than the current best. This is a mathematical sufficient condition, not a definitional or fitted relation, so it is not circular. The ESBO heuristics for handling partial observations are explicitly heuristic and are evaluated against vanilla BO, random search, and early-stopping random search on five simulation tasks and one hardware experiment; the final performance comparison is against actual full-episode outcomes, not against quantities derived from the proposed method's own virtual data. The paper does cite the authors' prior work, notably [13] for crash-constraint handling and [5] for benchmarks and metrics. These self-citations are not load-bearing for the central early-stopping claim: the crash-constraint formula is a borrowed auxiliary mechanism, and the benchmarks are external test problems and baselines. The reported speedup is an empirical observation about the proposed algorithm, not a prediction derived from a fitted parameter. The noise sensitivity of the threshold J*_k in Eq. (12) is a legitimate robustness concern, but it is a correctness risk rather than a circularity: the rule is not defined in terms of the outcomes it is claimed to predict. No uniqueness theorem is imported from the authors' prior work, and no known result is merely renamed. Overall, the derivation and evaluation are independent of the conclusions in a way that does not exhibit circular reasoning.
Assumptions & free parameters
free parameters (2)
- GP length-scale bounds =
correlation 0.1 at between half domain and 1% of domain
- ESBO-C pessimism multiplier =
3
assumptions (5)
- domain assumption Objective is a sum of non-negative per-step costs (Eq. 2).
- standard math Gaussian process surrogate with zero mean and anisotropic squared-exponential kernel.
- domain assumption Observation noise is additive, iid Gaussian (Eq. 4).
- ad hoc to paper The per-segment cost in ESBO-GP can be modeled by independent GPs across time sections.
- domain assumption Early stopping rule compares cumulative cost to the current best observation J*_k, treated as a reliable threshold.
Cite this review
Pith. "Pith review of Early Stopping Bayesian Optimization for Controller Tuning." pith.science (2026). https://pith.science/paper/BKJDE2JQ
@misc{pith2026250111532,
author = {Pith},
title = {Pith review of: Early Stopping Bayesian Optimization for Controller Tuning},
year = {2026},
howpublished = {\url{https://pith.science/paper/BKJDE2JQ}},
note = {Machine review of arXiv:2501.11532}
}
read the original abstract
Manual tuning of performance-critical controller parameters can be tedious and sub-optimal. Bayesian Optimization (BO) is an increasingly popular practical alternative to automatically optimize controller parameters from few experiments. Standard BO practice is to evaluate the closed-loop performance of parameters proposed during optimization on an episode with a fixed length. However, fixed-length episodes can be wasteful. For example, continuing an episode where already the start shows undesirable behavior such as strong oscillations seems pointless. Therefore, we propose a BO method that stops an episode early if suboptimality becomes apparent before an episode is completed. Such early stopping results in partial observations of the controller's performance, which cannot directly be included in standard BO. We propose three heuristics to facilitate partially observed episodes in BO. Through five numerical and one hardware experiment, we demonstrate that early stopping BO can substantially reduce the time needed for optimization.
Figures
Reference graph
Works this paper leans on
-
[1]
Garnett, Bayesian Optimization
R. Garnett, Bayesian Optimization . Cambridge University Press, 2023
work page 2023
-
[2]
Automatic LQR tuning based on Gaussian process global optimization,
A. Marco, P. Hennig, J. Bohg, S. Schaal, and S. Trimpe, “Automatic LQR tuning based on Gaussian process global optimization,” in 2016 IEEE International Conference on Robotics and Automation (ICRA), IEEE, 2016, pp. 270–277
work page 2016
-
[3]
G. Savaia, Y . Sohn, S. Formentin, G. Panzani, M. Corno, and S. M. Savaresi, “Experimental automatic calibration of a semi- active suspension controller via bayesian optimization,” Control Engineering Practice, vol. 112, p. 104 826, 2021
work page 2021
-
[4]
Contextual tuning of model predictive control for autonomous racing,
L. P. Fr ¨ohlich, C. K ¨uttel, E. Arcari, L. Hewing, M. N. Zeilinger, and A. Carron, “Contextual tuning of model predictive control for autonomous racing,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , 2022, pp. 10 555–10 562
work page 2022
-
[5]
D. Stenger and D. Abel, Benchmark of bayesian optimization and metaheuristics for control engineering tuning problems with crash constraints, 2022. [Online]. Available: https://arxiv.org/ abs/2211.02571
arXiv 2022
-
[6]
Bayesian optimization of soft exosuits using a metabolic estimator stopping process,
M. Kim, C. Liu, J. Kim, S. Lee, A. Meguid, C. J. Walsh, and S. Kuindersma, “Bayesian optimization of soft exosuits using a metabolic estimator stopping process,” in 2019 International Con- ference on Robotics and Automation (ICRA) , 2019, pp. 9173–9179
work page 2019
-
[7]
A. Chakrabarty, D. J. Burns, M. Guay, and C. R. Laughman, “Extremum seeking controller tuning for heat pump optimization using failure-robust bayesian optimization,” Journal of Process Control, vol. 120, pp. 86–96, 2022
work page 2022
-
[8]
Taking the Human Out of the Loop: A Review of Bayesian Opti- mization,
B. Shahriari, K. Swersky, Z. Wang, R. P. Adams, and N. de Freitas, “Taking the Human Out of the Loop: A Review of Bayesian Opti- mization,” Proceedings of the IEEE , vol. 104, no. 1, pp. 148–175, 2016
work page 2016
Show all 27 references
-
[9]
Automated reinforcement learning (autorl): A survey and open problems,
J. Parker-Holder, R. Rajan, X. Song, A. Biedenkapp, Y . Miao, T. Eimer, B. Zhang, V . Nguyen, R. Calandra, A. Faust, et al. , “Automated reinforcement learning (autorl): A survey and open problems,” Journal of Artificial Intelligence Research , vol. 74, pp. 517–568, 2022
2022
-
[10]
A survey on policy search algorithms for learning robot controllers in a handful of trials,
K. Chatzilygeroudis, V . Vassiliades, F. Stulp, S. Calinon, and J.-B. Mouret, “A survey on policy search algorithms for learning robot controllers in a handful of trials,” IEEE Transactions on Robotics , vol. 36, no. 2, pp. 328–347, 2020
2020
-
[11]
Automated tuning of nonlinear kalman filters for optimal trajectory tracking performance of auvs,
M. Nitsch, D. Stenger, and D. Abel, “Automated tuning of nonlinear kalman filters for optimal trajectory tracking performance of auvs,” IFAC-PapersOnLine, vol. 56, no. 2, pp. 11 608–11 614, 2023, 22nd IFAC World Congress
2023
-
[12]
Min-max hyperparam- eter tuning, with application to fault detection,
J. Marzat, H. Piet-Lahanier, and E. Walter, “Min-max hyperparam- eter tuning, with application to fault detection,” 18th IFAC World Congress, 2011
2011
-
[13]
Automatic tuning of control engineering algo- rithms with Bayesian optimization,
D. Stenger, “Automatic tuning of control engineering algo- rithms with Bayesian optimization,” Dissertation, Rheinisch- Westf¨alische Technische Hochschule Aachen, 2023. [Online]. Avail- able: https : / / publications . rwth - aachen . de / record/971795
2023
-
[14]
Safe contextual Bayesian optimization for sustainable room tem- perature PID control tuning,
M. Fiducioso, S. Curi, B. Schumacher, M. Gwerder, and A. Krause, “Safe contextual Bayesian optimization for sustainable room tem- perature PID control tuning,” in Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, (IJCAI-19), Jul. 2019,...
2019
-
[15]
Performance-oriented model learning for control via multi- objective Bayesian optimization,
G. Makrygiorgos, A. D. Bonzanini, V . Miller, and A. Mes- bah, “Performance-oriented model learning for control via multi- objective Bayesian optimization,”Computers & Chemical Engineer- ing, vol. 162, p. 107 770, 2022
2022
-
[16]
Safety-aware cascade controller tuning using con- strained Bayesian optimization,
M. Khosravi, C. K ¨onig, M. Maier, R. S. Smith, J. Lygeros, and A. Rupenyan, “Safety-aware cascade controller tuning using con- strained Bayesian optimization,” IEEE Transactions on Industrial Electronics, vol. 70, no. 2, pp. 2128–2138, 2023
2023
-
[17]
On controller tuning with time-varying bayesian optimization,
P. Brunzema, A. V on Rohr, and S. Trimpe, “On controller tuning with time-varying bayesian optimization,” in 2022 IEEE 61st Con- ference on Decision and Control (CDC) , 2022, pp. 4046–4052
2022
-
[18]
Virtual vs. real: Trading off simulations and physical experiments in reinforcement learning with bayesian optimization,
A. Marco, F. Berkenkamp, P. Hennig, A. P. Schoellig, A. Krause, S. Schaal, and S. Trimpe, “Virtual vs. real: Trading off simulations and physical experiments in reinforcement learning with bayesian optimization,” in 2017 IEEE International Conference on Robotics and Automation...
2017
-
[19]
Local bayesian optimization for controller tuning with crash constraints,
A. von Rohr, D. Stenger, D. Scheurenberg, and S. Trimpe, “Local bayesian optimization for controller tuning with crash constraints,” at–Automatisierungstechnik, vol. 72, no. 4, pp. 281–292, 2024
2024
-
[20]
Bayesian optimiza- tion with safety constraints: Safe and automatic parameter tuning in robotics,
F. Berkenkamp, A. Krause, and A. P. Schoellig, “Bayesian optimiza- tion with safety constraints: Safe and automatic parameter tuning in robotics,” Machine Learning, vol. 112, pp. 3713–3747, 2021
2021
-
[21]
Robot learning with crash constraints,
A. Marco, D. Baumann, M. Khadiv, P. Hennig, L. Righetti, and S. Trimpe, “Robot learning with crash constraints,” IEEE Robotics and Automation Letters , vol. 6, no. 2, pp. 1439–1446, 2021
2021
-
[22]
Swersky, J
K. Swersky, J. Snoek, and R. P. Adams, Freeze-thaw bayesian optimization, 2014. [Online]. Available: https://arxiv.org/ abs/1406.3896
2014 arXiv
-
[23]
Bayesian optimization meets Bayesian optimal stopping,
Z. Dai, H. Yu, B. K. H. Low, and P. Jaillet, “Bayesian optimization meets Bayesian optimal stopping,” in Proceedings of the 36th International Conference on Machine Learning (ICML) , PMLR, 2019, pp. 1496–1506
2019
-
[24]
Bayesian optimization with expensive integrands,
S. Toscano-Palmerin and P. I. Frazier, “Bayesian optimization with expensive integrands,” SIAM Journal on Optimization , vol. 32, no. 2, pp. 417–444, 2022
2022
-
[25]
Gaussian Processes for Machine Learning,
C. E. Rasmussen and Williams C. K. I., “Gaussian Processes for Machine Learning,” 2006
2006
-
[26]
Max-value entropy search for efficient bayesian optimization,
Z. Wang and S. Jegelka, “Max-value entropy search for efficient bayesian optimization,” in International Conference on Machine Learning, PMLR, 2017, pp. 3627–3635
2017
-
[27]
Evaluation of data enhanced model predictive control for a coupled tank system,
D. Scheurenberg, S. Stemmler, and D. Abel, “Evaluation of data enhanced model predictive control for a coupled tank system,” in 2023 IEEE Conference on Control Technology and Applications (CCTA), 2023, pp. 79–84
2023
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.